Remove Duplicates#
Use the Remove Duplicates node to identify and delete items that are identical across all fields or a subset of fields. This is helpful in situations where you can end up with duplicate data, such as a user creating multiple accounts, or a customer submitting the same order multiple times. When working with large datasets it becomes more difficult to spot and remove these items.
Examples and templates
For usage examples and templates to help you get started, refer to n8n's Remove Duplicates integrations page.
Node parameters#
- Compare: specify which fields of the input data n8n should compare to check if they're the same. The following options are available:
- All Fields: compares all fields of the input data.
- All Fields Except: enter which input data fields n8n should exclude from the comparison. You can provide multiple values separated by commas.
- Selected Fields: enter which input data fields n8n should include in the comparison. You can provide multiple values separated by commas.
- If you choose All Fields Except or Selected Fields, n8n displays Options > Add Field. Use this to add more optional settings, including:
- Disable Dot Notation: when disabled, you can't reference child fields (in the format
parent.child). - Remove Other Fields: keep the fields that you're comparing and remove the others.
- Disable Dot Notation: when disabled, you can't reference child fields (in the format
Related resources#
View example workflows and related content on n8n's website.
Learn more about data structure and data flow in n8n workflows.