Remove Duplicates#
Use the Remove Duplicates node to identify and delete items that are identical across all fields or a subset of fields. This is helpful in situations where you can end up with duplicate data, such as a user creating multiple accounts, or a customer submitting the same order multiple times. When working with large datasets it becomes more difficult to spot and remove these items.
Node parameters#
- Compare: Select which fields of the input data n8n should compare to check if they're the same. The following options are available:
- All Fields: Compares all fields of the input data.
- All Fields Except: Enter which input data fields n8n should exclude from the comparison. You can provide multiple values separated by commas.
- Selected Fields: Enter which input data fields n8n should include in the comparison. You can provide multiple values separated by commas.
Node options#
If you choose All Fields Except or Selected Fields as your compare type, you can add these options:
- Disable Dot Notation: Set whether to use dot notation to reference child fields in the format
parent.child(turned off) or not (turn on). - Remove Other Fields: Set whether to remove any fields that aren't used in the comparison (turned on) or not (turned off).
Templates and examples#
Related resources#
Learn more about data structure and data flow in n8n workflows.