> ## Documentation Index
> Fetch the complete documentation index at: https://docs.bondata.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Find Duplicates

> Detect duplicate records using fuzzy matching and configurable thresholds

Detects duplicate records using fuzzy matching and configurable thresholds. Outputs two paths: **All Records** and **Deduplicated** - so you can handle duplicates and clean records differently.

## Configuration tab

<Frame caption="Find Duplicates - preset selection, thresholds, and field comparison">
  <img src="https://mintcdn.com/bondata/DlMd45I1zc1MvbNc/images/guide/nodes/enrichment/find-duplicates-config.png?fit=max&auto=format&n=DlMd45I1zc1MvbNc&q=85&s=b52c9ade067a4b4984982f8078931db3" width="805" height="841" data-path="images/guide/nodes/enrichment/find-duplicates-config.png" />
</Frame>

| Setting                  | Description                                                                                               |
| ------------------------ | --------------------------------------------------------------------------------------------------------- |
| **Preset Configuration** | Choose a starting point - **Conservative** (95%), **Balanced** (92%), **Aggressive** (88%), or **Custom** |
| **Auto-merge threshold** | Records scoring above this are merged automatically                                                       |
| **Review threshold**     | Records between review and auto-merge thresholds need manual review                                       |
| **Fields to Compare**    | Select fields and set comparison type (**Fuzzy** or **Exact**) and weight                                 |

## Rules & Performance tab

<Frame caption="Find Duplicates - blocking keys and match rules">
  <img src="https://mintcdn.com/bondata/DlMd45I1zc1MvbNc/images/guide/nodes/enrichment/find-duplicates-rules.png?fit=max&auto=format&n=DlMd45I1zc1MvbNc&q=85&s=3481e4bf9b62526b1db912871a845943" width="832" height="626" data-path="images/guide/nodes/enrichment/find-duplicates-rules.png" />
</Frame>

| Setting                  | Description                                                                                                |
| ------------------------ | ---------------------------------------------------------------------------------------------------------- |
| **Blocking Keys**        | Only compare records that share a blocking key value - dramatically improves performance on large datasets |
| **Must-Match Rules**     | Records must match on these fields to be considered duplicates                                             |
| **Must-Not-Match Rules** | Records matching on these fields are never considered duplicates                                           |

## Advanced tab

<Frame caption="Find Duplicates - normalization options and safety rails">
  <img src="https://mintcdn.com/bondata/DlMd45I1zc1MvbNc/images/guide/nodes/enrichment/find-duplicates-advanced.png?fit=max&auto=format&n=DlMd45I1zc1MvbNc&q=85&s=2c97ae4009c1744c532131b3870b9e30" width="825" height="723" data-path="images/guide/nodes/enrichment/find-duplicates-advanced.png" />
</Frame>

**Normalization Options** - applied before comparison:

* Trim & lowercase
* Remove punctuation
* Unicode normalize
* Ignore company suffixes (e.g., "Inc", "LLC")
* Phone normalize (E.164)
* Address normalize

**Safety Rails** - prevent unintended mass merges:

| Setting              | Description                                           |
| -------------------- | ----------------------------------------------------- |
| **Dry run**          | Simulate without actually merging                     |
| **Max cluster size** | Prevents one bad key from merging hundreds of records |
| **Max total merges** | Stops after N merges for gradual rollout              |

## Output

Two output paths:

* **All Records** - every record with duplicate scores attached
* **Deduplicated** - clean dataset with duplicates removed

## Best Practices

* Start with the **Balanced** preset and adjust thresholds based on results
* Use **Blocking Keys** for large datasets - comparing every record pair is expensive
* Enable **Dry run** first to preview results before committing merges
* Set **Max total merges** for gradual rollout on critical data

## Related Nodes

* **[Bond Node](/guide/nodes/combine/bond-node)** - matches records across different entities, not within the same dataset
* **[Data Normalization](/guide/nodes/transform/data-normalization-node)** - clean field values before duplicate detection for better accuracy