Keeping False Duplicates from Reappearing

 

You may find that you have to regularly re-examine your database for duplicates.  This is necessary under the following conditions:

• When normal business activities cause an ongoing buildup of duplicate records.
• If you have different systems feeding new records into a common database, resulting in ongoing creation of duplicate records.
• If you import external data sources into your database on a regular basis, resulting in duplicate records. 

In these cases you will need to run the Setup and Examine tool on a reoccurring basis each time you want to de-duplicate your data.

 

However, with each examination you will need to spend some of your time searching through false duplicates (records which look similar to the fuzzy-matching tool, but are not true duplicates).  You can eliminate these with the “Ignore” link - however they will continue to reappear each time you re-examine your data – resulting in extra work each time the examination tool is run.

To prevent the false duplicates from re-appearing each time you should save the list of False Duplicates so the examination tool can ignore these next time its run.

 

To avoid false duplicates from reappearing with each examination you should follow these steps:

1)    Once you have finished processing your duplicates (I.e. You have clicked ‘Ignore’ many times on all the false duplicates), go to the ‘Save False Duplicates’ section of the Export Cleaned Data window:

2)    Specify the location where you want to save the list of false duplicates and click the  button.

3)    When you want to examine your data again for any new duplicate records, (See Step 7 - Examine your data for duplicates) you will want to specify this file as the “Saved False Duplicates File”:

4)    When the examination process runs, these matches will no longer appear in your results for processing.

 

Related Topics

Exporting Data