Identify Duplicate Records | Remove Duplicates | Data Cleaning
"The removal of duplicate contacts is a vital part of any data cleaning and database
integrity process. The Duplicate Record Remover finds and removes duplicate database rows
in any contacts database - and gives you complete control over how duplicate records
are merged in the de-duplication process."
Rodney Lake - Precision Data
Duplicate Record Remover | Deduplication Features
Automatic Merging
Records that are identified as clear duplicates (those that exceed a
customizable matching threshold of typically 95% - where 95% of the record
is the same) are automatically merged together.
Different merge rules can be set according to the data type to ensure no
data is lost during an automatic merge.
Manual Merging
To ensure false-positives (records that look the same, but are actually
different) are not matched by mistake, all duplicates below a configurable
threshold (95% by default) may require validation and editing before
merging. This is done with the intuitive
Merge Tool which shows, with color
coding, which fields are different and what data would be lost in a merge.
This easy-to-use tool allows you to edit each record directly and copy
across data from one record to another prior to deleting the duplicate to
ensure data is not lost in the merge process.
Database Independent: Import from CSV, Excel or SQL Server
You can import
CSV, Excel or
SQL Server data into the Duplicate Record
Remover for examination. Once your data is imported you can work on it
independently from your live data – importing the resulting updates and
merges at a later date.
Oracle, Access and ODBC data sources are under development and will be
released in the next version of the software. If you require one of these
other types please let us know.
Fuzzy Logic Comparisons
The Duplicate Record Remover checks each record against every other record in
your set of data using an advanced fuzzy-logic algorithm – resulting in
matches being found between words despite misspellings, abbreviations or
data being entered in incorrect fields.
Other tools use a dictionary-based comparison engine for misspellings and
abbreviations – resulting in a less accurate examination process. However
the Duplicate Record Remover’s fuzzy logic algorithm searches deeper for
similarities that a dictionary search would leave unfound.
Simple Setup and Flexible Configuration
A straightforward and non-technical
step-by-step setup wizard allows you to
configure your data ready for duplicate examining and processing. You choose
which fields should be examined during the examination process – allowing a
precise selection of what is checked and what is ignored when looking for
duplicate records.
Multiple Outputs
At the end of the processing and merging you can
output your cleaned data in multiple ways:
- Export the completed and cleaned database as either
CSV (Comma Delimited),
XLS (Excel), or XML (Raw Data) for direct
importing back into your database.
- Export the Record Change Log as a Printable Report (for manual
processing).
- Export the Record Change Log as XML which can be used programmatically to
run updates in your database.
- Export the Record Change Log as T-SQL statements that can be run directly
against your RDBMS database.
User Friendly & Easy to Use Interface
Intuitive and clear user interface allows you to validate your duplicates
and edit your merges with simplicity.
At any step in the process you can click on the ‘Help’ icon to get context
specific help – showing you what you need to know to find and process your
duplicate records.
Learn More
You can learn more about the Duplicate Record Remover by
visiting the
online help files or viewing the
product screenshots