Where do Duplicate Database Records come from?

What are the reasons why the same person, customer or contact might end up in your database multiple times?

There are many reasons why the same person, customer or contact might end up in your database multiple times. These include:

New Record Data Entry

Poor data-entry procedures:

Users are not checking the existing data before entering a new customer to see if they already exist.

Inconsistent formatting & abbreviations:

Even if checked, a differently formatted name may not be found or recognized as a duplicate. Eg: "Doctor Robert Smith" may not be recognized if he also exists as "Dr Bob Smith".

Change of addresses:

The same person may appear to not exist in the database if they have since changed their address.

Poor data integration:

Different systems and databases may integrate together but have inconsistent data integrity rules causing different departments within the organization to enter a customer multiple times.

Poorly designed software:

If a system makes it difficult for staff to check for an existing record, they will often enter a duplicate record out of ignorance that the customer already exists in the system.

Poor data validation:

When fields such as City, State and ZipCode are not validated to check they have been correctly entered – then human errors can creep into the system causing duplicate records to appear.

List Merging

Even with all the procedures, checks and validations in place – it is common to merge two lists of records together into one, only to find some instances where the same customer record existed in both original databases.

The result is that you now have multiple representations of the same customer in your system – requiring some sort of duplicate finding software to find and remove those duplicate records. This can happen under the following conditions:

Company merging and acquisitions:

Combining two organizations together often requires databases to also be merged. Some customer records may exist in both systems – resulting in duplicates in the merged data.

Prospect list purchases:

In order for a company to market to a new set of customers, a list may be purchased and imported into the existing prospect and customer databases. If a customer record was imported as a prospect, while they are already a customer, you risk losing credibility by marketing to them as a new prospect.

Do-Not-Contact-Lists (DNCL):

You may want to merge a DNCL into your database to ensure you do not contact someone who has explicitly requested to not be contacted and risk being fined.

Systems Integration:

Often multiple systems need to talk with each other – for example, perhaps your Financials system needs to talk to your Ordering and Quoting system which in turn needs to talk to your Customer Relationship Management system.

Each of these systems can have a different database and therefore a different representation of the same customer record. This becomes a problem if the different systems cannot reconcile who is who (with something like a company-wide unique identifier such as a Customer ID number) and may create new records for a customer who may already exist in the database.