Data Deduplication

In the process of collecting information and tracking customer behavior, data is often fragmented and creates many profiles with duplicate identifiers. The Data Deduplication feature will combine these profiles into a single profile, forming a complete Customer 360 View.

On the home interface, select Company Account.

In the Company Account section, at Data Management, select Data Deduplication.

On the Data Deduplication interface, there are 4 configuration sections: Merge RulesTrial TestingMerge History, and Notification Configuration. The interface also allows the user to review all Merge Rules that have been set, and whether these configurations have been enabled or not.

Merge Rules

This feature allows users to configure unified information fields and how accurate they are, as follows:

  • CIF Number: Accurate
  • Phone number: Accurate
  • Email: Accurate
  • Full name: Accurate or Approximate
  • Identification documents: Accurate
  • Profile picture: Approximate
  • Personal address: Accurate or Approximate
  • Profile ID on external system: Accurate
To add or edit an old rule, under Merge Rules, select Edit

Main functions on the Merge Rules Configuration window:

  • Add condition
  • Delete a condition
  • Copy the rule
  • Add a new rule: Applies when the above rule has not merged all duplicate profiles.
  • Continue: Switch to the cases of exclusion configuration
Usecase

The following Merge Rules configuration has 2 codes:

Rule 1: Merge all Profiles with the same Phone Number; Full Name and First Name only need to be approximate, do not merge Profiles with special characters in the name.

Example:

ProfileABC
Full nameNguyen Minh AnhM Anh NguyenMin_Min Nguyen
Phone number091929394091929394091929394

→ System merges Profiles A and B, ignoring C.

After searching and merging all Profiles according to the 1st rule, the system perform the same operation with the Profiles that meet the next rule.

Rule 2: Merge all Profiles with the same Email; Full Name and First Name only need to be approximate, do not merge Profiles with First Name without accents.

Example:

ProfileABC
Full nametran hongTrần Thị HồngHồng Trần
Emailname@abc.comname@abc.comname@abc.com

→ System merges Profiles B and C, ignoring A.

Cases of Exclusion

When merging data, the system will ignore profiles that meet the conditions of this configuration.

The default exclusion conditions are:

  • Profiles are joining the Journey
  • Profile with loyalty score
  • Profile with membership class information

In addition, the user can add some other criteria:

  • Profiles are recorded from reliable information sources: users can set data from which source is the most reliable and accurate. The system will ignore profiles coming from this source.

For example: When choosing a reliable information source asSmart Wifi, the system automatically ignores all profiles coming from this source during the merging process.

  • Profiles with different information: Profiles that meet the rules of merging but have different fields will be ignored.
Usecase

The Merge Rules have the following configuration:

Exclusion case:

Review the Profile list:

ProfileABC
Full nametran hongTrần Thị HồngHồng Trần
Phone numbername@abc.comname@abc.comname@abc.com
Card code426642664285

→ The system does not merge all 3 of these profiles. Profile A is excluded because First and Last names do not have accents, Profiles B and C have different card numbers.

Merging events

Interactive event activities through Social, Marketing, Sales, Ticket modules will be merged after merging Profiles

Example

Profile A has 3 events: Social, JB, Sale
Profile B has 2 events: Ticket, Transaction
=> after B merges into A, profile A has 5 events: Social, JB, Sale, Ticket, Transaction

Criteria for determining the main archive

After merging, all duplicate data will be merged and saved into a single profile. Users can configure to specify which profile those data will be saved to. The condition defines the main Profile to merge data based on the Original Source.

Use case

The Merge rules have the following main archive configuration:

ProfileABCPost-merge profiles
Full nameTrần Thị Hồngtran hongHồng TrầnHồng Trần
Phone number091929394091929394
Emailname@abc.comname@abc.comname@abc.comname@abc.com
Card code42994299
Attribution Profile SourceZaloFacebookSmart WifiSmart Wifi

→ The system merges all information of Profile A and B into Profile C

After all Merge rules have been configured, users can click Back if they want to review and edit the configuration, or click Finish to proceed to Trial testing/Activate.

Trial testing

This feature allows users to test and double-check the rules before officially merging data.

If user has already configured at Merge rules, user can choose to Run a test on current configuration.

Users can also create completely new testing rules right on the test running interface, or create new rules and copy the configuration of existing rules.

Every day, the system displays the progress report of the test run and the results of Profile merging.

Once the experiment is successful and suitable, users can apply the trial testing rules as the Merge rules.

Important notes

Configuring rules
  • When establishing a merge rules, conditions must not overlap or cover each other.
Notification of the system when there are duplicate / covered conditions
  • When there are multiple merge rules, the system will look from top to bottom.
  • Users can configure up to 20 merge rules in 1 ruleset.
Run a merge rule
  • The process of merging Profiles (including trial test and official runs) will take place at night, around 0am – 4am depending on the number of Profiles, helping the system avoid overload and operate smoother.
  • After 4 a.m., the system pauses merging even if it is not completed and will resume this process at 0 a.m. the next day. A data merging progress report will be sent to email.
Users can view and edit notification configurations here
Editing interface of Notification configuration
Testing rule
  • Running the test rule is similar to running a official rule: system scans the entire profile according to the configured conditions, but it only delivers a maximum merge result of about 100 profiles.
  • All results of existing testing rule will be deleted if the user creates a new testing rule.
  • Testing rules cannot be run while running a official Merge rules.
image-zoom
© MOBIO 2022 v1.2 Powered by MOBIO
Back to Top