Compare

Compare two values of the same field type and view the match score, along with details about how the match score was determined.

The comparison can be between a search value and a matched value or two new terms.

View match scores and understand how the score was determined.
Modify the settings and see the resulting change in match scores.

There are two ways to access the Compare section:

Select the Compare tab from the navigation bar.
Select the compare icon in an expanded search result. This method allows you to automatically compare a searched name with a retrieved name.

To compare two values:

Enter two values to compare.

Note

The values might be the search value with a matched value, or two new values. These will be pre-filled for you when navigating from the search results list.
(Optional) If comparing Names, select the language of each value from the drop-down list. Adding the source languages may improve the match scores.
Select Compare.

Compare Names

The Match Score Computation displays the computation table matrix, a tabular representation of component scores and, if selected, a textual explanation describing how the final match score was determined. For more details on how how name matching works, see Understanding Name Match Scores.

For Name fields, the computation table matrix includes:

Tokens identified
Weight given to each token
Match reason for the token pair
Match score for the token pair

The tokens from Name 1 are listed down the first column, while the tokens from Name 2 are along the top row.

The shaded boxes highlight the token pairs selected during matching that produce the best score. A token pair is a token from Name 1 and its matching token from Name 2. Each shaded box contains a match phenomenon and a match value.

The top line indicates Match phenomenon.
The match value takes into consideration the placement of the token in the score calculation. A penalty is applied if the tokens are out of order. When the tokens line up on the diagonal, they are all in order.

Under each token is a weight. The weightings determine how important the token pair match is in calculating the final score. For example, unusual tokens get a higher weighting than common names because it is more significant when they match and initials are weighted less than full names.

For more detail about the match score, select Explanation

Names Advanced Settings

You can modify the match parameters in real-time to see how each value impacts the final match score.

Rosette Match Studio and Rosette Name Indexer are turned to perform well in a variety of name matching scenarios. However, every use case uses different data with distinct match requirements. In Rosette Match Studio, you can easily modify and tune a set of the most common parameters, to improve the accuracy of matches.

To view and modify these settings:

Select the settings icon in the upper right corner of the compare window.
Select the configuration values to use. There are two ways to set the values:
1. Select a defined match configuration from the drop-down menu. You will not be able to manually change the values.
2. Change the value or values of interest. You may have to scroll down to view the entire list of available parameters.
Compare.

Changes are applied to the pairwise match currently displayed.

The default parameter values are based on the entity type (person, organization, or location) and the language.
Reset sets the values back to their default settings.
To save a set of parameter values, select the copy url icon in the upper right corner. The url contains the parameter values.

Match phenomena

Match phenomena describe why token spans did or did not match. For example, some match phenomena, such as HMM_MATCH, occur when tokens are matched by a particular scorer. Others, such as DELETION, occur when tokens cannot be matched at all.

Note

The examples provided represent match phenomena when using the default parameter values unless indicated otherwise. Parameters can be configured to adjust the score for each match phenomenon.

Name	Description	Example
CONFLICT	The tokens do not match.	"william" and "johnson" are a CONFLICT.
DELETION	The token is unmatched.	When comparing "Richard William Smith" with "Richard Smith", "william" would be considered a DELETION.
EMBEDDING_MATCH	The tokens are semantically similar as determined by word-embedding vectors.	When comparing "boston building company" and "boston construction company", "building" and "construction" are an EMBEDDING_MATCH.
FIELD_BLOCKED	This field cannot be matched because of a cross-field match involving the same field in the other name.	When comparing "Bob\|William\|Smith" with "William\|\|Smith", "bob" is a FIELD_BLOCKED since the cross-field william match prevents it from matching with its corresponding field.
FIELD_CONFLICT	When comparing two names that are divided into fields, these fields do not match.	When comparing "Richard\|William\|Smith" with "Richard\|Johnson\|Smith", "william" and "johnson" would be considered a FIELD_CONFLICT.
FIELD_DELETION	When comparing two names that are divided into fields, this field is unmatched.	When comparing "Richard\|William\|Smith" with "Richard\|\|Smith", "william" would be considered a FIELD_DELETION.
GIVEN_NAME_DELETION	When comparing two names that are divided into fields, the GIVEN_NAME field is unmatched.	When comparing "Richard\|William\|Smith" and "\|\|William\|Scott", "Richard" will be a GIVEN_NAME_DELETION if that field in both names is marked as a `Given_name` field.
HANI_ABBREVIATION	One Hani token appears to be an abbreviation of another Hani token.	"北京大学" and "北大" are a HANI_ABBREVIATION match.
HMM_MATCH	The tokens are similar but not identical, and the match was determined by a particular model (hidden Markov model). This is a type of fuzzy match.	"richard" and "richerd" are an HMM_MATCH.
INITIALISM	One token is a name and the other token is the initials of the words which make up the name.	"john fitzgerald kennedy" and "JFK" are an INITIALISM. "consumer value stores" and "CVS" are an INITIALISM.
INITIAL_MATCH	One token is the first initial of the other.	"w" and "william" are an INITIAL_MATCH.
LANGUAGE_SPECIFIC_MATCH	The match was determined by a language-specific matcher.	"laden" and "لادن" are a LANGUAGE_SPECIFIC_MATCH.
MATCH	The tokens are identical (after stop word elimination and normalization).	"john" and "john" are a MATCH.
NULL	The NULL phenomenon is only listed in this table for completeness. It is only used internally and will never be returned in the SpanMatch object.	N/A
OUT_OF_ORDER_DELETION	This unmatched token still leaves the remaining tokens out of order when it is removed.	When comparing "George Herbert Walker Bush" with "George Bush Walker", "herbert" would be considered an OUT_OF_ORDER_DELETION.
OVERRIDE	The tokens appear as a pair on the override list. This is often used for nicknames.	"john" and "jack" will be an OVERRIDE match if they appear as a pair on the override list.
OVERRIDE_WRAPPED_HMM	One token is similar, but not identical to, the other token's counterpart on the override list.	"john" and "jakk" will be an OVERRIDE_WRAPPED_HMM match if "john" and "jack" appear as a pair on the override list.
PREFIX_INITIAL	One token is an initial that matches a prefix in the other token. In practice, the PREFIX_INITIAL phenomenon is rare.	If the `initialsScore` parameter is set to 0.1, "E Silva" and "EduardoSil" will be a PREFIX_INITIAL match.
SEQ2SEQ_MATCH	The tokens are similar but not identical, and the match was determined by a particular model (neural seq2seq). This is a type of fuzzy match.	"バラック・オバマ" and "Barack Obama" will be a SEQ2SEQ_MATCH when that model is active.
STRING_SIMILARITY	The tokens are similar in string edit distance (number of insertions, deletions, and substitutions) but not similar enough to be a fuzzy match.	"akcd" and "xkcd" are a STRING_SIMILARITY match.
STUCK_INITIAL	One name appears to have an initial mistakenly attached to a preceding token.	"DavidK" and "David Keith" are a STUCK_INITIAL match.
SURNAME_DELETION	When comparing two names that are divided into fields, the SURNAME field is unmatched.	When comparing "Richard\|William\|Smith" and "Richard\|William\|\|", "Smith" will be a SURNAME_DELETION if that field in both names is marked as a `Surname` field.
TRAILING_PATRONYMIC_DELETION^[a]	The unmatched token is a patronymic which has been truncated in the other name.	When comparing "Faisal bin Fahd bin Abdullah" and "Faisal bin Fahd", "bin Abdullah" is considered a TRAILING_PATRONYMIC_DELETION.
TRUNCATED_EXACT_MATCH	The tokens are identical except that one has been slightly truncated.	"murgatroyd" and "murgatroy" are a TRUNCATED_EXACT_MATCH.
TRUNCATED_HMM_MATCH	The tokens are similar, but not identical, and one has been slightly truncated.	"gilpatrickz" and "gillpatrick" are a TRUNCATED_HMM_MATCH.
UNKNOWN_FIELD_MATCH	One of the tokens is part of an "unknown" field in a fielded name. The UNKNOWN_FIELD_MATCH phenomenon is rare and usually requires use of the Java API.	When comparing "Richard\|William\|Smith" with "Richard\|William\|Scott", if the first field is an "unknown" field, "richard" and "richard" would be considered an UNKNOWN_FIELD_MATCH.
^[a] Only applies to Latin script names of Arabic origin.

Compare Dates

Rosette Match Studio calculates the match score based on five different components:

Time: The number of days between Date 1 and Date 2.
Year: The difference of the year fields of Date 1 and Date 2.
Month: The difference of the month fields of Date 1 and Date 2.
Day The difference of the day fields of Date 1 and Date 2. 1 and 30 are far apart in value, even if they are close in time.
String The string distance is calculated by converting Date 1 and Date 2 to a standard format. The score is calculated on the edit distance between the two strings.

For more detail about the match score, select Explanation.

Supported Date Formats

RNI supports a wide variety of date formats.

Days can be represented by 1 or 2 digits.
Months can be numerics (1 or 2 digits) or English characters (full name or 3 character abbreviation).
Years can be represented by 1, 2, 3 or 4 digits.
Supported delimiters include , . - /, as well as a space.
Partial fields can be entered.
At this time, only English month names and abbreviations are recognized.
All words are case-insensitive; upper and lower case are interpreted the same.

The following table shows different acceptable formats for the date March 7, 1984.

Format	Valid Examples	Notes
Y-M-D	1984-03-07; 1984/3/7; 1984.3.07; 1984 Mar 07; 1984-March-7
M-D	03-07; 3/7; Mar-07; March 7
Y-M	1984-03; 1984 March; 1984-Mar
YYYYMMDD	19840307	All 8 digits must be included
M-D-Y	03-07-1984; 3/7/84; March 7 84; Mar. 7, 1984
M-YYYY	03-1984; March 1984; Mar-1984	The year must include 4 digits. March-84 will not be recognized.
D-M-Y	07 03 1984; 7/3/84; 07 March 84; 7/Mar/1984
D-M	07-03; 7/3; 07-Mar; 7 March
D(MONTH)Y	7MAR84; 07March1984	The month is a word or abbreviation
YYYY	1984
Month	March

Dates Advanced Settings

You can modify the weights given to each type of date match to see how each value impacts the final match score.

To view and modify the date settings:

Click on the settings icon in the upper right corner of the compare window.
Change the weightings of the date component matches. You can also choose to disable swap.
Compare.

Changes are applied to the date pair currently displayed; overall date search settings are not modified.

Reset sets the values back to their default settings.
To save a set of parameter values, select the copy url icon in the upper right corner. The url contains the parameter values.

Enable Swap

Because dates are sometimes written month day and other times written day month, swap tries matching the date fields as written as well as with the month and date fields switched. The best score is returned as the match score. For example, if the dates in question are 1970-3-5 and 1970-6-4, this feature will match the following four pairs:

1970-3-5	↔	1970-6-4
1970-3-5	↔	1970-4-6
1970-5-3	↔	1970-6-4
1970-5-3	↔	1970-4-6

The maximum score of the four pairs is then returned as the match score.

Check the Enable Swap box in the Advanced settings if you think there may be formatting inconsistencies in your dates and the dates you are matching may not always have days and months in the same positions.

If the selected match score is from a swapped pair, a penalty score is applied, indicating less certainty in the match. The displayed Pre Swap Penalty Score is the score returned for the selected swapped pair, before the penalty is applied.

Weighting Date Matches

The date weighting fields control the relative strength of each aspect of the date-matching algorithm. A separate score is calculated for each match type. The final match score is calculated by performing a weighted arithmetic mean over each of the similarity scores. If a field is missing from a record, that field is ignored and its weight evenly distributed across other fields.

Table 2. Date Weighting Parameters

Display Name	Parameter Name	Score based on	Example
Time Weight	timeDistanceWeight	The number of days in between the two input dates	1979-12-31 and 1980-1-1 look different, but their time difference is very close. They will have a high match score.
Year Weight	yearDistanceWeight	The difference of the year fields	Close years will have a high match score.
Month Weight	monthDistanceWeight	The difference of the month fields	1 and 12 are far, even if they are close in time. They will have a low match score.
Day Weight	dayDistanceWeight	The difference of the day fields	1 and 30 are far, even if they are close in time. They will have a low match score.
String Weight	stringDistanceWeight	The edit difference between the two dates, when converted to a standard string (05021974 for 5/2/1974)	1979-12-31 and 1980-1-1 will be 19791231 and 198000101. They will have a low match score.

Dates with a high time match score may have a very low string match score. Time finds dates that are close together; string gives high scores to similarly formatted dates.

Compare Addresses

Rosette Match Studio compares addresses by comparing the fields within each address. You can enter the address as a single field or as separate fields.

Select the format of the address - single-field or multi-field. Both address 1 and address 2 must be in the same format.
Enter the addresses.
Compare.

RMS optimizes the matching algorithm to the field type. Named entity fields, such as street name, city, and state are matched using an algorithm similar to name matching. Numeric and alphanumeric fields such as house number, postal code, and unit, are matched using numerically-based methods.

The Match Score Computation displays the match matrix for the address fields. Each entered field in address 1 is compared with each field in address 2, similar to how name tokens are scored.

Supported Address Fields

Addresses can be defined either as a single field or as a set of address fields. When defined as a single field, the jpostal library is used to parse the address string into address fields.

When entered as a set of fields, the address may include any of the fields below. At least one field must be specified, but no specific fields are required.

Table 3. Supported Address Fields

Field Name	Description	Example(s)
`house`	venue and building names	"Brooklyn Academy of Music", "Empire State Building"
`houseNumber`	usually refers to the external (street-facing) building number	"123"
`road`	street name(s)	"Harrison Avenue"
`unit`	an apartment, unit, office, lot, or other secondary unit designator	"Apt. 123"
`level`	expressions indicating a floor number	"3rd Floor", "Ground Floor"
`staircase`	numbered/lettered staircase	"2"
`entrance`	numbered/lettered entrance	"front gate"
`suburb`	usually an unofficial neighborhood name	"Harlem", "South Bronx", "Crown Heights"
`cityDistrict`	these are usually boroughs or districts within a city that serve some official purpose	"Brooklyn", "Hackney", "Bratislava IV"
`city`	any human settlement including cities, towns, villages, hamlets, localities, etc.	"Boston"
`island`	named islands	"Maui"
`stateDistrict`	usually a second-level administrative division or county	"Saratoga"
`state`	a first-level administrative division	"Massachusetts"
`countryRegion`	informal subdivision of a country without any political status	"South/Latin America"
`country`	sovereign nations and their dependent territories, which have a designated ISO-3166 code	"United States of America"
`worldRegion`	currently only used for appending "West Indies" after the country name, a pattern frequently used in the English-speaking Caribbean	"Jamaica, West Indies"
`postCode`	postal codes used for mail sorting	"02110"
`poBox`	post office box: typically found in non-physical (mail-only) addresses	"28"

Understanding Name Match Scores

To fully understand the name match scores, you need to understand how the match scores are determined. The match score is a value between 0.0 and 1.0; the higher the score, the stronger the match. The score is a relative indication of how similar the two names are; it is not an absolute value. When comparing different name matches, the relative values of the match scores are more relevant than the actual score. Similar name matches in different languages may generate different match scores.

A value of 1.0 is returned if and only if the two names are identical. Character strings, languages, languages of origin, and entity types must all match for the two names to be considered identical.

Calculating the match score is a complex process that involves multiple steps and algorithms.

Identify and normalize the tokens in each name. Each name will usually have multiple tokens.
Compare each token from name 1 with each token from name 2, calculating the score for every token pair.
Once all the token pairs have been scored, the best combination of tokens is selected to maximize the complete score.
Score unmatched tokens as deletions or conflicts.
Compute a weighted average score.
Adjust the final score. For example, the score is decreased if the gender of the two names does not appear to match.

Compare displays the results of the comparative analysis of the two names. The relative importance and values used in the match score calculations can be adjusted in real-time using the Names Advanced Settings.

Tokenize, Normalize, and Transliterate

Before any matching algorithms can be run, the names have to be transformed into tokens that can be compared. This step, as with many of the steps in name matching, often has language-specific components.

This step includes:

Removing stop words, such as Mr. or Senator or General. Stop words are language-dependent.
Transliterating into English and/or translating if necessary, including:
- Adding vocalizations, or vowels in the correct location for languages such as Arabic which are often written in an unvocalized form.
- Adding spaces (segmentation) to languages such as Chinese, Japanese, Korean, and Thai that don't use spaces, separating given and surname tokens.
Normalization, including removing diacritical marks, to get a canonical representation of the token.

The resulting token output enhances search accuracy and increases relevancy.

Calculate Scores for Token Pairs

Every token in Name 1 is matched and scored against every token in Name 2 to find the matching pairs that will result in the highest total score for the pair. All candidate token pairs are scored to determine the best match alignment.

Token scorers are modular, allowing different scorers to be chosen for each token pair. The choice of scorer applied will depend on the type of tokens, the type of matches, and the languages of the names.

There are multiple types of scorers used, including:

Fuzzy Matcher
Nicknames and Cognates
Initials
Truncation
String Similarity

Names that are very similar by their string edit distance (insertions, deletions, substitutions) but not considered close by the fuzzy matcher will still match.

Each matcher returns a (ts) score for the pair.

Pick the Matching Pairs

Once all the token pairs have been scored, the best combination of tokens is selected to maximize the complete score. The chosen pairs are displayed along with their scores.

Score Deletions and Conflicts

To be considered a match, token pairs must score higher than both the conflictThreshold and estimatedConflictOrDeletion parameters. Tokens not part of a satisfactory pair in this regard will be considered either conflicts or deletions.

A conflict is a score given to a token pair which context suggests should have matched, but whose score was not high enough to be considered a match. For example, when “Johann Sebastian Bach” is matched against “Johann Ambrosius Bach”, Sebastian is in conflict with Ambrosius. The token pair is considered a conflict.

A deletion is a score given to individual tokens that are not part of successful match pairs or conflict pairs.

There are two types of deletions:

A deletion is an unmatched token where its immediate surrounding tokens are matched.
An Out-of-order deletion is an unmatched token where its immediate surrounding tokens are not matched.

Calculate the Weighted Score

The token pair scorer returns 2 values, a (ts ) score and a (cs) score. The ts is the score of how well the tokens match, while the cs score includes the placement of the token in the score calculation. The cs will be lower if the tokens match but are out of order.

Each token has a weight. The weightings determine how important the token pair match is in calculating the final score. Full names are rated higher than initials, and unusual tokens get a higher weighting than more common names because it is more significant when they match. For example, Andrew is less common than John, so it gets a higher weighting. These weightings are used in calculating the final score, which is a weighted average of the cs scores.

The cs scores are displayed on the match score computation table, under the match type. The token weights are displayed under each token.

Tip

We apply specific cultural knowledge to the weighted scoring; for example in Spanish we handle parsing of matrilineal vs patrilineal surnames, with the onomastic understanding that the patrilineal name is likely to be treated as the person's primary surname.

Adjust the Final Score

At the end, all selected tokens and token-pairs are considered together and some final adjustments are made to the score. Examples of adjustments include:

A penalty is applied if the two names do not appear to have the same gender. This, of course, is language-dependent.
Sometimes there is a weighting penalty applied early in the process, which when considered at the end, in the full context of the match, is not as significant as initially determined. These values may be adjusted in the final score.

Articles in this section

Note