Match Studio Release Notes

Updated the download link for Apache Artemis. The link had been updated by Apache, which broke the RMS installer. This release not has the correct link. We have also published the workaround for impacted previous versions of RMS. (RMS-940)

The Docker build is not supported on machines with the Apple M1, M2 chip. This is a well-known Docker issue which will be addressed in a future release of Docker for Mac.

We recommend using a single entity type per Evaluation file to perform accuracy evaluations.

We recommend using Vault to encrypt and decrypt the authentication data being used to connect to external Elasticsearch. Note that if using the basic dev mode of Vault, the external servers with authentication or APIs will need to be re-added when you restart Vault.

The docker installation has two high vulnerabilities associated with the base image of docker being used. We will address this in a future release.

We recommend using the native version of Match Studio instead of the Docker version unless required.

RNI-ES 8.8.1.0 support: The local Elasticsearch server that RMS will be shipped with is updated to RNI-ES 8.8.1.0. RMS can now be connected to the following RNI-ES versions: 8.4.1.x, 8.5.3.x, 8.6.2.x and 8.8.1.x. (RMS-852)

SSL support: During installation, users can now set up SSL between server and client for encrypted communication using their own certificates. (RMS-739)

Spring Boot server elimination: We have removed a redundant server. RMS now works with two spring boot servers instead of three. (RMS-307)

.tsv and delimited .txt support for ingest: Expanding on our support for files that can be used to add data to the system, we have added support for .tsv and delimited .txt files (tab or comma separated) for ingesting data into Match Studio. We have removed support for .yaml files. (RMS-733)

Security updates: Updated more dependencies in the frontend and the backend to eliminate security vulnerabilities. (RMS-831)

Files with BOM characters are now handled by RMS correctly. (RMS-891)

Fixed the issue with batch search exports. The batch search results can now be exported correctly with all the data correctly present. (RMS-843)

Incorrect Apache Artemis download link in documents has been corrected. (RMS-826)Incorrect Apache Artemis download link in documents has been corrected. (RMS-826)

Fixed a bug in which the Compare Dates page did not send the correct request when selecting a Match Configuration. (RMS-825)

Match configurations with invalid values cannot be imported now. (RMS-822)

Users cannot run RMS install file as Sudo users anymore. (RMS-157)

Pagination now works correctly for overrides. (RMS-779)

Errors when installing and running Apache Artemis on Windows Server have been fixed. (RMS-838)

Match configuration display errors have been addressed. (RMS-857, RMS-858, RMS-859)

A duplicate of the default parameter profiles file from RNI can now be exported into RMS without failing. (RMS-874)

Invalid evaluation files are now displayed correctly in the list of evaluations. (RMS-817)

When selecting a specific threshold to evaluate accuracy, the user cannot set the values as decimals. This will prevent users from getting unnecessary exceptions. (RMS-824)

The Docker build is not supported on machines with the Apple M1, M2 chip. This is a well-known Docker issue which will be addressed in a future release of Docker for Mac.

We recommend using a single entity type per Evaluation file to perform accuracy evaluations.

We recommend using Vault to encrypt and decrypt the authentication data being used to connect to external Elasticsearch. Note that if using the basic dev mode of Vault, the external servers with authentication or APIs will need to be re-added when you restart Vault.

The docker installation has two high vulnerabilities associated with the base image of docker being used, we will address this in the next release.

We recommend using the native version of Match Studio instead of the Docker version unless required.

The Help guide has been removed from this release due to a vulnerability in the Help files.

Fixed a bug in which users were unable to add data to an existing index. (RMS-778)

Restored the ability to delete overrides. (RMS-723)

The Docker build is not supported on machines with the Apple M1, M2 chip. This is a known Docker issue which will be addressed in a future release of Docker for Mac.

We recommend using a single entity type per Evaluation file to perform accuracy evaluations.

We recommend using Vault to encrypt and decrypt the authentication data being used to connect to external Elasticsearch.

There are 3 medium vulnerabilities caused by spring Boot 3.0.5 version. These will be resolved once the new Spring boot version is released.

The Docker install has 3 high vulnerabilities associated with the base image of Docker being used. We will address this in the next release.

RNI-ES 8.6.2.0 support: The local Elasticsearch server shipped with RMS has been updated to RNI-ES 8.6.2.0. RMS can now be connected to the following RNI-ES versions: 8.4.1.x, 8.5.3.x and 8.6.2.x.

Docker image update: Docker images have all been moved to the Rosette Docker Hub repository. Older versions of Docker RMS installs will be affected once the old basistechnologycorporation repository is shut down. If users delete their images and try downloading them again, they will need to update the docker-compose.yaml file. Reach out to the RMS team for more details.

API key support for external Elasticsearch: Users can now connect to an external instance of Elasticsearch using API keys generated with Elasticsearch.

Vault support: Rosette Match Studio lets users add their Elasticsearch credentials (Basic Auth and API keys). These credentials are stored in the RMS relational database. Users can now set up Vault during installation to encrypt these credentials so they are not stored in the database as clear text.

Elastic Cloud support: Users can now connect to Elastic Cloud using either Basic Auth or API keys.

Security improvements: Version upgrades to fix vulnerabilities.

Usability enhancements: UI improvements on Compare page and Match Configuration page.

Fixed a bug in which Evaluation ingest was failing on some files.

The Compare page for address matching has been updated and issues with some address fields have been fixed.

Fixed a bug in which Batch Search deletion caused a 500 Server error.

In some Batch Searches, expanding Batch Search results expanded more than 1 row. This has now been fixed.

Spring Boot 2.7.x has now been upgraded to 3.0.x, resolving multiple vulnerabilities.

Fixed a bug in which the Match Configuration would fail when switching between servers.

The Docker build is not supported on machines with the Apple M1, M2 chip. This is a known Docker issue which will be addressed in a future release of Docker for Mac.

We recommend using a single entity type per Evaluation file to perform accuracy evaluations.

We recommend using Vault to encrypt and decrypt the authentication data being used to connect to external Elasticsearch.

There are 3 medium vulnerabilities caused by spring Boot 3.0.5 version. These will be resolved once the new Spring boot version is released.

The Docker install has 3 high vulnerabilities associated with the base image of Docker being used. We will address this in the next release.

Windows installation no longer depends on 7-Zip.  The only software prerequisite is Java 17 (RMS-684).

Permissions for the db_data directory are now set explicitly (RMS-682). 

Fixed a bug in which startup would fail if the Windows PATH variable contained parentheses (RMS-676).

Batch Search now properly reads the match threshold from the value set by the user during the batch job creation (RMS-553).

Search Details preserve the case of the retrieved name when linking to the Compare Page (RMS-678).

Search Details use the correct match configuration when linking to the Compare Page (RMS-691).

The user can now see new results immediately when selecting "Specific Threshold" in an evaluation (RMS-647).

Several UI fixes and improvements related to overrides and stopwords (RMS-616, RMS-660, RMS-688, RMS-689, RMS-693).

The Docker build is not supported on machines with the Apple M1 chip. This is a known Docker issue which will be addressed in a future release of Docker for Mac.

We recommend using a single entity type per evaluation file to perform accuracy evaluations.

Overrides: This feature defines token pairs that are considered to be matches regardless of similarity. This can be used for nicknames. For example, if "Margaret" and “Peggy” are a pair on the override list, they will have a high match score even though the strings are not similar. RMS comes with some predefined overrides for various language pairs and entity types. Users can also add new overrides, delete existing overrides, or search for specific overrides.

Stopwords: This feature defines words that are ignored during string matching. A stopword can either be a prefix (like “Major”) or a regex (regular expression, like “\bfnu\b”). For example, if RMS is comparing “John Smith” and “Major John Smith,” and "major" is on the overrides list, then the word “major” will be ignored and the strings will have very high match score. RMS comes with some predefined stopwords for various languages and entity types. Users can also add new stopwords, delete existing stopwords, or search for specific stopwords.

Add data to existing index: You can now add additional data to an existing index via the the Add Data button (located on the Data tab of the Configure Index page). The source file for the new data must contain all the imported columns from the original index, with identical field headers. Additional columns will be ignored.

Configure field width type: You can now configure how RMS handles search results that are too wide to display in that field's column. (RMS-596)

There are three options:

Windows 10 and 11 installation prerequisites: The prerequisites for installation of the Windows 10 and 11 version of RMS have changed to make installation easier.

See the complete new prerequisites in the Rosette Match Studio installation guide.

Exposed score_if_null parameter: The RNI-ES parameter score_if_null is exposed in the rest-module.yml (Docker) and rest-module-nodocker.yml (Windows/MacOS/Linux) configuration files. The default value is -1.0 (disabled). Set this parameter to a value between 0 and 1 to specify the score that should be returned when a field is missing (null). (RMS-658)

Updated Docker images: RMS Docker images have been updated to use the Redhat base image. Previous versions of RMS used Openjdk base images, which have been deprecated.

Improved installation: The installation process now prompts you for RMS hostname(s) and adds them to the allowed origins for configuring HTTPS. This reduces manual configuration steps. (RMS-622)

Improved handling of large files for index creation, batch search, and evaluation (RMS-638)

Help page can now be opened in a new tab. (RMS-573)

Fixed a bug in which the Compare button (in the Action column for search results) was causing a crash with certain configurations. (RMS-581)

Restored the spinner on Search page when waiting for results. (RMS-582)

The Compare page for organizations now correctly sets detected languages unless the match reason is REAL_WORLD_ID. (RMS-590)

Search terms can now contain double quotes. (RMS-617)

Evaluation files now accept upper or lower case language codes. (RMS-627)

System memory check during Windows installation now works for all locales. (RMS-659)

Ports 8081 and 8082, used internally by the application, can now be changed in startup.sh or startup.bat. This allows you to work around local port restrictions. (RMS-602)

Upon installation and startup, license check failure now produces a warning rather than an error. This ensures you will not get blocked during installation. (RMS-620)

The Docker build is not supported on machines with the Apple M1 chip. This is a known issue which will be addressed in a future release of Docker for Mac.

We recommend using a single entity type per evaluation file to perform accuracy evaluations.

Multiple index support: You can now create and manage more than one index in a single RMS installation. Search or batch search any uploaded index directly from the Search Indices page.

Delete index: You can now delete any index via the Search Indices page. Deleting an index also deletes all batch search data associated with it.

New Index Wizard: We have added a 4-step index wizard to make importing data easier. This includes progress indicators

Improved file ingest: RMS now has better exception handling and error messages for the index, batch search, and evaluation file ingest processes. We have also added row-level details for import errors, making it easier to spot and resolve problems with your data files.

Configure Index: You can manage each index via the Configure Index page. Configurable settings include index name, query fields, search results display, window size, match configuration, display threshold, and match threshold.

Window size and match configuration: These settings can now be changed directly from the search pane. This must be enabled via the Configure Index page.

Batch search: We have redesigned batch search for better performance and simplified the file mapping process to make starting a new batch search easier.

Batch search export results: Improved the batch search download file format and contents to reflect the index configuration and batch search settings.

Progress indicators: We have added more progress indicators to all processes including file ingest, batch search, and evaluation.

Import and export match configurations: You can now import any RNI parameter_profiles.yaml configuration file into RMS as a match configuration. You can also export any RMS match configuration for use in any compatible RNI environment.

Installation guide: We have added RMS-installation-guide.pdf to all RMS shipments. This document walks you through the process of installing RMS on a single machine.

New installer: We have added install_rms.sh to install all the services required to run RMS, check for common issues, and provide useful error messages, making installation easier.

Improved Windows installation experience: Windows installation is now easier. We have added more installation details to the README and updated the install script to catch more errors and exceptions.

Java support: Java 17 is now supported and required. Support for earlier versions of Java has been deprecated.

RNI-ES: RNI-ES has been updated to version 8.4.1.0.

Improved performance for evaluations.

Fixed inconsistent match scores when using compare.

Searches on "Gender" type columns are now working properly.

The Docker build is not supported on machines with the Apple M1 chip. This is a known issue that should be addressed in a future release of Docker for Mac.

Match configurations on Compare page: Added a match configuration selector to the Advanced Settings panel on the Compare page. Users can now select a match configuration when using pairwise match.

Compare page advanced settings: Improved the usability of the Advanced Settings panel in Compare page by limiting parameter mix/max values to the “sane” values prescribed in the RNI documentation.

Docker: Docker package (in addition to a dockerless version) is now available.

Invalid evaluation file: Added error message upon upload of an invalid evaluation data file.

Java support: Java 11 is now supported. Java 8 is no longer supported.

Node support: Node 16.0.0 is now supported.

Spring support: Spring 2.5.14 is now supported.

Previously, the Compare page displayed only a match score but no "Match Score Computation" matrix for organizations that were matched with REAL_WORLD_ID. The matrix has now been added.

Fixed a bug in which Advanced Settings parameters would show up as NaN when navigating to the Compare Page from a search result.

Improved performance when running accuracy evaluations on large files.

Fixed a bug in which the threshold report did not properly display the name of the match configuration.

Fixed a Windows encoding issue which prevented users from using any language other than the default language set up on Windows.

Miscellaneous additional bug fixes and usability improvements

The time taken to perform evaluations increases as more evaluations are run. This will be improved in a future version.

Searches on "Gender" type columns are not working properly.

In some cases, running the startup.bat file in the Windows build will result in Elasticsearch failing to start and the parameters_def.yaml file in the RNI-ES plugin being deleted.

It is currently possible to upload the same evaluation data file multiple times.

Match configurations: Parameters can now be updated for a language pair and an entity type and saved as a named Match configuration. This can be used to perform search, batch search, and accuracy evaluations. The first five RNI parameters we support are: initialScore, deletionScore, reorderPenalty, conflictScore and boostWeightAtRightEnd. These parameters can be applied to PER, LOC, and ORG entity types.

Select match configuration: We have added a new option to select a match configuration on the the Search Configuration page. The selected match configuration will be used for ad hoc search and batch search.

Evaluations: “Gold” data files containing pairs of names can now be uploaded for evaluation. During evaluation, the RNI pairwise score is calculated for each name pair in the data file and compared to the Match column in the evaluation data file (which confirms whether the names are supposed to match) to calculate Precision, Recall and F1. Evaluations can be performed on a single evaluation file with different match configurations.

Threshold report: After creating an evaluation, you can view a report of accuracy at each threshold between .5 and 1.0. The threshold and accuracy metrics with the best F1 score are automatically displayed. The underlying match results and summary statistics are available for download.

Search tab dropdown: You can now access some features by mousing over the Search tab and selecting them from a dropdown menu.

No docker version. A docker version will be added in the future.

Evaluation input file only tested to 2000 rows. More rows will be tested in the future.

The time taken to perform evaluations increases as more evaluations are run. This will be improved in a future version.

Real World ID matches do not properly display the "Match Score Computation" matrix on the Compare page.

Once a match configuration is used for an evaluation, it cannot be deleted.

The threshold report does not properly display the name of the match configuration.

There is a Windows encoding issue which prevents users from using any language other than the default language set up on Windows.