Now, let's combine what we've learned about evaluations and match configurations to evaluate the performance of our now match configuration.
-
Select the Evaluate tab from the navigation bar.
-
Select New Evaluation in the Options column for the uploaded gold data file.
-
Select KH Test from the Match Configuration dropdown menu.
-
Select Start Evaluation.
-
Select Specific Threshold next to Display and set it to 70%. We know that our test query, Heart Kirsten, is a match at this threshold for this configuration.
Looking at the evaluation table, you will see that precision, recall, and F1 are unchanged. However, both precision and F1 are higher at our 0.7 match threshold than they were for the default configuration at a 0.62 match threshold. By creating a new match configuration, we have managed to make RMS match our two test names without becoming overly vulnerable to false positives.
Also note that the ideal threshold of the new configuration is 0.71, up 1% from the default configuration's ideal threshold of 0.71. Keep in mind that changing match configuration settings also changes the ideal threshold.