Once you've created your data corpus and loaded the documents, at any point in the annotation process, you can review the corpus and the annotations. Select View Annotations to see a detailed view of your work.
View annotations provides a detailed view of annotations and adjudication. It consists of the Filters panel and the sample list.
Select Filters on the View Annotations page to expand the Filters panel. The Filters panel allows you to select which samples to display in the sample list using the following criteria:
-
Annotated by: Allows adjudicators and managers to select which annotated samples to display based on who annotated them. If you select All selected annotators or Any selected annotators, select annotators with the dropdown menu.
-
Date annotated: View only samples annotated within a specific date range.
-
Assigned to: Allows adjudicators and managers to select which unannotated samples to display based on their assigned annotator(s). If you select All selected annotators or Any selected annotators, select annotators with the dropdown menu.
-
Commented: Display only samples with comments.
-
Text: Display samples containing specific terms within the sample text. The text field supports regular expression searches.
-
Doc ID: Display samples with a specified doc id.
-
Dataset: Display samples from the training dataset, evaluation dataset, or both.
-
Labels: Select which samples to display based on which label(s) they contain and whether the sample is adjudicated or non-adjudicated. Sselect one or more labels with the dropdown menu. This filter only applies to NER.
-
Events: Select which samples to display based on which event(s) they contain. If you select Annotated samples with all selected events or Annotated samples with one or more selected events, select events with the dropdown menu. This filter only applies to events.
Select Update results to filter the sample list so that it only displays samples with all chosen criteria.
All samples in the project that match the applied filters are listed below the filters panel.
Each sample is displayed next to a table showing the tags selected by each annotator for each entity. If you are a project manager, this view is useful for identifying and inspecting specific points of high annotator disagreement. Adjudicators and managers can also see any comments added by annotators on the rightmost column of the table.
The following are displayed below each sample:
-
The sample’s dataset (training or evaluation)
-
A link to view annotation predictions. Predictions are the tags suggested by the currently trained model.
-
Links to perform any available actions
-
Annotate: Review the annotation and, if desired, clear and modify the annotations.
-
Adjudicate: View and resolve annotation differences. This option is only available to adjudicators and managers.
-
Copy document ID: Copy the Document ID to your clipboard.
If there are two or more identical entities, each will be labeled with a corresponding id in the sample text and the Entity column. If there is annotator disagreement for an entity, that entity will be highlighted in the sample text. Mouse over any entity in the Entity column to highlight that entity in the sample text.