Once you've created your data corpus and loaded the documents, at any point in the annotation process, you can review the corpus and the annotations. Select View Annotations to see a detailed view of your work.
View annotations provides a detailed view of annotations and adjudication. It consists of the Filters panel and the sample list.
The Filters panel allows you to select which samples to display in the sample list using the following criteria:
-
Annotated: Allows annotators to select which samples to display based on whether or not they have been annotated.
-
Annotated by: Allows adjudicators and managers to select which samples to display based on who annotated them. If you select All selected annotators or Any selected annotators, select annotators with the dropdown menu below the "Annotated by" options.
-
Assigned to: Allows adjudicators and managers to select which samples to display based on their assigned annotator(s). If you select All selected annotators or Any selected annotators, select annotators with the dropdown menu below the "Annotated by" options.
-
Date annotated: Display samples that were annotated within a specified date range.
-
Commented: Display all samples with comments.
-
Adjudicated: Allows adjudicators and managers to select which samples to display based on adjudication method or status. You can select multiple options for this filter.
-
Date adjudicated: Allows adjudicators and managers to display samples adjudicated within a specified date range.
-
Text: Display samples containing specific terms within the sample text. The text field supports regular expression searches.
-
Document id#: Display samples with a specified document id#.
-
Dataset: Display samples from the training dataset, evaluation dataset, or both.
-
Event type: This filter only applies to events.
-
Labels: Select which samples to display based on which label(s) they contain. If you select Adjudicated samples with all selected labels or Adjudicated samples with one or more selected labels, select labels with the dropdown menu below the "Labels" options. This filter only applies to NER.
Click Update results to filter the sample list so that it only displays samples with all chosen criteria.
All samples in the project that match the current filters are listed below the filters panel. Set the number of rows per page using the dropdown menu at the top of the list. Navigate between pages using the Previous and Next buttons at the top of the list, or by typing the desired page into the Page field.
Each sample is displayed next to a table showing selected labels by annotator for each entity in that sample. If you are a project manager, this view is useful for identifying and inspecting specific points of high annotator disagreement. Adjudicators and managers can also see any comments added by annotators on the rightmost column of the table.
The following are displayed below the annotation table for each sample:
-
The sample state (annotated or not annotated)
-
List of annotators who have annotated the sample
-
List of pending annotators (annotators who have not yet annotated the sample)
-
The sample’s dataset (training or evaluation)
-
A link to view annotation predictions. Predictions are the labels suggested by the currently trained model.
-
Links to perform any available actions
-
Annotate: Review the annotation and, if desired, clear and modify the annotations.
-
Adjudicate: View and resolve annotation differences. This option is only available to adjudicators and managers.
-
Copy Document ID: Copy the Document ID to your clipboard.