Skip to content
Snippets Groups Projects
Select Git revision
  • master default protected
  • redeWiedergabe_dev
  • redewiedergabe_dev
  • patch-4
  • patch-3
  • ATHEN_web
  • patch-2
  • patch-1
  • SkipDS
  • goldstandardN-ary
  • pu/collaboration
  • ATHEN_showcase
  • newEditorBranch
  • nappiUI
  • img_editor_refacotrListener
  • kallimachosViews
16 results
Blame Permalink
ATHEN_goldstandard.md 2.58 KiB

The Goldstandard View

The Goldstandard View enables you to compare the annotations of two types. It can be found among the Views on the right side of ATHEN.

When you use this View for the first time you most likely need to create a mapping first. This mapping can be saved in a file so you do not have to create it anew if you want to use it again later.

The mapping is necessary so ATHEN knows which features of Gold- and System annotations correspond to each other.

Creating a mapping

The wizard for creating a new mapping is started via the button labeled "Create Mapping" at the top of the Goldstandard View. The creation of a mapping is divided into several steps:

In the first step you have to choose which types of annotations should be compared by selecting a Gold- and a System type. Drawing

Next you can map features of the Gold type to features of the System type. The wizard ensures that no feature can be mapped to more than one feature of the other type. If there is a feature you do not want to map you can just leave the corresponding drop-down box empty. All features which have not been mapped are ignored in the comparison of the annotations. The begin and end indices of the annotations are not present on the mapping page since these two features are always compared. Drawing

After this step the wizard informs you whether the mapping is complete or not. Additional steps are required if you mapped features which are annotations themselves (instead of primitive types like Strings or Integers).

The following steps only appear if the wizard informs you that the mapping is not yet complete: In the third step you select which additional annotation types can appear as features of the Gold or System type. After that you map these types.

For each mapped pair of Gold and System type you then again have to map features.

After the mapping

After a mapping has been created or loaded the annotations of the specified Gold and System type can be compared by pressing the button labeled "Compare".

In the bottom part of the View you can see the annotations that could be found in a table. The left column contains the annotations of the Gold type, the right column the ones of the System type. If two matching annotations were found a line contains entries in both columns. Otherwise one of the columns in the line is empty.

Above the table the counts for true positives (TP), false positives (FP) and false negatives (FN) are displayed, as well as precision, recall and the F1 score.