Skip to content
GitLab
Menu
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
kallimachos
Athen
Commits
a9354772
Commit
a9354772
authored
Feb 22, 2018
by
Markus Krug
Browse files
fixed a very strange bug in the goldstandard analyzer
parent
f41fd1f2
Pipeline
#7835
passed with stages
in 19 minutes and 14 seconds
Changes
6
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
de.uniwue.mk.athen/bundles/de.uniwue.mk.kall.athen.appDelegation/ressources/corefViewConfig.xml
0 → 100644
View file @
a9354772
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<nappiPipelineConfiguration>
<engines>
<configurationParams>
<entry>
<key>
PARAM_MODEL_LOCATION
</key>
<value
xmlns:xsi=
"http://www.w3.org/2001/XMLSchema-instance"
xmlns:xs=
"http://www.w3.org/2001/XMLSchema"
xsi:type=
"xs:string"
>
http://ki.informatik.uni-wuerzburg.de/nappi/nappiOnlineRepository/models/openNLP/sentence/de-sent.bin
</value>
</entry>
</configurationParams>
<engineClass>
OpenNLPSentenceSplitter
</engineClass>
<reader>
false
</reader>
</engines>
<engines>
<configurationParams>
<entry>
<key>
PARAM_MODEL_LOCATION
</key>
<value
xmlns:xsi=
"http://www.w3.org/2001/XMLSchema-instance"
xmlns:xs=
"http://www.w3.org/2001/XMLSchema"
xsi:type=
"xs:string"
>
http://ki.informatik.uni-wuerzburg.de/nappi/nappiOnlineRepository/models/openNLP/token/de-token.bin
</value>
</entry>
</configurationParams>
<engineClass>
OpenNLPTokenizer
</engineClass>
<reader>
false
</reader>
</engines>
<engines>
<configurationParams>
<entry>
<key>
PARAM_EXECUTABLE_LOCATION
</key>
<value
xmlns:xsi=
"http://www.w3.org/2001/XMLSchema-instance"
xmlns:xs=
"http://www.w3.org/2001/XMLSchema"
xsi:type=
"xs:string"
>
http://ki.informatik.uni-wuerzburg.de/nappi/nappiOnlineRepository/executables/treetagger/bin/tree-tagger.exe
</value>
</entry>
<entry>
<key>
PARAM_MODEL_LOCATION
</key>
<value
xmlns:xsi=
"http://www.w3.org/2001/XMLSchema-instance"
xmlns:xs=
"http://www.w3.org/2001/XMLSchema"
xsi:type=
"xs:string"
>
http://ki.informatik.uni-wuerzburg.de/nappi/nappiOnlineRepository/models/treetagger/pos/german-utf8.par
</value>
</entry>
</configurationParams>
<engineClass>
TreeTaggerLemmatizer
</engineClass>
<reader>
false
</reader>
</engines>
<engines>
<configurationParams>
<entry>
<key>
PARAM_EXECUTABLE_LOCATION
</key>
<value
xmlns:xsi=
"http://www.w3.org/2001/XMLSchema-instance"
xmlns:xs=
"http://www.w3.org/2001/XMLSchema"
xsi:type=
"xs:string"
>
http://ki.informatik.uni-wuerzburg.de/nappi/nappiOnlineRepository/executables/treetagger/bin/tree-tagger.exe
</value>
</entry>
<entry>
<key>
PARAM_MODEL_LOCATION
</key>
<value
xmlns:xsi=
"http://www.w3.org/2001/XMLSchema-instance"
xmlns:xs=
"http://www.w3.org/2001/XMLSchema"
xsi:type=
"xs:string"
>
http://ki.informatik.uni-wuerzburg.de/nappi/nappiOnlineRepository/models/treetagger/pos/german-utf8.par
</value>
</entry>
</configurationParams>
<engineClass>
TreeTaggerPOSTagger
</engineClass>
<reader>
false
</reader>
</engines>
<engines>
<configurationParams>
<entry>
<key>
PARAM_MODEL_LOCATION
</key>
<value
xmlns:xsi=
"http://www.w3.org/2001/XMLSchema-instance"
xmlns:xs=
"http://www.w3.org/2001/XMLSchema"
xsi:type=
"xs:string"
>
http://ki.informatik.uni-wuerzburg.de/nappi/nappiOnlineRepository/models/rft/german-pc-64bit-utf8.par
</value>
</entry>
</configurationParams>
<engineClass>
RFTTagger
</engineClass>
<reader>
false
</reader>
</engines>
<engines>
<configurationParams>
<entry>
<key>
PARAM_MODEL_LOCATION
</key>
<value
xmlns:xsi=
"http://www.w3.org/2001/XMLSchema-instance"
xmlns:xs=
"http://www.w3.org/2001/XMLSchema"
xsi:type=
"xs:string"
>
http://ki.informatik.uni-wuerzburg.de/nappi/nappiOnlineRepository/models/kallimachos/DROCTagger/nerAllModel.bin
</value>
</entry>
</configurationParams>
<engineClass>
DROCTagger
</engineClass>
<reader>
false
</reader>
</engines>
</nappiPipelineConfiguration>
de.uniwue.mk.athen/bundles/de.uniwue.mk.kall.athen.goldstandardAnalyzer/src/de/uniwue/mk/kall/athen/goldstandardAnalyzer/GoldstandardAnalyzer.java
View file @
a9354772
...
...
@@ -49,44 +49,44 @@ import de.uniwue.mk.kall.athen.part.editor.annotations.UIMAInfo.VariableType;
import
de.uniwue.mk.kall.athen.part.editor.subordinate.AEditorSubordinateViewPart
;
public
class
GoldstandardAnalyzer
extends
AEditorSubordinateViewPart
{
private
static
final
String
TP
=
"de.uniwue.kalimachos.coref.type.TP"
;
private
static
final
String
FP
=
"de.uniwue.kalimachos.coref.type.FP"
;
private
static
final
String
FN
=
"de.uniwue.kalimachos.coref.type.FN"
;
// the reference to the UI
private
GoldstandardAnalyzerWidget
widget
;
// the mappings
List
<
GoldMappingConfig
>
mappings
;
private
Type
currentSystemType
;
private
List
<
NAryTupel
>
content
;
private
StyleRange
styleRange
;
private
Map
<
AnnotationFS
,
String
>
m
=
new
HashMap
<
AnnotationFS
,
String
>();
private
Map
<
AnnotationFS
,
String
>
fpMap
=
new
HashMap
<
AnnotationFS
,
String
>();
private
Map
<
AnnotationFS
,
String
>
fnMap
=
new
HashMap
<
AnnotationFS
,
String
>();
private
Map
<
AnnotationFS
,
String
>
tpMap
=
new
HashMap
<
AnnotationFS
,
String
>();
private
GoldstandardAnalyzerDrawingStrategy
goldstandardAnalyzerDrawingStrategy
=
new
GoldstandardAnalyzerDrawingStrategy
(
styleRange
,
m
);
private
Map
<
AnnotationFS
,
String
>
m
=
new
HashMap
<
AnnotationFS
,
String
>();
private
Map
<
AnnotationFS
,
String
>
fpMap
=
new
HashMap
<
AnnotationFS
,
String
>();
private
Map
<
AnnotationFS
,
String
>
fnMap
=
new
HashMap
<
AnnotationFS
,
String
>();
private
Map
<
AnnotationFS
,
String
>
tpMap
=
new
HashMap
<
AnnotationFS
,
String
>();
private
GoldstandardAnalyzerDrawingStrategy
goldstandardAnalyzerDrawingStrategy
=
new
GoldstandardAnalyzerDrawingStrategy
(
styleRange
,
m
);
// UIMA Types, everything here will be injected by the annotation processor
@ATHENEditorDrawingInfo
(
mode
=
true
,
layer
=
3
)
@UIMAInfo
(
variableType
=
VariableType
.
TYPE
,
nameOrKey
=
TP
)
private
Type
tpType
;
@ATHENEditorDrawingInfo
(
mode
=
true
,
layer
=
2
)
@UIMAInfo
(
variableType
=
VariableType
.
TYPE
,
nameOrKey
=
FP
)
private
Type
fpType
;
@ATHENEditorDrawingInfo
(
mode
=
true
,
layer
=
1
)
@UIMAInfo
(
variableType
=
VariableType
.
TYPE
,
nameOrKey
=
FN
)
private
Type
fnType
;
private
List
<
AnnotationFS
>
annoList
;
//
private List<AnnotationFS> annoList;
@PostConstruct
public
void
postConstruct
(
Composite
parent
,
MPart
part
)
{
...
...
@@ -214,49 +214,47 @@ public class GoldstandardAnalyzer extends AEditorSubordinateViewPart {
@Override
public
boolean
initTypes
()
{
annoList
=
new
ArrayList
<
AnnotationFS
>();
// process the annotations on the fields
AnnotationProcessor
.
processUIMAInfoAnnotations
(
this
,
editor
.
getCas
());
AnnotationProcessor
.
processUIMAInfoAnnotations
(
this
,
editor
.
getCas
());
// set everything unneeded invisible - editor should only display what
// is needed in this view
this
.
editor
.
allStylesInvisible
();
// layer
editor
.
changeLayer
(
tpType
,
3
);
editor
.
changeLayer
(
fpType
,
2
);
editor
.
changeLayer
(
fnType
,
1
);
//true positive yellowgreen
//
true positive yellowgreen
StyleRange
rangeOv
=
new
StyleRange
();
rangeOv
.
background
=
new
Color
(
Display
.
getCurrent
(),
154
,
205
,
50
);
rangeOv
.
background
=
new
Color
(
Display
.
getCurrent
(),
154
,
205
,
50
);
AnnoStyle
tpStyle
=
new
AnnoStyle
(
new
GoldstandardAnalyzerDrawingStrategy
(
rangeOv
,
m
),
rangeOv
);
editor
.
changeStyle
(
tpType
,
tpStyle
);
editor
.
changeVisibility
(
tpType
,
true
);
editor
.
setActiveMode
(
tpType
);
//false positive firebrick
// false positive firebrick
StyleRange
rangeOv1
=
new
StyleRange
();
rangeOv1
.
background
=
new
Color
(
Display
.
getCurrent
(),
178
,
34
,
34
);
rangeOv1
.
background
=
new
Color
(
Display
.
getCurrent
(),
178
,
34
,
34
);
AnnoStyle
fpStyle
=
new
AnnoStyle
(
new
GoldstandardAnalyzerDrawingStrategy
(
rangeOv1
,
m
),
rangeOv1
);
editor
.
changeStyle
(
fpType
,
fpStyle
);
editor
.
changeVisibility
(
fpType
,
true
);
editor
.
setActiveMode
(
fpType
);
//false negative firebrick
//
false negative firebrick
StyleRange
rangeOv2
=
new
StyleRange
();
rangeOv2
.
background
=
new
Color
(
Display
.
getCurrent
(),
178
,
34
,
34
);
AnnoStyle
fnStyle
=
new
AnnoStyle
(
new
GoldstandardAnalyzerDrawingStrategy
(
rangeOv2
,
m
),
rangeOv2
);;
rangeOv2
.
background
=
new
Color
(
Display
.
getCurrent
(),
178
,
34
,
34
);
AnnoStyle
fnStyle
=
new
AnnoStyle
(
new
GoldstandardAnalyzerDrawingStrategy
(
rangeOv2
,
m
),
rangeOv2
);
;
editor
.
changeStyle
(
fnType
,
fnStyle
);
editor
.
changeVisibility
(
fnType
,
true
);
editor
.
setActiveMode
(
fnType
);
return
false
;
}
...
...
@@ -290,35 +288,34 @@ public class GoldstandardAnalyzer extends AEditorSubordinateViewPart {
}
}
private
void
deleteAnnotations
(
AnnotationFS
...
annos
)
{
List
<
AnnotationFS
>
toDelete
=
new
ArrayList
<>();
toDelete
.
addAll
(
Arrays
.
asList
(
annos
));
editor
.
removeAnnotations
(
toDelete
);
for
(
AnnotationFS
anno
:
toDelete
)
{
for
(
AnnotationFS
anno
:
toDelete
)
{
editor
.
getCas
().
removeFsFromIndexes
(
anno
);
}
}
public
void
deleteAnnotations
(
List
<
AnnotationFS
>
annoList
)
{
public
void
deleteAnnotations
()
{
List
<
AnnotationFS
>
annoList
=
new
ArrayList
<>();
AnnotationIndex
<
AnnotationFS
>
annotationIndex
=
editor
.
getCas
().
getAnnotationIndex
(
tpType
);
FSIterator
<
AnnotationFS
>
iterator
=
annotationIndex
.
iterator
();
while
(
iterator
.
hasNext
())
{
annoList
.
add
((
AnnotationFS
)
iterator
.
next
());
}
AnnotationIndex
<
AnnotationFS
>
annotationIndex1
=
editor
.
getCas
().
getAnnotationIndex
(
fpType
);
FSIterator
<
AnnotationFS
>
iterator1
=
annotationIndex1
.
iterator
();
while
(
iterator1
.
hasNext
())
{
annoList
.
add
((
AnnotationFS
)
iterator1
.
next
());
}
AnnotationIndex
<
AnnotationFS
>
annotationIndex2
=
editor
.
getCas
().
getAnnotationIndex
(
fnType
);
FSIterator
<
AnnotationFS
>
iterator2
=
annotationIndex2
.
iterator
();
while
(
iterator2
.
hasNext
())
{
...
...
@@ -326,11 +323,11 @@ public class GoldstandardAnalyzer extends AEditorSubordinateViewPart {
}
deleteAnnotations
(
annoList
.
toArray
(
new
AnnotationFS
[
0
]));
}
public
void
compareTypes
()
{
deleteAnnotations
(
annoList
);
deleteAnnotations
();
content
=
mappingsToContent
(
mappings
);
widget
.
refresh
(
content
);
if
(!
content
.
isEmpty
())
{
...
...
@@ -342,50 +339,51 @@ public class GoldstandardAnalyzer extends AEditorSubordinateViewPart {
EvaluationStruct
evalStruct
=
calculateEvalStruct
(
content
);
widget
.
updateScores
(
evalStruct
);
}
for
(
NAryTupel
tupel
:
content
){
if
(
tupel
.
getElementAt
(
0
)
==
null
){
for
(
NAryTupel
tupel
:
content
)
{
if
(
tupel
.
getElementAt
(
0
)
==
null
)
{
createFalsePositive
(
tupel
);
}
else
if
(
tupel
.
getElementAt
(
0
)
!=
null
){
else
if
(
tupel
.
getElementAt
(
0
)
!=
null
)
{
createFalseNegativeOrTruePositive
(
tupel
);
}
}
}
public
void
createFalsePositive
(
NAryTupel
t
){
for
(
int
k
=
1
;
k
<
t
.
getTupelDimension
();
k
++){
if
(
t
.
getElementAt
(
k
)
!=
null
){
AnnotationFS
fpAnnotation
=
editor
.
getCas
().
createAnnotation
(
fpType
,
t
.
getElementAt
(
k
).
getBegin
(),
t
.
getElementAt
(
k
).
getEnd
());
//annoList.add(fpAnnotation);
public
void
createFalsePositive
(
NAryTupel
t
)
{
for
(
int
k
=
1
;
k
<
t
.
getTupelDimension
();
k
++)
{
if
(
t
.
getElementAt
(
k
)
!=
null
)
{
AnnotationFS
fpAnnotation
=
editor
.
getCas
().
createAnnotation
(
fpType
,
t
.
getElementAt
(
k
).
getBegin
(),
t
.
getElementAt
(
k
).
getEnd
());
// annoList.add(fpAnnotation);
String
s
=
"Annotator"
+
k
;
fpMap
.
put
(
fpAnnotation
,
s
);
}
}
createMap
();
}
public
void
createFalseNegativeOrTruePositive
(
NAryTupel
t
){
public
void
createFalseNegativeOrTruePositive
(
NAryTupel
t
)
{
int
counter
=
0
;
for
(
int
l
=
1
;
l
<
t
.
getTupelDimension
();
l
++){
if
(
t
.
getElementAt
(
l
)
!=
null
){
for
(
int
l
=
1
;
l
<
t
.
getTupelDimension
();
l
++)
{
if
(
t
.
getElementAt
(
l
)
!=
null
)
{
counter
++;
}
}
if
(
counter
==
t
.
getTupelDimension
()-
1
){
createTruePositive
(
t
);
}
}
else
{
if
(
counter
==
t
.
getTupelDimension
()
-
1
)
{
createTruePositive
(
t
);
}
else
{
createFalseNegative
(
t
);
}
}
public
void
createFalseNegative
(
NAryTupel
t
){
for
(
int
k
=
0
;
k
<
t
.
getTupelDimension
();
k
++){
if
(
t
.
getElementAt
(
k
)
!=
null
){
AnnotationFS
fnAnnotation
=
editor
.
getCas
().
createAnnotation
(
fnType
,
t
.
getElementAt
(
k
).
getBegin
(),
t
.
getElementAt
(
k
).
getEnd
());
//annoList.add(fnAnnotation);
public
void
createFalseNegative
(
NAryTupel
t
)
{
for
(
int
k
=
0
;
k
<
t
.
getTupelDimension
();
k
++)
{
if
(
t
.
getElementAt
(
k
)
!=
null
)
{
AnnotationFS
fnAnnotation
=
editor
.
getCas
().
createAnnotation
(
fnType
,
t
.
getElementAt
(
k
).
getBegin
(),
t
.
getElementAt
(
k
).
getEnd
());
// annoList.add(fnAnnotation);
String
s
=
"Annotator"
+
k
;
fnMap
.
put
(
fnAnnotation
,
s
);
}
...
...
@@ -393,26 +391,28 @@ public class GoldstandardAnalyzer extends AEditorSubordinateViewPart {
createMap
();
}
public
void
createTruePositive
(
NAryTu
pe
l
t
){
AnnotationFS
tpAnnotation
=
editor
.
getCas
().
createAnnotation
(
tpType
,
t
.
getElementAt
(
0
).
getBegin
(),
t
.
getElementAt
(
0
).
getEnd
());
//annoList.add(tpAnnotation);
public
void
createTruePositive
(
NAryTupel
t
)
{
AnnotationFS
tpAnnotation
=
editor
.
getCas
().
createAnnotation
(
tpTy
pe
,
t
.
getElementAt
(
0
).
getBegin
(),
t
.
getElementAt
(
0
).
getEnd
());
//
annoList.add(tpAnnotation);
String
s
=
"Annotator"
+
"tp"
;
tpMap
.
put
(
tpAnnotation
,
s
);
createMap
();
}
public
void
createMap
()
{
public
void
createMap
()
{
m
.
putAll
(
tpMap
);
m
.
putAll
(
fpMap
);
m
.
putAll
(
fnMap
);
goldstandardAnalyzerDrawingStrategy
.
updateAnnotations
(
m
);
for
(
AnnotationFS
anno
:
m
.
keySet
())
{
editor
.
getCas
().
addFsToIndexes
(
anno
);
if
(
anno
!=
null
||
!(
anno
instanceof
AnnotationFS
))
{
editor
.
getCas
().
addFsToIndexes
(
anno
);
}
else
{
System
.
out
.
println
(
"How can this even happen! A null annotation!"
);
}
}
editor
.
updateWidget
();
}
...
...
de.uniwue.mk.athen/bundles/de.uniwue.mk.kall.athen.goldstandardAnalyzer/src/de/uniwue/mk/kall/athen/goldstandardAnalyzer/struct/NAryTupel.java
View file @
a9354772
...
...
@@ -65,6 +65,7 @@ public class NAryTupel implements Iterable<AnnotationFS> {
public
void
insert
(
AnnotationFS
anno
)
{
if
(
anno
!=
null
)
{
//TODO this cannot deal with comparisons of the same type
Integer
integer
=
typeToIndexMap
.
get
(
anno
.
getType
());
tupelEntries
.
set
(
integer
,
anno
);
}
...
...
de.uniwue.mk.athen/bundles/de.uniwue.mk.kall.athen.goldstandardAnalyzer/src/de/uniwue/mk/kall/athen/goldstandardAnalyzer/ui/GoldstandardAnalyzerDrawingStrategy.java
View file @
a9354772
...
...
@@ -53,6 +53,9 @@ public class GoldstandardAnalyzerDrawingStrategy implements IAnnotationDrawingSt
if
(
m
.
containsKey
(
anno
));
{
annotator
=
m
.
get
(
anno
);
if
(
annotator
==
null
){
return
;
}
if
(
annotator
.
equals
(
"Annotatortp"
)){
gc
.
setBackground
(
yellowgreen
);
}
else
if
(
annotator
.
equals
(
"Annotator0"
)){
...
...
de.uniwue.mk.athen/bundles/de.uniwue.mk.kall.athen.goldstandardAnalyzer/src/de/uniwue/mk/kall/athen/goldstandardAnalyzer/util/AnnotationComparisonUtil.java
View file @
a9354772
...
...
@@ -43,6 +43,8 @@ public class AnnotationComparisonUtil {
/*
* Check Gold Annotations
*/
System
.
out
.
println
(
goldList
.
size
());
System
.
out
.
println
(
systemRemaining
.
size
());
goldList
.
forEach
(
goldAnno
->
{
AnnotationFS
foundAnno
=
null
;
for
(
AnnotationFS
systemAnno
:
systemRemaining
)
{
...
...
de.uniwue.mk.athen/bundles/de.uniwue.mk.kall.athen.goldstandardAnalyzer/src/de/uniwue/mk/kall/athen/goldstandardAnalyzer/widgets/GoldStandardTreeTableComposite.java
View file @
a9354772
...
...
@@ -112,6 +112,8 @@ public class GoldStandardTreeTableComposite extends Composite {
public
void
refresh
(
List
<
NAryTupel
>
content
)
{
if
(
content
==
null
)
return
;
createTreeTable
(
content
);
addTreeTableSelectionListener
();
...
...
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment