danieldux commited on
Commit
889c3a5
1 Parent(s): a1dd3b0

Update metric description in README.md

Browse files
Files changed (2) hide show
  1. README.md +0 -1
  2. metric_template_1.py +1 -30
README.md CHANGED
@@ -17,7 +17,6 @@ pinned: false
17
  ***Module Card Instructions:*** *Fill out the following subsections. Feel free to take a look at existing metric cards if you'd like examples.*
18
 
19
  ## Metric Description
20
- The ISCO-08 Hierarchical Accuracy Measure is an implementation of the measure described in [Functional Annotation of Genes Using Hierarchical Text Categorization](https://www.researchgate.net/publication/44046343_Functional_Annotation_of_Genes_Using_Hierarchical_Text_Categorization) (Kiritchenko, Svetlana and Famili, Fazel. 2005) and adapted for the ISCO-08 classification scheme by the International Labour Organization.
21
 
22
  The measure accounts for the hierarchical structure of the ISCO-08 classification scheme by rewarding partial correctness in classification, penalizing distant errors, and applying higher penalties for higher-level errors. It is designed to evaluate the accuracy of predictions within a hierarchical occupation classification system, such as the ISCO-08 scheme, which organizes occupations into a tree structure with four levels: major group, sub-major group, minor group, and unit group.
23
 
 
17
  ***Module Card Instructions:*** *Fill out the following subsections. Feel free to take a look at existing metric cards if you'd like examples.*
18
 
19
  ## Metric Description
 
20
 
21
  The measure accounts for the hierarchical structure of the ISCO-08 classification scheme by rewarding partial correctness in classification, penalizing distant errors, and applying higher penalties for higher-level errors. It is designed to evaluate the accuracy of predictions within a hierarchical occupation classification system, such as the ISCO-08 scheme, which organizes occupations into a tree structure with four levels: major group, sub-major group, minor group, and unit group.
22
 
metric_template_1.py CHANGED
@@ -35,36 +35,7 @@ _CITATION = """
35
  """
36
 
37
  _DESCRIPTION = """
38
- The ISCO-08 Hierarchical Accuracy Measure is an implementation
39
- of the measure described in [Functional Annotation of Genes Using Hierarchical Text Categorization](https://www.researchgate.net/publication/44046343_Functional_Annotation_of_Genes_Using_Hierarchical_Text_Categorization)
40
- (Kiritchenko, Svetlana and Famili, Fazel. 2005) with the ISCO-08 taxonomy by the International Labour Organization.
41
-
42
- 1. The measure gives credit to partially correct classification,
43
- e.g. misclassification into node $I$ (ISCO unit group "1120")
44
- when the correct category is $G$ (ISCO unit group "1111")
45
- should be penalized less than misclassification into node $D$
46
- (e.g., ISCO unit group "1211") since $I$ is in the same subgraph (ISCO sub-major group "11")
47
- as $G$ and $D$ is not.
48
- 2. The measure punishes distant errors more heavily:
49
- 1. the measure gives higher evaluation for correctly classifying one level down compared to staying at the parent node, e.g. classification into node $E$ (ISCO minor group "111") is better than classification into its parent $C$ (ISCO sub-major group "11") since $E$ is closer to the correct category $G$;
50
- 2. the measure gives lower evaluation for incorrectly classifying one level down comparing to staying at the parent node, e.g. classification into node $F$ (ISCO minor group "112") is worse than classification into its parent $C$ since $F$ is farther away from $G$.
51
-
52
- The features described are accomplished by pairing hierarchical variants of precision ($hP$) and recall ($hR$) to form a hierarchical F1 ($hF_β$) score where each sample belongs not only to its class (e.g., a unit group level code), but also to all ancestors of the class in a hierarchical graph (i.e., the minor, sub-major, and major group level codes).
53
-
54
- Hierarchical precision can be computed with:
55
- `$hP = \frac{| \v{C}_i ∩ \v{C}^′_i|} {|\v{C}^′_i |} = \frac{1}{2}$`
56
-
57
- Hierarchical recall can be computed with:
58
- `$hR = \frac{| \v{C}_i ∩ \v{C}^′_i|} {|\v{C}_i |} = \frac{1}{2}$`
59
-
60
- Combining the two values $hP$ and $hR$ into one hF-measure:
61
- `$hF_β = \frac{(β^2 + 1) · hP · hR}{(β^2 · hP + hR)}, β ∈ [0, +∞)$`
62
-
63
- Note:
64
- **TP**: True positive
65
- **TN**: True negative
66
- **FP**: False positive
67
- **FN**: False negative
68
  """
69
 
70
  _KWARGS_DESCRIPTION = """
 
35
  """
36
 
37
  _DESCRIPTION = """
38
+ The ISCO-08 Hierarchical Accuracy Measure is an implementation of the measure described in [Functional Annotation of Genes Using Hierarchical Text Categorization](https://www.researchgate.net/publication/44046343_Functional_Annotation_of_Genes_Using_Hierarchical_Text_Categorization) (Kiritchenko, Svetlana and Famili, Fazel. 2005) and adapted for the ISCO-08 classification scheme by the International Labour Organization.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
39
  """
40
 
41
  _KWARGS_DESCRIPTION = """