---
language: 
  - en
tags:
- classics
- citation mining
widget:
- text: "Homer's Iliad opens with an invocation to the muse (1. 1)."
---

### Model and entities
`roberta_classics_ner` is a domain-specific RoBERTa-based model for named entity recognition in Classical Studies. It recognises bibliographical entities, such as:


| id  | label         | desciption                                  | Example               |
| --- | ------------- | ------------------------------------------- | --------------------- |
| 0   | 'O'           | Out of entity                               |                       | 
| 1   | 'B-AAUTHOR'   | Ancient authors                             | *Herodotus*           |
| 2   | 'I-AAUTHOR'   |                                             |                       |
| 3   | 'B-AWORK'     | The title of an ancient work                | *Symposium*, *Aeneid* |
| 4   | 'I-AWORK'     |                                             |                       |
| 5   | 'B-REFAUWORK' | A structured reference to an ancient work   | *Homer, Il.*          |
| 6   | 'I-REFAUWORK' |                                             |                       |
| 7   | 'B-REFSCOPE'  | The scope of a reference                    | *II.1.993a30–b11*     |
| 8   | 'I-REFSCOPE'  |                                             |                       |
| 9   | 'B-FRAGREF'   | A reference to fragmentary texts or scholia | *Frag. 19. West*      |
| 10  | 'I-FRAGREF'   |                                             |                       |


### Example


```
B-AAUTHOR   B-AWORK                                      B-REFSCOPE
Homer  's   Iliad opens with an invocation to the muse ( 1. 1).
```



### Dataset


`roberta_classics_ner` was fine-tuned and evaluated on `EpiBau`, a dataset which has not been released publicly yet. It is composed of four volumes of [Structures of Epic Poetry](https://www.epische-bauformen.uni-rostock.de/), a compendium on the narrative patterns and structural elements in ancient epic. 
Entity counts of the `Epibau` dataset are the following: 

|                | train-set | dev-set | test-set |
| -------------- | --------- | ------- | -------- |
| word count     | 712462    | 125729  | 122324   |
| AAUTHOR        | 4436      | 1368    | 1511     |
| AWORK          | 3145      | 780     | 670      |
| REFAUWORK      | 5102      | 988     | 1209     |
| REFSCOPE       | 14768     | 3193    | 2847     |
| FRAGREF        | 266       | 29      | 33       |
| total entities | 13822     | 1415    | 2419     |


### Results

The model was developed in the context of experiments reported [here](http://infoscience.epfl.ch/record/291236?&ln=en).Trained and tested on `EpiBau` with a 85-15 split, the model yields a general F1 score of **.82** (micro-averages). Detailed scores are displayed below. Evaluation was performed with the [CLEF-HIPE-scorer](https://github.com/impresso/CLEF-HIPE-2020-scorer), in strict mode) 


| metric    | AAUTHOR | AWORK | REFSCOPE | REFAUWORK |
| --------- | ------- | ----- | -------- | --------- |
| F1        | .819    | .796  | .863     | .756      |
| Precision | .842    | .818  | .860     | .755      |
| Recall    | .797    | .766  | .756     | .866      | 




Questions, remarks, help or contribution ? Get in touch [here](https://github.com/AjaxMultiCommentary), we'll be happy to chat !