File size: 2,095 Bytes
cf390d4 c3ba182 cf390d4 c3ba182 cf390d4 c3ba182 066d8b5 c3ba182 066d8b5 7d40a0e 066d8b5 7d40a0e 066d8b5 7d40a0e 066d8b5 c3ba182 7d40a0e c3ba182 066d8b5 c3ba182 c7af39a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 |
---
language: et
license: cc-by-4.0
datasets:
- ERRnews
---
# mBART ERRnews
Pretrained mbart-large-cc25 model finetuned on ERRnews Estonian news story dataset.
## How to use
Here is how to use this model to get a summary of a given text in PyTorch:
```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained("TalTechNLP/mBART-ERRnews")
model = AutoModelForSeq2SeqLM.from_pretrained("TalTechNLP/mBART-ERRnews")
text = "Riigikogu rahanduskomisjon võttis esmaspäeval maha riigieelarvesse esitatud investeeringuettepanekutest siseministeeriumi investeeringud koolidele ja lasteaedadele, sest komisjoni hinnangul ei peaks siseministeerium tegelema investeeringutega väljaspoole oma vastutusala. Komisjoni esimees Aivar Kokk ütles, et komisjon lähtus otsuse tegemisel riigikontrolör Janar Holmi soovitusest ja seadustest."
inputs = tokenizer(text, return_tensors='pt', max_length=1024)
summary_ids = model.generate(inputs['input_ids'])
summary = [tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=False) for g in summary_ids]
```
## Training data
The mBART model was finetuned on [ERRnews](https://huggingface.co/datasets/TalTechNLP/ERRnews), a dataset consisting of 10 420
Estonian news story transcripts and summaries.
### Training
The model was trained on 2 cloud GPUs with a batch size of 16 for 16 epochs. The optimizer
used is Adam with a learning rate of 5e-05, betas of 0.9 and 0.999.
## Evaluation results
This model achieves the following results:
| Dataset | ROUGE-1 | ROUGE-2 | ROUGE-L | ROUGE-L-SUM |
|:-------:|:-------:|:-------:|:-------:|:-----------:|
| ERRnews | 19.2 | 6.7 | 16.1 | 17.4 |
### BibTeX entry and citation info
```bibtex
article{henryabstractive,
title={Abstractive Summarization of Broadcast News Stories for {Estonian}},
author={Henry, H{\"a}rm and Tanel, Alum{\"a}e},
journal={Baltic J. Modern Computing},
volume={10},
number={3},
pages={511-524},
year={2022}
}
``` |