---
license: mit
base_model: FacebookAI/xlm-roberta-base
tags:
- generated_from_trainer
metrics:
- accuracy
- precision
- recall
- f1
model-index:
- name: gg-bert-base-uncased
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# gg-bert-base-uncased

This model is a fine-tuned version of [FacebookAI/xlm-roberta-base](https://huggingface.co/FacebookAI/xlm-roberta-base) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.7791
- Accuracy: 0.752
- Precision: 0.7388
- Recall: 0.7570
- F1: 0.7396

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 25

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy | Precision | Recall | F1     |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|:---------:|:------:|:------:|
| 2.2541        | 1.0   | 469   | 2.2063          | 0.1488   | 0.2698    | 0.1552 | 0.1036 |
| 1.8967        | 2.0   | 938   | 1.8773          | 0.5168   | 0.5264    | 0.5331 | 0.4788 |
| 1.5747        | 3.0   | 1407  | 1.5546          | 0.5984   | 0.6125    | 0.6118 | 0.5636 |
| 1.4206        | 4.0   | 1876  | 1.3029          | 0.6528   | 0.6732    | 0.6666 | 0.6224 |
| 1.2804        | 5.0   | 2345  | 1.1876          | 0.6928   | 0.6972    | 0.6989 | 0.6844 |
| 1.1587        | 6.0   | 2814  | 1.0644          | 0.7136   | 0.7105    | 0.7136 | 0.6858 |
| 1.1589        | 7.0   | 3283  | 0.9883          | 0.7216   | 0.7173    | 0.7261 | 0.7031 |
| 1.0745        | 8.0   | 3752  | 0.9485          | 0.728    | 0.7151    | 0.7318 | 0.7195 |
| 1.0348        | 9.0   | 4221  | 0.9278          | 0.7328   | 0.7306    | 0.7372 | 0.7166 |
| 1.0019        | 10.0  | 4690  | 0.9114          | 0.72     | 0.7316    | 0.7231 | 0.7006 |
| 1.0204        | 11.0  | 5159  | 0.8967          | 0.7152   | 0.7187    | 0.7215 | 0.6895 |
| 1.0651        | 12.0  | 5628  | 0.8574          | 0.7424   | 0.7446    | 0.7474 | 0.7327 |
| 0.9841        | 13.0  | 6097  | 0.8461          | 0.7328   | 0.7495    | 0.7370 | 0.7076 |
| 0.9794        | 14.0  | 6566  | 0.8510          | 0.7248   | 0.7157    | 0.7319 | 0.7022 |
| 1.0242        | 15.0  | 7035  | 0.8127          | 0.7264   | 0.7112    | 0.7300 | 0.6998 |
| 0.9614        | 16.0  | 7504  | 0.8146          | 0.7312   | 0.7210    | 0.7376 | 0.7149 |
| 0.9358        | 17.0  | 7973  | 0.8288          | 0.736    | 0.7487    | 0.7439 | 0.7275 |
| 0.9719        | 18.0  | 8442  | 0.7958          | 0.7488   | 0.7403    | 0.7530 | 0.7349 |
| 0.9159        | 19.0  | 8911  | 0.7973          | 0.7472   | 0.7388    | 0.7522 | 0.7357 |
| 0.9824        | 20.0  | 9380  | 0.7921          | 0.7504   | 0.7439    | 0.7562 | 0.7363 |
| 1.0215        | 21.0  | 9849  | 0.7831          | 0.7536   | 0.7415    | 0.7586 | 0.7392 |
| 0.9191        | 22.0  | 10318 | 0.7780          | 0.7504   | 0.7387    | 0.7554 | 0.7399 |
| 0.9087        | 23.0  | 10787 | 0.7843          | 0.7472   | 0.7352    | 0.7536 | 0.7345 |
| 0.9198        | 24.0  | 11256 | 0.7793          | 0.7504   | 0.7358    | 0.7554 | 0.7374 |
| 0.9162        | 25.0  | 11725 | 0.7791          | 0.752    | 0.7388    | 0.7570 | 0.7396 |


### Framework versions

- Transformers 4.43.3
- Pytorch 2.3.1+cu121
- Datasets 2.20.0
- Tokenizers 0.19.1