---
license: mit
base_model: roberta-base
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: default
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# default

This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2050
- Accuracy: 0.9350

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 128
- eval_batch_size: 512
- seed: 42
- distributed_type: multi-GPU
- gradient_accumulation_steps: 4
- total_train_batch_size: 512
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 10000
- num_epochs: 3

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 1.0664        | 0.06  | 20   | 1.0651          | 0.4448   |
| 1.0423        | 0.12  | 40   | 1.0188          | 0.5034   |
| 1.0137        | 0.18  | 60   | 0.9871          | 0.5279   |
| 1.0027        | 0.24  | 80   | 0.9889          | 0.5308   |
| 0.9914        | 0.3   | 100  | 0.9763          | 0.5308   |
| 0.9826        | 0.36  | 120  | 0.9713          | 0.5388   |
| 0.9788        | 0.42  | 140  | 0.9766          | 0.5313   |
| 0.984         | 0.48  | 160  | 0.9590          | 0.5398   |
| 0.9694        | 0.54  | 180  | 0.9535          | 0.5423   |
| 0.9676        | 0.6   | 200  | 0.9274          | 0.5672   |
| 0.9753        | 0.66  | 220  | 0.9126          | 0.5736   |
| 0.9557        | 0.72  | 240  | 0.9053          | 0.5760   |
| 0.9508        | 0.78  | 260  | 0.9179          | 0.5767   |
| 0.9355        | 0.84  | 280  | 0.8937          | 0.5892   |
| 0.9           | 0.9   | 300  | 0.8469          | 0.6130   |
| 0.993         | 0.96  | 320  | 0.8615          | 0.6047   |
| 0.8527        | 1.02  | 340  | 0.7896          | 0.6439   |
| 0.966         | 1.08  | 360  | 1.0124          | 0.5316   |
| 0.8441        | 1.14  | 380  | 0.7911          | 0.6489   |
| 0.8226        | 1.2   | 400  | 0.7472          | 0.6700   |
| 0.7948        | 1.26  | 420  | 0.7664          | 0.6581   |
| 0.7428        | 1.32  | 440  | 0.6994          | 0.6992   |
| 0.7109        | 1.38  | 460  | 0.6511          | 0.7284   |
| 0.6882        | 1.44  | 480  | 0.5988          | 0.7577   |
| 0.7296        | 1.5   | 500  | 0.5993          | 0.7564   |
| 0.5677        | 1.57  | 520  | 0.5068          | 0.8126   |
| 0.5096        | 1.63  | 540  | 0.4273          | 0.8520   |
| 0.4452        | 1.69  | 560  | 0.3796          | 0.8722   |
| 0.3836        | 1.75  | 580  | 0.3855          | 0.8757   |
| 0.3783        | 1.81  | 600  | 0.3586          | 0.8894   |
| 0.3496        | 1.87  | 620  | 0.3210          | 0.8972   |
| 0.3585        | 1.93  | 640  | 0.3006          | 0.9035   |
| 0.345         | 1.99  | 660  | 0.3054          | 0.9014   |
| 0.3327        | 2.05  | 680  | 0.3174          | 0.8913   |
| 0.2962        | 2.11  | 700  | 0.2770          | 0.9122   |
| 0.3032        | 2.17  | 720  | 0.2979          | 0.9062   |
| 0.27          | 2.23  | 740  | 0.2973          | 0.8998   |
| 0.2912        | 2.29  | 760  | 0.2467          | 0.9222   |
| 0.2412        | 2.35  | 780  | 0.2761          | 0.9113   |
| 0.2746        | 2.41  | 800  | 0.2410          | 0.9260   |
| 0.2637        | 2.47  | 820  | 0.2447          | 0.9221   |
| 0.2605        | 2.53  | 840  | 0.2475          | 0.9237   |
| 0.2517        | 2.59  | 860  | 0.2590          | 0.9265   |
| 0.2453        | 2.65  | 880  | 0.2248          | 0.9300   |
| 0.2247        | 2.71  | 900  | 0.2285          | 0.9273   |
| 0.2402        | 2.77  | 920  | 0.2304          | 0.9306   |
| 0.2033        | 2.83  | 940  | 0.2228          | 0.9319   |
| 0.2315        | 2.89  | 960  | 0.2275          | 0.9271   |
| 0.2231        | 2.95  | 980  | 0.2115          | 0.9343   |


### Framework versions

- Transformers 4.34.1
- Pytorch 2.1.0
- Datasets 2.14.5
- Tokenizers 0.14.1