Training in progress, step 500

README.md

@@ -0,0 +1,88 @@
+---
+license: mit
+base_model: FacebookAI/roberta-large
+tags:
+- generated_from_trainer
+metrics:
+- accuracy
+- precision
+- recall
+- f1
+model-index:
+- name: absa-train-service-roberta-large
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/cunho2803032003/absa-1721959498.2993438/runs/tad25dun)
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/cunho2803032003/absa-1721959940.7872202/runs/bsprskdy)
+# absa-train-service-roberta-large
+
+This model is a fine-tuned version of [FacebookAI/roberta-large](https://huggingface.co/FacebookAI/roberta-large) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.8683
+- Accuracy: 0.7424
+- Precision: 0.7345
+- Recall: 0.7367
+- F1: 0.7302
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 0.0002
+- train_batch_size: 8
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 500
+- num_epochs: 20
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Accuracy | Precision | Recall | F1     |
+|:-------------:|:-----:|:----:|:---------------:|:--------:|:---------:|:------:|:------:|
+| 2.2255        | 1.0   | 469  | 2.0677          | 0.3296   | 0.1937    | 0.3250 | 0.2297 |
+| 1.8236        | 2.0   | 938  | 1.7061          | 0.504    | 0.5413    | 0.4914 | 0.4567 |
+| 1.5384        | 3.0   | 1407 | 1.4381          | 0.552    | 0.5944    | 0.5549 | 0.5196 |
+| 1.4301        | 4.0   | 1876 | 1.3316          | 0.5984   | 0.6000    | 0.5990 | 0.5618 |
+| 1.3776        | 5.0   | 2345 | 1.1645          | 0.6576   | 0.6817    | 0.6491 | 0.6332 |
+| 1.2078        | 6.0   | 2814 | 1.0967          | 0.6448   | 0.7035    | 0.6348 | 0.6110 |
+| 1.2535        | 7.0   | 3283 | 1.0565          | 0.7008   | 0.7467    | 0.6967 | 0.7066 |
+| 1.2921        | 8.0   | 3752 | 1.0049          | 0.6976   | 0.7013    | 0.6884 | 0.6813 |
+| 1.178         | 9.0   | 4221 | 1.0438          | 0.648    | 0.7746    | 0.6423 | 0.6387 |
+| 1.2324        | 10.0  | 4690 | 1.0203          | 0.6896   | 0.7096    | 0.6831 | 0.6704 |
+| 1.1899        | 11.0  | 5159 | 1.0193          | 0.6864   | 0.7391    | 0.6819 | 0.6834 |
+| 1.1515        | 12.0  | 5628 | 0.9722          | 0.6944   | 0.7164    | 0.6924 | 0.6860 |
+| 1.1604        | 13.0  | 6097 | 0.9372          | 0.7312   | 0.7543    | 0.7311 | 0.7259 |
+| 1.1229        | 14.0  | 6566 | 0.9265          | 0.72     | 0.7278    | 0.7139 | 0.7147 |
+| 1.1459        | 15.0  | 7035 | 0.8896          | 0.7376   | 0.7264    | 0.7323 | 0.7183 |
+| 1.1281        | 16.0  | 7504 | 0.9074          | 0.7152   | 0.7107    | 0.7087 | 0.7012 |
+| 1.1794        | 17.0  | 7973 | 0.8914          | 0.7424   | 0.7293    | 0.7354 | 0.7266 |
+| 1.1101        | 18.0  | 8442 | 0.8707          | 0.7216   | 0.7161    | 0.7141 | 0.7059 |
+| 1.1215        | 19.0  | 8911 | 0.8656          | 0.7408   | 0.7322    | 0.7348 | 0.7274 |
+| 1.0483        | 20.0  | 9380 | 0.8683          | 0.7424   | 0.7345    | 0.7367 | 0.7302 |
+
+
+### Framework versions
+
+- Transformers 4.43.2
+- Pytorch 2.3.1+cu121
+- Datasets 2.20.0
+- Tokenizers 0.19.1

config.json

@@ -0,0 +1,53 @@
+{
+  "_name_or_path": "google-bert/bert-base-uncased",
+  "architectures": [
+    "BertForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "LABEL_0",
+    "1": "LABEL_1",
+    "2": "LABEL_2",
+    "3": "LABEL_3",
+    "4": "LABEL_4",
+    "5": "LABEL_5",
+    "6": "LABEL_6",
+    "7": "LABEL_7",
+    "8": "LABEL_8",
+    "9": "LABEL_9",
+    "10": "LABEL_10"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "LABEL_0": 0,
+    "LABEL_1": 1,
+    "LABEL_10": 10,
+    "LABEL_2": 2,
+    "LABEL_3": 3,
+    "LABEL_4": 4,
+    "LABEL_5": 5,
+    "LABEL_6": 6,
+    "LABEL_7": 7,
+    "LABEL_8": 8,
+    "LABEL_9": 9
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "problem_type": "single_label_classification",
+  "torch_dtype": "float32",
+  "transformers_version": "4.43.3",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa5fa450964384897e4e1f3951412531de915750de9708d2a5752ef6b22b19bb
+size 437986332

special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,55 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8698313c9987e0f0f82120c4351eab3d2efa49862c5da38da872974451d727af
+size 5304