End of training

README.md

@@ -0,0 +1,60 @@
+---
+license: mit
+base_model: microsoft/deberta-base
+tags:
+- generated_from_trainer
+model-index:
+- name: Augmented-PreTraining-MLM
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# Augmented-PreTraining-MLM
+
+This model is a fine-tuned version of [microsoft/deberta-base](https://huggingface.co/microsoft/deberta-base) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 3.1688
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 8
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 3.0
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| 5.1204        | 1.0   | 445  | 3.9255          |
+| 3.685         | 2.0   | 890  | 3.3359          |
+| 3.3332        | 3.0   | 1335 | 3.1859          |
+
+
+### Framework versions
+
+- Transformers 4.40.1
+- Pytorch 2.2.1+cu121
+- Datasets 2.19.1
+- Tokenizers 0.19.1

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c5bbc274bf612b69719c30a315591525a2ae897da6530cc4c1b56196c7bd284b
+oid sha256:605186a8d003d7ad487a5903331de2c5ac92d06cce080960df790d9da81ade5c
 size 557000804

runs/May06_16-49-46_957dbf472760/events.out.tfevents.1715014282.957dbf472760.10287.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c46c5578ea8e00bdc722ccb6b98b41e3ed5f10c9ec04df0c1287a2586e961354
-size 5932
+oid sha256:5253ebaa32161a7c310f317e9a1ab0fa26f2bf2228f53e960c1c74f7621c302b
+size 6768

runs/May06_16-49-46_957dbf472760/events.out.tfevents.1715015403.957dbf472760.10287.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3e61d062236dad3f3086d56fac3857e9993f87f6a1dd8096e3663f763d5a95dc
+size 359