Training complete!

README.md

@@ -0,0 +1,92 @@
+---
+license: apache-2.0
+base_model: EleutherAI/pythia-410m-deduped-v0
+tags:
+- generated_from_trainer
+model-index:
+- name: eleuter-foodie-test-2
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# eleuter-foodie-test-2
+
+This model is a fine-tuned version of [EleutherAI/pythia-410m-deduped-v0](https://huggingface.co/EleutherAI/pythia-410m-deduped-v0) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 1.3148
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 0.0003
+- train_batch_size: 4
+- eval_batch_size: 4
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: constant
+- lr_scheduler_warmup_ratio: 0.03
+- num_epochs: 2
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| 1.3434        | 0.06  | 200  | 1.4594          |
+| 1.2535        | 0.11  | 400  | 1.4383          |
+| 1.2794        | 0.17  | 600  | 1.4265          |
+| 1.2913        | 0.22  | 800  | 1.4185          |
+| 1.2776        | 0.28  | 1000 | 1.4054          |
+| 1.2929        | 0.34  | 1200 | 1.3950          |
+| 1.2357        | 0.39  | 1400 | 1.3975          |
+| 1.2235        | 0.45  | 1600 | 1.3886          |
+| 1.2609        | 0.5   | 1800 | 1.3854          |
+| 1.2002        | 0.56  | 2000 | 1.3774          |
+| 1.2729        | 0.61  | 2200 | 1.3745          |
+| 1.2143        | 0.67  | 2400 | 1.3680          |
+| 1.2011        | 0.73  | 2600 | 1.3670          |
+| 1.2697        | 0.78  | 2800 | 1.3625          |
+| 1.2499        | 0.84  | 3000 | 1.3593          |
+| 1.2409        | 0.89  | 3200 | 1.3536          |
+| 1.2393        | 0.95  | 3400 | 1.3497          |
+| 1.3622        | 1.01  | 3600 | 1.3411          |
+| 1.2965        | 1.06  | 3800 | 1.3408          |
+| 1.3515        | 1.12  | 4000 | 1.3366          |
+| 1.3451        | 1.17  | 4200 | 1.3359          |
+| 1.3427        | 1.23  | 4400 | 1.3343          |
+| 1.3735        | 1.28  | 4600 | 1.3344          |
+| 1.348         | 1.34  | 4800 | 1.3322          |
+| 1.3536        | 1.4   | 5000 | 1.3315          |
+| 1.284         | 1.45  | 5200 | 1.3281          |
+| 1.2912        | 1.51  | 5400 | 1.3272          |
+| 1.3119        | 1.56  | 5600 | 1.3261          |
+| 1.3422        | 1.62  | 5800 | 1.3221          |
+| 1.3139        | 1.68  | 6000 | 1.3201          |
+| 1.3222        | 1.73  | 6200 | 1.3181          |
+| 1.3038        | 1.79  | 6400 | 1.3181          |
+| 1.296         | 1.84  | 6600 | 1.3169          |
+| 1.3562        | 1.9   | 6800 | 1.3166          |
+| 1.2887        | 1.95  | 7000 | 1.3148          |
+
+
+### Framework versions
+
+- Transformers 4.32.0
+- Pytorch 2.0.1+cu118
+- Datasets 2.14.4
+- Tokenizers 0.13.3

adapter_config.json

@@ -0,0 +1,20 @@
+{
+  "auto_mapping": null,
+  "base_model_name_or_path": "EleutherAI/pythia-410m-deduped-v0",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "revision": null,
+  "target_modules": [
+    "query_key_value"
+  ],
+  "task_type": "CAUSAL_LM"
+}

adapter_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b25efb90e09e96673b66064a04a0173c530e7f0f575b49eb6fcfa3d8b80d1012
+size 6308993

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c152074a486243089e4fc0fdee0a373a30fb0e0a6e40eb5fd0d36fdafc97a155
+size 443

special_tokens_map.json

@@ -0,0 +1,6 @@
+{
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "pad_token": "<|endoftext|>",
+  "unk_token": "<|endoftext|>"
+}

tokenizer_config.json

@@ -0,0 +1,9 @@
+{
+  "add_prefix_space": false,
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|endoftext|>",
+  "model_max_length": 1000000000000000019884624838656,
+  "tokenizer_class": "GPTNeoXTokenizer",
+  "unk_token": "<|endoftext|>"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9535bd7f3a5914d20c374be47528c94e0cb3873b2d5fc46d26cd2c95bd86aa68
+size 4027