Model save

README.md

@@ -6,35 +6,30 @@ tags:
 - generated_from_trainer
 - trl
 - sft
-license: apache-2.0
-datasets:
-- OpenAssistant/oasst1
-- allenai/c4
+licence: license
 ---
 
-# notHumpback-M1
+# Model Card for notHumpback-M1
 
-This model follows the Humpback architecture, proposed in the paper [Self-Alignment with Instruction Backtranslation](https://arxiv.org/pdf/2308.06259) 
-by Li et al.
+This model is a fine-tuned version of [meta-llama/Llama-3.2-3B](https://huggingface.co/meta-llama/Llama-3.2-3B).
+It has been trained using [TRL](https://github.com/huggingface/trl).
+
+## Quick start
 
-It represents the resulting model after the first iteration of self-curation, which is trained on a small amount of gold data 
-and a set of generated data curated by the ["seed model"](https://huggingface.co/Alepach/notHumpback-M0). 
+```python
+from transformers import pipeline
 
-This model can be used for instruction-following.
-It may also be used to, again, score the instruction-response pairs 
-generated by the ["backward model"](https://huggingface.co/Alepach/notHumpback-Myx) for a second iteration of self-curation. 
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="Alepach/notHumpback-M1", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
 
-Humpback uses instruction backtranslation on a web corpus to generate input-output pairs (self-augmentation), 
-creating a richer dataset for fine-tuning models without the need for additional manual annotation.
-The model then iteratively curates the created dataset, scoring the pairs by quality, and is then finetuned on the resulting subset 
-of all pairs with the highest possible score (self-curation).
+## Training procedure
 
-Varying from the original paper, this model is a fine-tuned version of [meta-llama/Llama-3.2-3B](https://huggingface.co/meta-llama/Llama-3.2-3B).
-It has been trained using [TRL](https://github.com/huggingface/trl).
 
-The dataset used to train this model is a combination of data sampled from the [oasst1](https://huggingface.co/datasets/OpenAssistant/oasst1) 
-dataset and the synthetic dataset which was mentioned above. The latter has been created by applying self-augmentation and self-curation 
-on 502k entries from the english subset ("en") of the [c4](https://huggingface.co/datasets/allenai/c4) dataset.
+
+This model was trained with SFT.
 
 ### Framework versions
 
@@ -46,18 +41,7 @@ on 502k entries from the english subset ("en") of the [c4](https://huggingface.c
 
 ## Citations
 
-Original paper:
 
-```bibtex
-@misc{li2023selfalignment,
-    title={Self-Alignment with Instruction Backtranslation},
-    author={Xian Li and Ping Yu and Chunting Zhou and Timo Schick and Luke Zettlemoyer and Omer Levy and Jason Weston and Mike Lewis},
-    year={2023},
-    eprint={2308.06259},
-    archivePrefix={arXiv},
-    primaryClass={cs.CL}
-}
-```
 
 Cite TRL as:
     

model-00001-of-00002.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8d327951fde732bd51038ef1131904ba807e24e03306f410c6ccd12fe92874f9
+oid sha256:5da643852670932849ee782872adef71d2d4362f7aa454add0d1f448d0d6f234
 size 4965799096

model-00002-of-00002.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ff35194bc23a85a89e01c2f4f9c076ac894846a8f64cae9553609f5060844264
+oid sha256:f1c3971b5437eafc0393c548fdffb66d3ec806ebcad96d5d9c3d0fe75fcedaef
 size 1459729952

special_tokens_map.json

@@ -13,11 +13,5 @@
     "rstrip": false,
     "single_word": false
   },
-  "pad_token": {
-    "content": "<|finetune_right_pad_id|>",
-    "lstrip": false,
-    "normalized": false,
-    "rstrip": false,
-    "single_word": false
-  }
+  "pad_token": "<|finetune_right_pad_id|>"
 }

tokenizer.json

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:76cfe2f054560aae896b2b75e273dc97a39e304d4ad19c44a9727a1d6b33c4cc
-size 17210021
+oid sha256:6b9e4e7fb171f92fd137b777cc2714bf87d11576700a1dcd7a399e7bbe39537b
+size 17209920

tokenizer_config.json

@@ -2050,18 +2050,14 @@
     }
   },
   "bos_token": "<|begin_of_text|>",
-  "chat_template": "{{- bos_token }}\n{% set ns = namespace(system_message='') %}\n{%- for message in messages %}\n    {%- if message['role'] == 'system' %}\n        {% set ns.system_message = message['content'].strip() %}\n    {%- elif message['role'] == 'user' %}\n        {{- '<|start_header_id|>user<|end_header_id|>' + ns.system_message + '\\n' + message['content'].strip() + '<|eot_id|>' }}\n    {%- elif message['role'] == 'assistant' %}\n        {{- '<|start_header_id|>assistant<|end_header_id|>' + message['content'] + '<|eot_id|>' }}\n    {%- endif %}\n{%- endfor %}\n",
+  "chat_template": "{{- bos_token }}\n{% set ns = namespace(system_message='') %}\n{%- for message in messages %}\n    {%- if message['role'] == 'system' %}\n        {% set ns.system_message = message['content'].strip() + '\\n\\n' %}\n    {%- elif message['role'] == 'user' %}\n        {{- '<|start_header_id|>user<|end_header_id|>' + '\\n\\n' + ns.system_message + message['content'].strip() + '\\n\\n' + 'Answer in the style of an AI assistant.' + '<|eot_id|>' }}\n    {%- elif message['role'] == 'assistant' %}\n        {{- '<|start_header_id|>assistant<|end_header_id|> + '\\n\\n' + message['content'] + '<|eot_id|>' }}\n    {%- endif %}\n{%- endfor %}\n",
   "clean_up_tokenization_spaces": true,
   "eos_token": "<|end_of_text|>",
-  "max_length": 131072,
   "model_input_names": [
     "input_ids",
     "attention_mask"
   ],
   "model_max_length": 131072,
   "pad_token": "<|finetune_right_pad_id|>",
-  "stride": 0,
-  "tokenizer_class": "PreTrainedTokenizerFast",
-  "truncation_side": "right",
-  "truncation_strategy": "longest_first"
+  "tokenizer_class": "PreTrainedTokenizerFast"
 }

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:eccc0c589e734ebe616745b1f0705c7db5f2eb370228ab81b43c81184f7faf31
+oid sha256:419c006815acf66d129b9f00c38ad7afee8119736845f234b523e4b073da74f9
 size 5560