Update README.md
Browse files
README.md
CHANGED
@@ -4,11 +4,13 @@ license_name: other
|
|
4 |
license_link: https://ai.meta.com/llama/license
|
5 |
---
|
6 |
Repairing the chat template for the model.
|
7 |
-
There is a slight problem with
|
|
|
|
|
8 |
|
9 |
<|begin_of_text|><|start_header_id|>user<|end_header_id|>
|
10 |
|
11 |
-
<|image|>
|
12 |
|
13 |
Here is a haiku for the image:
|
14 |
|
|
|
4 |
license_link: https://ai.meta.com/llama/license
|
5 |
---
|
6 |
Repairing the chat template for the model.
|
7 |
+
There is a slight problem with the eorigianl llam 3.2 chat template. If you train a model with that current chat template and if the training script builds the prompts
|
8 |
+
frmm a json file with the chat temaplte the model starts to output as its first token <|eot_id|> and naturally the script will then halt generation.
|
9 |
+
the model learns to see this:
|
10 |
|
11 |
<|begin_of_text|><|start_header_id|>user<|end_header_id|>
|
12 |
|
13 |
+
<|image|>Wite Haiku: <|eot_id|><|start_header_id|>assistant<|end_header_id|>
|
14 |
|
15 |
Here is a haiku for the image:
|
16 |
|