jondurbin commited on
Commit
6cf5bc6
1 Parent(s): a2c4881

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -13
README.md CHANGED
@@ -28,27 +28,24 @@ This is an instruction fine-tuned llama-2 model, using synthetic data generated
28
 
29
  ### Prompt format
30
 
 
 
 
 
31
  ```
32
  A chat. USER: [prompt] ASSISTANT:
33
  ```
34
 
35
- So in other words, it's the preamble/system prompt, followed by a single space, then "USER: " (single space after colon) then the prompt (which can have multiple lines, spaces, whatever), then a single space, followed by "ASSISTANT: " (with a single space after the colon).
36
-
37
- I updated the training code to randomize whitespace so it will hopefully be less important:
38
- https://github.com/jondurbin/qlora/blob/main/qlora.py#L559C1-L559C1
39
-
40
- It also uses the system prompt provided in the instructions, when available.
41
-
42
- In addition, I included a few thousand examples with alpaca style, but I would not recommend using this format.
43
 
44
  ```
45
- Below is an instruction that describes a task. Write a response that appropriately completes the request.
46
-
47
- ### Instruction:
48
- {prompt}
49
- ### Response:
50
  ```
51
 
 
 
52
  ### Helpful usage tips
53
 
54
  *The prompts shown here are are just the text that would be included after USER: and before ASSISTANT: in the full prompt format above, the system prompt and USER:/ASSISTANT: have been omited for readability.*
 
28
 
29
  ### Prompt format
30
 
31
+ The training code was updated to randomize newline vs space:
32
+ https://github.com/jondurbin/qlora/blob/main/qlora.py#L559C1-L559C1
33
+
34
+
35
  ```
36
  A chat. USER: [prompt] ASSISTANT:
37
  ```
38
 
39
+ or
 
 
 
 
 
 
 
40
 
41
  ```
42
+ A chat.
43
+ USER: {prompt}
44
+ ASSISTANT:
 
 
45
  ```
46
 
47
+ So in other words, it's the preamble/system prompt, followed by a single space or newline, then "USER: " (single space after colon) then the prompt (which can have multiple lines, spaces, whatever), then a single space or newline, followed by "ASSISTANT: " (with a single space after the colon).
48
+
49
  ### Helpful usage tips
50
 
51
  *The prompts shown here are are just the text that would be included after USER: and before ASSISTANT: in the full prompt format above, the system prompt and USER:/ASSISTANT: have been omited for readability.*