dvruette commited on
Commit
c8aec45
1 Parent(s): 0a22702

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -14,12 +14,12 @@ widget:
14
  # Open-Assistant StableLM-7B SFT-7 Model
15
 
16
 
17
- This is the 4th iteration English supervised-fine-tuning (SFT) model of
18
  the [Open-Assistant](https://github.com/LAION-AI/Open-Assistant) project.
19
- It is based on a Pythia 12B that was fine-tuned on human demonstrations
20
  of assistant conversations collected through the
21
  [https://open-assistant.io/](https://open-assistant.io/) human feedback web
22
- app before March 25, 2023.
23
 
24
  ## Model Details
25
 
@@ -51,7 +51,7 @@ start generating the assistant reply.
51
  - base model: [stabilityai/stablelm-base-alpha-7b](https://huggingface.co/stabilityai/stablelm-base-alpha-7b)
52
  - checkpoint: 3 epochs (12000 steps)
53
 
54
- command: `deepspeed trainer_sft.py --configs defaults reference-data reference-pythia-12b --cache_dir /home/ubuntu/data_cache --output_dir .saved/oasst-sft-3-pythia-12b-reference_2kpre --num_train_epochs 8 --residual_dropout 0.2 --deepspeed --use_flash_attention true --model_name andreaskoepf/pythia-12b-pre-2000`
55
 
56
  data:
57
  ```
 
14
  # Open-Assistant StableLM-7B SFT-7 Model
15
 
16
 
17
+ This is the 7th iteration English supervised-fine-tuning (SFT) model of
18
  the [Open-Assistant](https://github.com/LAION-AI/Open-Assistant) project.
19
+ It is based on a StableLM 7B that was fine-tuned on human demonstrations
20
  of assistant conversations collected through the
21
  [https://open-assistant.io/](https://open-assistant.io/) human feedback web
22
+ app before April 12, 2023.
23
 
24
  ## Model Details
25
 
 
51
  - base model: [stabilityai/stablelm-base-alpha-7b](https://huggingface.co/stabilityai/stablelm-base-alpha-7b)
52
  - checkpoint: 3 epochs (12000 steps)
53
 
54
+ command: `deepspeed trainer_sft.py --configs defaults stablelm-7b oasst-mix --cache_dir /home/ubuntu/data_cache --output_dir .saved/stable-lm-7b-1 --num_train_epochs 4 --deepspeed`
55
 
56
  data:
57
  ```