jprafael commited on
Commit
7453303
1 Parent(s): 8d8911a

Update README.md

Browse files

Add reference to original repository.
Change model_name and revision to point to the updated model weights.

Files changed (1) hide show
  1. README.md +9 -10
README.md CHANGED
@@ -9,19 +9,18 @@ datasets:
9
  tags:
10
  - mosaicML
11
  - sharded
12
- - story
13
  ---
14
 
15
- # mpt-7b-storywriter: sharded
16
 
17
 
18
- <a href="https://colab.research.google.com/gist/pszemraj/a979cdcc02edb916661c5dd97cf2294e/mpt-storywriter-sharded-inference.ipynb">
19
- <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>
20
- </a>
 
21
 
22
- This is a version of the [mpt-7b-storywriter](https://huggingface.co/mosaicml/mpt-7b-storywriter) model, sharded to 2 GB chunks for low-RAM loading (i.e. Colab). The weights are stored in `bfloat16` so in theory you can run this on CPU, though it may take forever.
23
-
24
- Please refer to the previously linked repo for details on usage/implementation/etc. This model was downloaded from the original repo under Apache-2.0 and is redistributed under the same license.
25
 
26
 
27
  ## Basic Usage
@@ -40,12 +39,12 @@ Load the model:
40
  import torch
41
  from transformers import AutoModelForCausalLM, AutoTokenizer
42
 
43
- model_name = 'ethzanalytics/mpt-7b-storywriter-sharded'
44
  model = AutoModelForCausalLM.from_pretrained(
45
  model_name,
46
  torch_dtype=torch.bfloat16,
47
  trust_remote_code=True,
48
- revision='197d14245ad874da82194248cab1ce8cf87fa713', # optional, but a good idea
49
  device_map='auto',
50
  load_in_8bit=False, # install bitsandbytes then set to true for 8-bit
51
  )
 
9
  tags:
10
  - mosaicML
11
  - sharded
12
+ - instruct
13
  ---
14
 
15
+ # mpt-7b-instruct: sharded
16
 
17
 
18
+ This is a version of the [mpt-7b-instruct](https://huggingface.co/mosaicml/mpt-7b-instruct) model, sharded to 2 GB chunks for low-RAM loading (i.e. Colab).
19
+ The weights are stored in `bfloat16` so in theory you can run this on CPU, though it may take forever.
20
+ Original code and credits go to [mpt-7b-storywriter-sharded](https://huggingface.co/ethzanalytics/mpt-7b-storywriter-sharded).
21
+ See the [community discussion](https://huggingface.co/ethzanalytics/mpt-7b-storywriter-sharded/discussions/2) on how to replicate this.
22
 
23
+ Please refer to the previously linked repo for details on usage/implementation/etc. This model was downloaded from the original repo under Apache-2.0 and is redistributed under the same license.
 
 
24
 
25
 
26
  ## Basic Usage
 
39
  import torch
40
  from transformers import AutoModelForCausalLM, AutoTokenizer
41
 
42
+ model_name = 'jprafael/mpt-7b-instruct-sharded'
43
  model = AutoModelForCausalLM.from_pretrained(
44
  model_name,
45
  torch_dtype=torch.bfloat16,
46
  trust_remote_code=True,
47
+ revision='8d8911ad980f48f8a791e5f5876dea891dcbc064', # optional, but a good idea
48
  device_map='auto',
49
  load_in_8bit=False, # install bitsandbytes then set to true for 8-bit
50
  )