Update README.md
Browse filesAdd reference to original repository.
Change model_name and revision to point to the updated model weights.
README.md
CHANGED
@@ -9,19 +9,18 @@ datasets:
|
|
9 |
tags:
|
10 |
- mosaicML
|
11 |
- sharded
|
12 |
-
-
|
13 |
---
|
14 |
|
15 |
-
# mpt-7b-
|
16 |
|
17 |
|
18 |
-
|
19 |
-
|
20 |
-
|
|
|
21 |
|
22 |
-
|
23 |
-
|
24 |
-
Please refer to the previously linked repo for details on usage/implementation/etc. This model was downloaded from the original repo under Apache-2.0 and is redistributed under the same license.
|
25 |
|
26 |
|
27 |
## Basic Usage
|
@@ -40,12 +39,12 @@ Load the model:
|
|
40 |
import torch
|
41 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
42 |
|
43 |
-
model_name = '
|
44 |
model = AutoModelForCausalLM.from_pretrained(
|
45 |
model_name,
|
46 |
torch_dtype=torch.bfloat16,
|
47 |
trust_remote_code=True,
|
48 |
-
revision='
|
49 |
device_map='auto',
|
50 |
load_in_8bit=False, # install bitsandbytes then set to true for 8-bit
|
51 |
)
|
|
|
9 |
tags:
|
10 |
- mosaicML
|
11 |
- sharded
|
12 |
+
- instruct
|
13 |
---
|
14 |
|
15 |
+
# mpt-7b-instruct: sharded
|
16 |
|
17 |
|
18 |
+
This is a version of the [mpt-7b-instruct](https://huggingface.co/mosaicml/mpt-7b-instruct) model, sharded to 2 GB chunks for low-RAM loading (i.e. Colab).
|
19 |
+
The weights are stored in `bfloat16` so in theory you can run this on CPU, though it may take forever.
|
20 |
+
Original code and credits go to [mpt-7b-storywriter-sharded](https://huggingface.co/ethzanalytics/mpt-7b-storywriter-sharded).
|
21 |
+
See the [community discussion](https://huggingface.co/ethzanalytics/mpt-7b-storywriter-sharded/discussions/2) on how to replicate this.
|
22 |
|
23 |
+
Please refer to the previously linked repo for details on usage/implementation/etc. This model was downloaded from the original repo under Apache-2.0 and is redistributed under the same license.
|
|
|
|
|
24 |
|
25 |
|
26 |
## Basic Usage
|
|
|
39 |
import torch
|
40 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
41 |
|
42 |
+
model_name = 'jprafael/mpt-7b-instruct-sharded'
|
43 |
model = AutoModelForCausalLM.from_pretrained(
|
44 |
model_name,
|
45 |
torch_dtype=torch.bfloat16,
|
46 |
trust_remote_code=True,
|
47 |
+
revision='8d8911ad980f48f8a791e5f5876dea891dcbc064', # optional, but a good idea
|
48 |
device_map='auto',
|
49 |
load_in_8bit=False, # install bitsandbytes then set to true for 8-bit
|
50 |
)
|