ve-forbryderne
commited on
Commit
•
789d41c
1
Parent(s):
409fd6c
Fill in some of the missing fields in the model card
Browse files
README.md
CHANGED
@@ -11,10 +11,10 @@ tags:
|
|
11 |
|
12 |
|
13 |
- **Developed by:** KoboldAI
|
14 |
-
- **Shared by [Optional]:**
|
15 |
- **Model type:** Text Generation
|
16 |
-
- **Language(s) (NLP):**
|
17 |
-
- **License:**
|
18 |
- **Related Models:** [GPT-J 6B](https://huggingface.co/EleutherAI/gpt-j-6B?text=My+name+is+Mariama%2C+my+favorite)
|
19 |
- **Parent Model:** GPT-J
|
20 |
- **Resources for more information:**
|
@@ -26,7 +26,15 @@ tags:
|
|
26 |
|
27 |
## Direct Use
|
28 |
|
29 |
-
This model
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
30 |
|
31 |
## Downstream Use [Optional]
|
32 |
|
@@ -52,18 +60,18 @@ Users (both direct and downstream) should be made aware of the risks, biases and
|
|
52 |
|
53 |
## Training Data
|
54 |
|
55 |
-
|
56 |
|
57 |
## Training Procedure
|
58 |
|
59 |
|
60 |
### Preprocessing
|
61 |
|
62 |
-
|
63 |
|
64 |
### Speeds, Sizes, Times
|
65 |
|
66 |
-
|
67 |
|
68 |
# Evaluation
|
69 |
|
@@ -114,7 +122,7 @@ More information needed
|
|
114 |
More information needed
|
115 |
|
116 |
### Software
|
117 |
-
|
118 |
|
119 |
# Citation
|
120 |
|
|
|
11 |
|
12 |
|
13 |
- **Developed by:** KoboldAI
|
14 |
+
- **Shared by [Optional]:** KoboldAI
|
15 |
- **Model type:** Text Generation
|
16 |
+
- **Language(s) (NLP):** English
|
17 |
+
- **License:** Apache License 2.0
|
18 |
- **Related Models:** [GPT-J 6B](https://huggingface.co/EleutherAI/gpt-j-6B?text=My+name+is+Mariama%2C+my+favorite)
|
19 |
- **Parent Model:** GPT-J
|
20 |
- **Resources for more information:**
|
|
|
26 |
|
27 |
## Direct Use
|
28 |
|
29 |
+
This model is designed for creative story generation. It can understand both free-form text and text written in interactive fiction style with actions starting with "> You", such as:
|
30 |
+
|
31 |
+
```
|
32 |
+
You become aware of her breathing -- the slight expansion of her ribs, the soft exhalation -- natural, and yet somehow studied. "Ah -- by the way," she says, in a way that utterly fails to be casual, "have you seen the artist out there? -- My artist, that is."
|
33 |
+
|
34 |
+
"No," you respond, uneasy. You open your mouth and close it again.
|
35 |
+
|
36 |
+
> You ask about the experience of waking up
|
37 |
+
```
|
38 |
|
39 |
## Downstream Use [Optional]
|
40 |
|
|
|
60 |
|
61 |
## Training Data
|
62 |
|
63 |
+
The data are mostly comprised of light novels from the dataset of the [KoboldAI/GPT-Neo-2.7B-Horni-LN](https://huggingface.co/KoboldAI/GPT-Neo-2.7B-Horni-LN) model and assorted interactive fiction. The dataset uses `[Themes: <comma-separated list of genres>]` for tagging, which means that if similar text is placed in the context, the model will attempt to generate text in the specified style(s). For more details about the dataset, consult [this document](https://wandb.ai/ve-forbryderne/skein/runs/files/files/datasets/README.txt).
|
64 |
|
65 |
## Training Procedure
|
66 |
|
67 |
|
68 |
### Preprocessing
|
69 |
|
70 |
+
The data were preprocessed using the Python package ftfy to eliminate as much as possible non-ASCII punctuation characters and possible encoding errors. The interactive fiction in the dataset also underwent deduplication since interactive fiction logs often contain duplicate text from, for example, visiting the same in-game area several times. spaCy was used for grammatical analysis with the purpose of reformatting the actions commonly found in old text adventure games into more complete sentences. There was also some manual elimination of things such as "thank you for playing" messages and title messages.
|
71 |
|
72 |
### Speeds, Sizes, Times
|
73 |
|
74 |
+
Training took approximately 14 hours in total, with the average speed being 5265 tokens per second.
|
75 |
|
76 |
# Evaluation
|
77 |
|
|
|
122 |
More information needed
|
123 |
|
124 |
### Software
|
125 |
+
https://github.com/kingoflolz/mesh-transformer-jax
|
126 |
|
127 |
# Citation
|
128 |
|