ve-forbryderne commited on
Commit
789d41c
1 Parent(s): 409fd6c

Fill in some of the missing fields in the model card

Browse files
Files changed (1) hide show
  1. README.md +16 -8
README.md CHANGED
@@ -11,10 +11,10 @@ tags:
11
 
12
 
13
  - **Developed by:** KoboldAI
14
- - **Shared by [Optional]:** More information needed
15
  - **Model type:** Text Generation
16
- - **Language(s) (NLP):** More information needed
17
- - **License:** More information needed
18
  - **Related Models:** [GPT-J 6B](https://huggingface.co/EleutherAI/gpt-j-6B?text=My+name+is+Mariama%2C+my+favorite)
19
  - **Parent Model:** GPT-J
20
  - **Resources for more information:**
@@ -26,7 +26,15 @@ tags:
26
 
27
  ## Direct Use
28
 
29
- This model can be used for the task of text generation
 
 
 
 
 
 
 
 
30
 
31
  ## Downstream Use [Optional]
32
 
@@ -52,18 +60,18 @@ Users (both direct and downstream) should be made aware of the risks, biases and
52
 
53
  ## Training Data
54
 
55
- More information needed
56
 
57
  ## Training Procedure
58
 
59
 
60
  ### Preprocessing
61
 
62
- More information needed
63
 
64
  ### Speeds, Sizes, Times
65
 
66
- More information needed
67
 
68
  # Evaluation
69
 
@@ -114,7 +122,7 @@ More information needed
114
  More information needed
115
 
116
  ### Software
117
- More information needed
118
 
119
  # Citation
120
 
 
11
 
12
 
13
  - **Developed by:** KoboldAI
14
+ - **Shared by [Optional]:** KoboldAI
15
  - **Model type:** Text Generation
16
+ - **Language(s) (NLP):** English
17
+ - **License:** Apache License 2.0
18
  - **Related Models:** [GPT-J 6B](https://huggingface.co/EleutherAI/gpt-j-6B?text=My+name+is+Mariama%2C+my+favorite)
19
  - **Parent Model:** GPT-J
20
  - **Resources for more information:**
 
26
 
27
  ## Direct Use
28
 
29
+ This model is designed for creative story generation. It can understand both free-form text and text written in interactive fiction style with actions starting with "> You", such as:
30
+
31
+ ```
32
+ You become aware of her breathing -- the slight expansion of her ribs, the soft exhalation -- natural, and yet somehow studied. "Ah -- by the way," she says, in a way that utterly fails to be casual, "have you seen the artist out there? -- My artist, that is."
33
+
34
+ "No," you respond, uneasy. You open your mouth and close it again.
35
+
36
+ > You ask about the experience of waking up
37
+ ```
38
 
39
  ## Downstream Use [Optional]
40
 
 
60
 
61
  ## Training Data
62
 
63
+ The data are mostly comprised of light novels from the dataset of the [KoboldAI/GPT-Neo-2.7B-Horni-LN](https://huggingface.co/KoboldAI/GPT-Neo-2.7B-Horni-LN) model and assorted interactive fiction. The dataset uses `[Themes: <comma-separated list of genres>]` for tagging, which means that if similar text is placed in the context, the model will attempt to generate text in the specified style(s). For more details about the dataset, consult [this document](https://wandb.ai/ve-forbryderne/skein/runs/files/files/datasets/README.txt).
64
 
65
  ## Training Procedure
66
 
67
 
68
  ### Preprocessing
69
 
70
+ The data were preprocessed using the Python package ftfy to eliminate as much as possible non-ASCII punctuation characters and possible encoding errors. The interactive fiction in the dataset also underwent deduplication since interactive fiction logs often contain duplicate text from, for example, visiting the same in-game area several times. spaCy was used for grammatical analysis with the purpose of reformatting the actions commonly found in old text adventure games into more complete sentences. There was also some manual elimination of things such as "thank you for playing" messages and title messages.
71
 
72
  ### Speeds, Sizes, Times
73
 
74
+ Training took approximately 14 hours in total, with the average speed being 5265 tokens per second.
75
 
76
  # Evaluation
77
 
 
122
  More information needed
123
 
124
  ### Software
125
+ https://github.com/kingoflolz/mesh-transformer-jax
126
 
127
  # Citation
128