Update README.md

README.md

@@ -20,11 +20,9 @@ model-index:
       value: 0.5792084706530948
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
-
 # mistral-1L-tiny
 
+A tiny single-layer 35.1M parameter Mistral model, with a hidden size of 512, and an MLP intermediate size of 1024.
 This model is trained on the roneneldan/TinyStories dataset.
 It achieves the following results on the evaluation set:
 - Loss: 1.6868
@@ -32,18 +30,17 @@ It achieves the following results on the evaluation set:
 
 ## Model description
 
-More information needed
+This work is inspired by the 21M parameter one-layer GPT-Neo of the [Tiny Stories paper](https://arxiv.org/abs/2305.07759).
+Results reproduced to acquire high-frequency checkpoints for further analysis.
 
 ## Intended uses & limitations
 
-More information needed
-
-## Training and evaluation data
-
-More information needed
+Analysis of feature dynamics and emergence in real-world language models.
 
 ## Training procedure
 
+Trained for 90171 steps, corresponding to ~2 hours on a single H100.
+
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
@@ -57,7 +54,7 @@ The following hyperparameters were used during training:
 
 ### Training results
 
-
+Quite consistent English text generation.
 
 ### Framework versions