Update README.md

README.md

@@ -5,34 +5,27 @@ tags:
 - generated_from_trainer
 metrics:
 - accuracy
-model-index:
-- name: griffin-v0.01-c3t-8layer-simplewiki-silu-fineweb-1M_en-med-vN
-  results: []
+datasets:
+- BEE-spoke-data/fineweb-1M_en-med
+language:
+- en
 ---
 
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment. -->
 
-# griffin-v0.01-c3t-8layer-simplewiki-silu-fineweb-1M_en-med-vN
+# griffin-c3t-8L-v0.02-fineweb
 
-This model is a fine-tuned version of [pszemraj/griffin-v0.01-c3t-8layer-simplewiki-silu](https://huggingface.co/pszemraj/griffin-v0.01-c3t-8layer-simplewiki-silu) on the BEE-spoke-data/fineweb-1M_en-med dataset.
+Pretraining experiment with griffin/recurrent_gemma arch
+
+## Model description
+
+Further training of [pszemraj/griffin-v0.01-c3t-8layer-simplewiki-silu](https://hf.co/pszemraj/griffin-v0.01-c3t-8layer-simplewiki-silu) on the BEE-spoke-data/fineweb-1M_en-med dataset.
 It achieves the following results on the evaluation set:
 - Loss: 5.1888
 - Accuracy: 0.2326
 - Num Input Tokens Seen: 798621696
 
-## Model description
-
-More information needed
-
-## Intended uses & limitations
-
-More information needed
-
-## Training and evaluation data
-
-More information needed
-
 ## Training procedure
 
 ### Training hyperparameters
@@ -75,4 +68,4 @@ The following hyperparameters were used during training:
 - Transformers 4.40.1
 - Pytorch 2.3.0+cu121
 - Datasets 2.19.0
-- Tokenizers 0.19.1
+- Tokenizers 0.19.1