Update README.md
Browse files
README.md
CHANGED
|
@@ -21,6 +21,7 @@ I'm Happy to share the training progress of my new language model with a 32k set
|
|
| 21 |
|
| 22 |
7 look forward to sharing the results of this exciting project with you all!
|
| 23 |
|
|
|
|
| 24 |
# **SAMPLE**
|
| 25 |
This was a 330 million model that still has a slightly high loss:
|
| 26 |
|
|
@@ -58,7 +59,9 @@ simulated and real DNA sequences.
|
|
| 58 |
|
| 59 |
## STATUS TRAINING -
|
| 60 |
in my last tests with length 2048, I got great models, I trained models in 24 hours with only a 4090 GPU, I'll try to do the same with this 32k, in the following hours and I'll post the result
|
| 61 |
-
|
|
|
|
|
|
|
| 62 |
1 - OK
|
| 63 |
2 - RUNNING - next upload 9/9 - 00:30 GMT
|
| 64 |
3 -
|
|
|
|
| 21 |
|
| 22 |
7 look forward to sharing the results of this exciting project with you all!
|
| 23 |
|
| 24 |
+
|
| 25 |
# **SAMPLE**
|
| 26 |
This was a 330 million model that still has a slightly high loss:
|
| 27 |
|
|
|
|
| 59 |
|
| 60 |
## STATUS TRAINING -
|
| 61 |
in my last tests with length 2048, I got great models, I trained models in 24 hours with only a 4090 GPU, I'll try to do the same with this 32k, in the following hours and I'll post the result
|
| 62 |
+
In training, step 2/6
|
| 63 |
+
Each stage lasts 4-6 hours.
|
| 64 |
+
I am releasing the partial models, in the end I will also release the datasets. 100% synthetic data in markdown
|
| 65 |
1 - OK
|
| 66 |
2 - RUNNING - next upload 9/9 - 00:30 GMT
|
| 67 |
3 -
|