This is karpathy's model from the llm.c project converted to HF format to investigate bfloat16 performance.
The training run was 150B tokens, 1.5 epochs over the 100B FineWeb sample dataset.
There's active work underway at https://github.com/karpathy/llm.c so I'd suggest following the developments there!
- Downloads last month
- 2,365
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.