| ## version： v20240318 | |
| - train 12B tokens with 256k context window | |
| - recall of "Needle in A HayStack": 97.5% [](./images/v20240318.png) | |
| ## version: initial | |
| - train 6B tokens with 256k context window | |
| - recall of "Needle in A HayStack": 87.1% [](./images/initail.png) | |