|
--- |
|
license: apache-2.0 |
|
tags: |
|
- llama |
|
--- |
|
# TinyLlama-1.1B-ckpt-2.5T-exl2 |
|
|
|
EXL2 quants of [TinyLlama/TinyLlama-1.1B-intermediate-step-1195k-token-2.5T](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1195k-token-2.5T) intended for use in speculative decoding. |
|
|
|
- [3.0bpw-h6](https://huggingface.co/royallab/TinyLlama-1.1B-ckpt-2.5T-exl2/tree/3.0bpw-h6) |
|
- [4.0bpw-h6](https://huggingface.co/royallab/TinyLlama-1.1B-ckpt-2.5T-exl2/tree/4.0bpw-h6) |
|
- [6.0bpw-h6](https://huggingface.co/royallab/TinyLlama-1.1B-ckpt-2.5T-exl2/tree/6.0bpw-h6) |
|
- [8.0bpw-h8](https://huggingface.co/royallab/TinyLlama-1.1B-ckpt-2.5T-exl2/tree/8.0bpw-h8) |
