---
license: apache-2.0
---
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6589d7e6586088fd2784a12c/IPxdU9JFGcrEmqzOdV46Z.png)
# It will run...but on what hardware?

## 512 experts in one MoE

[A giant merge of all the same model](https://huggingface.co/ahxt/LiteLlama-460M-1T)