README.md

---
license: apache-2.0
tags:
- mixtral
- llamafile
- llm
- moe
---


# Mixtral 8X7B Instruct v0.1 - Llamafile 🦙

## Overview
This model card describes the `mixtral-8x7b-instruct-v0.1.Q3_K_M.llamafile`, a single-file executable version of the Mixtral 8X7B Instruct v0.1 model. <br>
It is built upon the original work by TheBloke and Mistral AI, repackaged for ease of use as a standalone application.  <br>
See [here](https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF)

Like many of you, i am GPU poor. The goal behind this approach was to have easy access to a good opensourced model with limited GPU resources, like a Macbook Pro M1 32GB.  <br>
It's not the full model, but it's the most feasible given the resource constraints - see [here](https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF#provided-files) for notes on performance 


## Usage
Because the model is converted to `llamafile`, it can be executed on any OS with no additional installations required.Read more about llamafile [here](https://github.com/Mozilla-Ocho/llamafile). <br>
To use this model, ensure you have execution permissions set:

```bash
chmod +x mixtral-8x7b-instruct-v0.1.Q3_K_M.llamafile
./mixtral-8x7b-instruct-v0.1.Q3_K_M.llamafile
```

See [here](https://github.com/Mozilla-Ocho/llamafile/blob/6423228b5ddd4862a3ab3d275a168692dadf4cdc/llama.cpp/server/README.md) for local API server details.

## Credits and Acknowledgements
This executable is a derivative of TheBloke's original Mixtral model, repurposed for easier deployment. It is licensed under the same terms as TheBloke's model.

## Limitations
As with the original Mixtral model, this executable does not include moderation mechanisms and should be used with consideration for its capabilities and limitations.

## Additional Information
For more detailed instructions and insights, please refer to the original model documentation provided by TheBloke and Mistral AI.