| This repository contains AWS Inferentia2 and neuronx compatible checkpoints for [Mistral-Large-Instruct](https://huggingface.co/mistralai/Mistral-Large-Instruct-2407). You can find detailed information about the base model on its [Model Card](https://huggingface.co/mistralai/Mistral-Large-Instruct-2407). | |
| This model has been exported to the neuron format using specific input_shapes and compiler parameters detailed in the paragraphs below. | |
| It has been compiled to run on an inf2.48xlarge instance on AWS. Note that while the inf2.48xlarge has 24 cores, this compilation uses 24. | |
| --- | |
| SEQUENCE_LENGTH = 4096 | |
| BATCH_SIZE = 4 | |
| NUM_CORES = 24 | |
| PRECISION = "bf16" | |
| license: other | |
| license_name: mrl | |
| license_link: LICENSE | |
| language: | |
| - en | |
| base_model: | |
| - mistralai/Mistral-Large-Instruct-2407 | |
| pipeline_tag: text-generation | |
| --- |