Hugging Face's logo

Infermatic
/
Miquliz-120b-v2.0-FP8-dynamic

Text Generation
Transformers
Safetensors
llama
conversational
text-generation-inference
Inference Endpoints
fp8
Model card Files Community
Edit model card

Miquliz-120b-v2.0-FP8-dynamic

image/jpeg

This quant was made for infermatic.ai

Dynamic FP8 quant of Miquliz 120B v2.0 made with AutoFP8.

Model Details

  • Max Context: 32768 tokens
  • Layers: 140

Prompt template: Mistral 

<s>[INST] {prompt} [/INST]
Downloads last month
85
Safetensors
Model size
120B params
Tensor type
FP16
·
F8_E4M3
·
Inference Examples
Text Generation
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Collection including Infermatic/Miquliz-120b-v2.0-FP8-dynamic