defog

/

sqlcoder-7b-2

like 286

Follow

Defog.ai 80

I have deployed this model on a Text Generation Inference with an Nvidia L4 · 1x GPU · 24 GB,
and I get this error
{'error': 'Input validation error: `inputs` tokens + `max_new_tokens` must be <= 1512. Given: 6644 `inputs` tokens and 100 `max_new_tokens`', 'error_type': 'validation'}

but looking the the model config I can see that the model has : "max_position_embeddings": 16384

I'm not sure what the issue is !