Hugging Face's logo

bartowski
/
Phi-3.1-mini-4k-instruct-GGUF

Text Generation
GGUF
English
nlp
code
Inference Endpoints
imatrix
conversational
Model card Files Community
3

GGUF Model

#2
by Pecc - opened
Discussion
Pecc

Can I load the model with the standard from_pretained model and tokenizer fro transformers library?

bartowski
 Owner

no you'll want to use the a llama.cpp library like this one:

https://github.com/abetlen/llama-cpp-python

Sign up or log in to comment