GGUF
#1
by
MikeLightheart
- opened
Would it be possible to get a GGUF version of this model? No worries if not, but I've been using the GUI apps (like LM Studio, AnythingLLM, and Jan) and I don't have a way to get this model into those. Especially for use with RAG due to the absurd context window this model has.
@bartowski please :)
Not supported yet :') new architecture
Support just got merged 2 hours ago :)
Use with ollama
Note: this model requires Ollama 0.5.5
You can use huihui_ai/command-r7b-abliterated directly
ollama run huihui_ai/command-r7b-abliterated