how yo use whe 2 GGuf
#1
by
Gomer99
- opened
Is there any guide or ready code to use L3.1-70B-Euryale-v2.2.Q8_0.gguf.part1of2, L3.1-70B-Euryale-v2.2.Q8_0.gguf.part2of2 to use the model and get API for use in SillyTaver? on Spaces or Inference Endpoints.
yes, directly on the model page it has a link on how to use them. in this specific case, you have to concatenate the files (e.g. with cat on unix, or copy/b on ms-dos) in order into one big file, which you then should be able to use directly in your software. for an api, you can use llama.cpp or koboldcpp and probably some others. if you cna use inference endpoints you can also use the original model (also linked form the model page) directly.
mradermacher
changed discussion status to
closed