What library was used to quantize the model?

by KirillR - opened Apr 6

Apr 6

Did you use AutoGPTQ to quantize the model or something else? I'm trying to replicate this myself but AutoGPTQ doesn't support cohere

Owner Apr 6

Please see #1

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment