Stop token fix?

by americancookie - opened May 22

Discussion

americancookie

May 22

•

edited May 22

Hi Michael,

Thanks for your quants, Vielen Dank!

What stop token for LLama 3 quants do you use during the conversion?

Please see the discussion at: https://www.reddit.com/r/LocalLLaMA/comments/1c7dkxh/tutorial_how_to_make_llama3instruct_ggufs_less/

Cheers :)

americancookie changed discussion title from End of token fix? to Stop token fix? May 22

mradermacher

Owner May 22

•

edited May 22

I'm not using any tokens, I merely quantize the model as-is. Also, the "fix" you link to breaks the model, so should not be applied. The problem is in the software, not the gguf. The only correct way to fix this is to update your software for llama-3.

mradermacher changed discussion status to closed May 22

americancookie

May 22

Thank you :)!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment