jbp's picture

jbp

jensbosseparra
ยท

AI & ML interests

None yet

Recent Activity

liked a model 11 days ago
tencent/HunyuanVideo
liked a model 16 days ago
gokaygokay/Flux-Miniature-LoRA
liked a model 4 months ago
mlx-community/pixtral-12b-8bit
View all activity

Organizations

MLX Community's profile picture

jensbosseparra's activity

reacted to singhsidhukuldeep's post with ๐Ÿค— 8 months ago
view post
Post
1031
You are happy that @Meta has open-sourced Llama 3 ๐Ÿ˜ƒ...

So you jump on @HuggingFace Hub to download the new shiny Llama 3 model only to see a few quintillion Llama 3's! ๐Ÿฆ™โœจ

Which one should you use? ๐Ÿค”

Not all Llamas are created equal! ๐Ÿฆ™โš–๏ธ

An absolutely crazy comparison experiment by Wolfram Ravenwolf ( @Wolfram ) might answer your question! ๐Ÿงช๐Ÿง™โ€โ™‚๏ธ

- Comprehensive assessment of Llama 3 Instruct 70B and 8B models. ๐Ÿ“Š
- Tested 20 versions across HF, GGUF, and EXL2 formats. ๐Ÿ”„
- Methodology: The process tested translation capabilities and cross-language understanding, using deterministic generation settings to minimize random factors. Used German data protection training exams to evaluate cross-language understanding. ๐ŸŒ๐Ÿ“
- Best performance from EXL2 4.5bpw quant, scoring perfect in all tests. ๐Ÿ†โœ…
- GGUF 8-bit to 4-bit quants also performed exceptionally. ๐ŸŒŸ
- Llama 3 8B unquantized is best in its size class but not as good as 70B quants. ๐Ÿ“๐Ÿ”
- 1-bit quantizations showed significant quality drops. โš ๏ธโฌ‡๏ธ

Best models:
- turboderp/Llama-3-70B-Instruct-exl2
- casperhansen/llama-3-70b-instruct-awq

Blog: https://huggingface.co/blog/wolfram/llm-comparison-test-llama-3