Clémentine Fourrier's picture

Clémentine Fourrier

clefourrier

·

http://clefourrier.github.io

AI & ML interests

None yet

Recent Activity

new activity 3 days ago

cais/hle:Please gate the dataset

upvoted a collection 5 days ago

liked a model 5 days ago

deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B

View all activity

Articles

CO₂ Emissions and Models Performance: Insights from the Open LLM Leaderboard

Rethinking LLM Evaluation with 3C3H: AraGen Benchmark and Leaderboard

Introduction to the Open Leaderboard for Japanese LLMs

Letting Large Models Debate: The First Multilingual LLM Debate Competition

Judge Arena: Benchmarking LLMs as Evaluators

Introducing the Open FinLLM Leaderboard

BigCodeBench: Benchmarking Large Language Models on Solving Practical and Challenging Programming Tasks

Falcon 2: An 11B parameter pretrained language model and VLM, trained on over 5000B tokens tokens and 11 languages

CyberSecEval 2 - A Comprehensive Evaluation Framework for Cybersecurity Risks and Capabilities of Large Language Models

Let's talk about LLM evaluation

Introducing the Open Arabic LLM Leaderboard

Introducing the Open Leaderboard for Hebrew LLMs!

Bringing the Artificial Analysis LLM Performance Leaderboard to Hugging Face

Improving Prompt Consistency with Structured Generations

Introducing the Open Chain of Thought Leaderboard

The Open Medical-LLM Leaderboard: Benchmarking Large Language Models in Healthcare

Introducing the LiveCodeBench Leaderboard - Holistic and Contamination-Free Evaluation of Code LLMs

Introducing the Chatbot Guardrails Arena

Introducing ConTextual: How well can your Multimodal model jointly reason over text and image in text-rich scenes?

TTS Arena: Benchmarking Text-to-Speech Models in the Wild

Introducing the Red-Teaming Resistance Leaderboard

Introducing the Open Ko-LLM Leaderboard: Leading the Korean LLM Evaluation Ecosystem

NPHardEval Leaderboard: Unveiling the Reasoning Abilities of Large Language Models through Complexity Classes and Dynamic Updates

Introducing the Enterprise Scenarios Leaderboard: a Leaderboard for Real World Use Cases

The Hallucinations Leaderboard, an Open Effort to Measure Hallucinations in Large Language Models

A guide to setting up your own Hugging Face leaderboard: an end-to-end example with Vectara's hallucination leaderboard

2023, year of open LLMs

Open LLM Leaderboard: DROP deep dive

Overview of natively supported quantization schemes in 🤗 Transformers

What's going on with the Open LLM Leaderboard?

Introduction to Graph Machine Learning

Organizations

clefourrier's activity

New activity in cais/hle 3 days ago

Please gate the dataset

#1 opened 4 days ago by

upvoted a collection 5 days ago

DeepSeek-R1

8 items • Updated 6 days ago • 165

liked a model 5 days ago

deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B

Text Generation • Updated about 21 hours ago • 107k • • 414

commented on CO₂ Emissions and Models Performance: Insights from the Open LLM Leaderboard 6 days ago

@patmcguinness Calculating the cost of energy input vs the value of the AI output is literally what we did, using CO2 as our metric for energy consumption (we assume a linear relationship between CO2 and energy). However, we did not include all our graphs (only the most salient ones).

Since all the data is open, you could easily compute a CO2/energy per param to score graph, wdyt?

commented on CO₂ Emissions and Models Performance: Insights from the Open LLM Leaderboard 6 days ago

@alielfilali01 Thanks! Per model, but we could display it per #params too! :)
We initially did not want to assume only the number of params would play a role in the analysis.

upvoted a collection 7 days ago

most ducked models 🦆🦆🦆

https://x.com/jeremyphoward/status/1881264223646576786 • 5 items • Updated 7 days ago • 3

reacted to BrigitteTousi's post with ❤️ 13 days ago

Post

1040

Community fine-tuned models are more carbon efficient than the models they are derived from! 🥳🌿

@alozowski @clefourrier @SaylorTwift @albertvillanova evaluated CO₂ emissions associated with model inference for over 3000 models on the Open LLM Leaderboard. Interesting trends and new insights emerged...👀

Blog Post: https://huggingface.co/blog/leaderboard-emissions-analysis

Leaderboard: open-llm-leaderboard/open_llm_leaderboard

New activity in open-llm-leaderboard/open_llm_leaderboard 13 days ago

[Question] Are no models being evaluated due to the lighteval/MATH-Hard dataset being unavailable?

#1071 opened 14 days ago by

updated a collection 14 days ago

Leaderboards and benchmarks ✨

Cool leaderboard spaces collection for models across modalities! Text, vision, audio, ... • 89 items • Updated 14 days ago • 94

upvoted 2 articles 17 days ago

Article

🌁#81: Key AI Concepts to Follow in 2025

By

•

Dec 23, 2024

• 24

Article

🐺🐦‍⬛ LLM Comparison/Test: DeepSeek-V3, QVQ-72B-Preview, Falcon3 10B, Llama 3.3 70B, Nemotron 70B in my updated MMLU-Pro CS benchmark

By

•

24 days ago

• 39

upvoted an article 19 days ago

Article

Fine-tune a SmolLM on domain-specific synthetic data from a LLM

By

•

24 days ago

• 31

updated a dataset 19 days ago

open-llm-leaderboard/results

Preview • Updated about 2 hours ago • 32.5k • 8

updated a collection 21 days ago

Leaderboards and benchmarks ✨

Cool leaderboard spaces collection for models across modalities! Text, vision, audio, ... • 89 items • Updated 14 days ago • 94

New activity in open-llm-leaderboard/open_llm_leaderboard 21 days ago

need help

#1061 opened 21 days ago by

Open LLM Leaderboard Results Dataset

#1046 opened about 1 month ago by