Spaces:

open-llm-leaderboard

/

open_llm_leaderboard

like 11.7k

Running on CPU Upgrade

Hi @lvkaokao !
We are currently doing a very big upgrade of the leaderboard so all evaluations are halted till the end of the month. @SaylorTwift did a small announcement about this on twitter a week ago.
Thank you very much for your patience, we're going as fast as we can! 🤗

hi, @clefourrier

Thanks for your reply!

(Let's not close this issue yet so that other people wondering the same thing get the information too if that works for you :) )

(Let's not close this issue yet so that other people wondering the same thing get the information too if that works for you :) )

hhh

Any updates? It was supposed to resume at the end of october, but we have november now.

Personally I would prefer a stable and reliable leaderboard over some updates. This is the best and only way to evaluate models for people who don't have the hardware power themselves.

Hi @Dampfinchen !
We are in the last stages of the update, and expect to do an announcement next week.

Side note, but the update should be increasing the leaderboard's reliability quite a bit :)

We are in the last stages of the update, and expect to do an announcement next week.

Thanks 🙏 I just wanted to chime in and say it would be great if status updates could be here or somewhere else on this space instead of Twitter or other even more soul-destroying social media (LinkedIn … 🤢)

thank you for your work

Yes, agree, you are doing excellent work.
I saw the queue of models and there’s like 200.
Any chance we could bump the testing of
DeepSeeks Code 6.7B and 34Bmodels and ofcourse Openchat 3.5 7B which I feel( but could of course be wrong) might be big in terms of results..I realize it is not completely fair to all who are patiently waiting in line. BUt that is what I am looking out for at moment.
Again thanks for great work on this leaderboard!!! 🙏

Yes, agree, you are doing excellent work.
I saw the queue of models and there’s like 200.
Any chance we could bump the testing of
DeepSeeks Code 6.7B and 34Bmodels and ofcourse Openchat 3.5 7B which I feel( but could of course be wrong) might be big in terms of results..I realize it is not completely fair to all who are patiently waiting in line. BUt that is what I am looking out for at moment.
Again thanks for great work on this leaderboard!!! 🙏

lol you and the rest of us buddy!

Hi @clefourrier ,
Noticed the queue of models seems to be stuck. Are you actively adding new models to leaderboard?
How many can you actually handle / day? Are there any resources that would help from community.
I’d like to ask/ suggest again if you might bump deepseek coder, openchat 3.5 and maybe..? Yi? in queue, as I think many are eager to see results.

Thank you for providing an excellent resource! 👍

Hi @Olofp ,
Thank you for your concerns! The Yi base models (7B and 34B) are already in the leaderboard :)
We are starting the queue again (slowly because our cluster is quite full), as our update is taking a couple of days more than we thought and we don't want to keep you folks waiting!

But stay tuned for something very cool tomorrow if all goes well

@lvkaokao @lunaalice @Dampfinchen @pszemraj @ehartford @Olofp

Upgrade is finally out, and the queue restarted! Find out more here 🔥

Thank you all very much for your patience! 🤗