Spaces:

open-llm-leaderboard

/

open_llm_leaderboard

like 12.5k

Running on CPU Upgrade

Even 7b mistral v0.2 is failing (it started 2 days ago). Tried to test already tested model - same fail.
https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/main/icefog72/Ice0.77-02.02-RP_eval_request_False_bfloat16_Original.json
https://huggingface.co/datasets/open-llm-leaderboard/results/tree/main/icefog72/Ice0.77-02.02-RP

Hi @ehristoforu and @icefog72 ,

I checked the logs for ehristoforu/della-70b-test-v1, there was a CUDA error, and the same for icefog72/Ice0.77-02.02-R model. I'll fix the evaluation myself and get back with the results

@alozowski no need to test my Ice0.77-02.02-RP again, just look why it's beginning to fail, thx

All failed again. Like I was writing previously, fails begin to appear 2-3 days ago. For me results is not critical(hours ruining llm-rp-test-framework to test 1 model better than nothing) but still not only I will get error.

All failed again. Like I was writing previously, fails begin to appear 2-3 days ago. For me results is not critical(hours ruining llm-rp-test-framework to test 1 model better than nothing) but still not only I will get error.

ehristoforu/della-70b-test-v1 also failed again(

@alozowski my merge failed a while ago various attempts but no success
https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/main/Aryanne/QwentileSwap_eval_request_False_bfloat16_Original.json

Both ehristoforu/della-70b-test-v1 and icefog72/Ice0.77-02.02-R are finished, the results will be on the Leaderboard within an hour

@Aryanne , please, open a separate discussion for this model

Closing this discussion, please, feel free to open a new one in case of any questions!