open-llm-leaderboard/open_llm_leaderboard · Not results found in LLM Benchmark and no in running evaluation queue

Sep 16

I submitted my model, xinchen9/Mistral-7B-CoT and found it in running evaluation queue. Last Saturday, I found my task disappeared in the queue but don't get the results till now. What happened?

Lyte

Sep 17

same here i submitted mine about 10 hours ago still no queue or any updates.

xinchen9

Sep 17

same here i submitted mine about 10 hours ago still no queue or any updates.

It happened a lot of time for me

alozowski

Open LLM Leaderboard org Sep 18

Hi @xinchen9 ,

According to our FAQ, please, provide me the link to the request file. It's the best way to track you model status and it will help me to check its current state

xinchen9

Sep 18

The model is xinchen9/Mistral-7B-CoT. But I just commit to new model. The submitted version is c280cee9005d4b39f0ab4addae3f8b27f0aad63a

alozowski

Open LLM Leaderboard org Sep 19

@xinchen9 ,

You can find the request file here, in the Requests dataset:
https://huggingface.co/datasets/open-llm-leaderboard/requests/tree/main

Here is the request file for xinchen9/Mistral-7B-CoT:
https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/main/xinchen9/Mistral-7B-CoT_eval_request_False_bfloat16_Original.json

You can see the status, it's FINISHED now, so you can find your model on the Leaderboard! 🤗

Please do not resubmit your models next time if you have already submitted them. Instead, please open a discussion and provide us with a request file. You can ask us to change the status of your model, change the revision of your model, or we can check the log for you. You can find more info in our documentation and FAQ: https://huggingface.co/docs/leaderboards/open_llm_leaderboard/faq

Let me close this discussion, please, open a new one in case of any questions!

alozowski changed discussion status to closed Sep 19