why evaluation is taking long time for large llm?

#1079
by Imran1 - opened

for small model the evaluation working. but large model is taking to much time.
it will be good, if the evaluation is async/multiprocess for llm or at least for 10 llm.

Open LLM Leaderboard org

Hi! The evaluation is multiprocess for the LLMs, and the flucutations of evaluation depend on how much of our cluster is taken at a given moment.

clefourrier changed discussion status to closed

Sign up or log in to comment