Submit models with custom code.

#341
by acrastt - opened

Hello, I was wondering if I can submit the model Marx 3B V3 onto the leaderboard manually? It contains custom code so it cannot be submitted in the application, though I can promise that this model does not contain any malcious code, it is simply the custom code for StableLM 3B 4E1T model. You can manually check the code if you don't believe me. It would be great if you can submit Airoboros 3B 3.11 too, though it is not mine.

Tagging: @clefourrier

Open LLM Leaderboard org

Hi!
No, we do not allow submission of models with custom code on the Open LLM Leaderboard for security reasons, and because we don't have the time to manually check new code bases. We really want to focus our energy and time on upgrading the leaderboard.
We'll however do a big announcement if we find a system which allows such submissions safely!

clefourrier changed discussion status to closed

Sign up or log in to comment