view post Post 1133 Wow, impressive 340B model by nvidia with a nice permissive license! π The technical report is full of insights and seems to use a different learning rate schedule than cosine, probably a variant of WSD. Hope to get more info on that! π nvidia/nemotron-4-340b-666b7ebaf1b3867caf2f1911
LLM.C Fineweb vs Edu-Fineweb eliebak/wsd_124M_150B_edu Text Generation β’ Updated Jun 11 β’ 22 eliebak/wsd_124M_150B_fw Text Generation β’ Updated Jun 11 β’ 25 eliebak/wsd_124M_300B_edu Text Generation β’ Updated Jun 11 β’ 22 eliebak/wsd_124M_300B_fw Text Generation β’ Updated Jun 11 β’ 20