Loser Cheems's picture

9 3 9

Loser Cheems

JingzeShi

·

https://github.com/LoserCheems

LoserCheems

AI & ML interests

I like training small languge models.

Recent Activity

updated a model about 3 hours ago

JingzeShi/Doge-160M-checkpoint

updated a model about 12 hours ago

JingzeShi/Doge-20M

updated a model about 14 hours ago

JingzeShi/Doge-60M-Instruct

View all activity

Organizations

Posts 2

Post

1549

🤩warmup -> stable -> decay leanring rate scheduler:
😎use the Stable Phase CheckPoints to Continue Training the model on Any New Dataset without spikes of the training!!!
JingzeShi/Doge-20M-checkpoint
JingzeShi/Doge-60M-checkpoint

Post

1997

Only a single RTX 4090 running model pre-training is really slow, even for small language models!!! ( JingzeShi/doge-slm-677fd879f8c4fd0f43e05458)

Collections 1

Papers 1

arxiv:2412.11834

models 10

JingzeShi/Doge-160M-checkpoint

Text Generation • Updated about 3 hours ago • 74 • 1

JingzeShi/Doge-20M

Text Generation • Updated about 12 hours ago • 3.29k • 2

JingzeShi/Doge-60M-Instruct

Question Answering • Updated about 14 hours ago • 2.61k • 1

JingzeShi/Doge-20M-Instruct

Question Answering • Updated about 14 hours ago • 2.7k • 1

JingzeShi/Doge-60M-Instruct-SFT

Text Generation • Updated about 14 hours ago • 7

JingzeShi/Doge-20M-Instruct-SFT

Text Generation • Updated about 14 hours ago • 21

JingzeShi/Doge-60M-checkpoint

Text Generation • Updated about 14 hours ago • 7 • 1

JingzeShi/Doge-20M-checkpoint

Text Generation • Updated about 14 hours ago • 17 • 2

JingzeShi/Doge-60M

Text Generation • Updated about 14 hours ago • 3.16k • 1

JingzeShi/Doge-tokenizer

Updated 21 days ago

datasets

None public yet