jdzw2014 (jdzw2014)

liked a model 21 days ago

sthenno-com/miscii-14b-1225

Text Generation • Updated Dec 26, 2024 • 762 • 23

liked a dataset 2 months ago

PleIAs/common_corpus

Viewer • Updated Nov 22, 2024 • 397M • 7.68k • 198

updated a collection 5 months ago

great_dataset

Collection

1 item • Updated Aug 22, 2024

liked a dataset 5 months ago

fka/awesome-chatgpt-prompts

Viewer • Updated 22 days ago • 203 • 7.15k • 7.12k

upvoted an article 6 months ago

Article

How to Finetune phi-3 on MacBook Pro

By

•

Apr 24, 2024

• 65

reacted to fdaudens's post with 🚀 6 months ago

Post

2288

🚀 Introducing the Model Drops Tracker! 🕵️‍♂️

Feeling overwhelmed by the AI model release frenzy? 🤯 You're not alone!

I built this simple tool to help us all keep up:
- Filter recent models from the 🤗 Hub
- Set minimum likes threshold
- Choose how recent you want to go

Try it out and let me know what you think: fdaudens/Model-Drops-Tracker

Any features you'd like to see added?
#AIModels

3 replies

·

liked 2 datasets 6 months ago

zxbsmk/webnovel_cn

Viewer • Updated Aug 9, 2023 • 50k • 116 • 102

liwu/MNBVC

Updated Aug 23, 2024 • 17.7k • 510

reacted to thomwolf's post with ❤️ 7 months ago

Post

5136

A Little guide to building Large Language Models in 2024

This is a post-recording of a 75min lecture I gave two weeks ago on how to train a LLM from scratch in 2024. I tried to keep it short and comprehensive – focusing on concepts that are crucial for training good LLM but often hidden in tech reports.

In the lecture, I introduce the students to all the important concepts/tools/techniques for training good performance LLM:
* finding, preparing and evaluating web scale data
* understanding model parallelism and efficient training
* fine-tuning/aligning models
* fast inference

There is of course many things and details missing and that I should have added to it, don't hesitate to tell me you're most frustrating omission and I'll add it in a future part. In particular I think I'll add more focus on how to filter topics well and extensively and maybe more practical anecdotes and details.

Now that I recorded it I've been thinking this could be part 1 of a two-parts series with a 2nd fully hands-on video on how to run all these steps with some libraries and recipes we've released recently at HF around LLM training (and could be easily adapted to your other framework anyway):
*datatrove for all things web-scale data preparation: https://github.com/huggingface/datatrove
*nanotron for lightweight 4D parallelism LLM training: https://github.com/huggingface/nanotron
*lighteval for in-training fast parallel LLM evaluations: https://github.com/huggingface/lighteval

Here is the link to watch the lecture on Youtube: https://www.youtube.com/watch?v=2-SPH9hIKT8
And here is the link to the Google slides: https://docs.google.com/presentation/d/1IkzESdOwdmwvPxIELYJi8--K3EZ98_cL6c5ZcLKSyVg/edit#slide=id.p

Enjoy and happy to hear feedback on it and what to add, correct, extend in a second part.