1.58k
The Ultra-Scale Playbook
🌌
The ultimate guide to training LLM on large GPU Clusters
Dmitry Ryumin
DmitryRyumin
AI & ML interests
Machine Learning and Applications, Multi-Modal Understanding
Recent Activity
liked
a Space
about 12 hours ago
nanotron/ultrascale-playbook
liked
a model
about 15 hours ago
yandex/YandexGPT-5-Lite-8B-pretrain
upvoted
a
paper
1 day ago
SurveyX: Academic Survey Automation via Large Language Models
Organizations
DmitryRyumin's activity
liked
a
Space
about 12 hours ago
liked
a
model
about 15 hours ago
upvoted
a
paper
1 day ago
reacted to
m-ric's
post with 🚀
1 day ago
Post
2595
We now have a Deep Research for academia: SurveyX automatically writes academic surveys nearly indistinguishable from human-written ones 🔥
Researchers from Beijing and Shanghai just published the first application of a deep research system to academia: their algorithm, given a question, can give you a survey of all papers on the subject.
To make a research survey, you generally follow two steps, preparation (collect and organize papers) and writing (outline creation, writing, polishing). Researchers followed the same two steps and automated them.
🎯 For the preparation part, a key part is find all the important references on the given subject.
Researchers first cast a wide net of all relevant papers. But then finding the really important ones is like distilling knowledge from a haystack of information. To solve this challenge, they built an “AttributeTree” object that structures key information from citations. Ablating these AttributeTrees significantly decreased structure and synthesis scores, so they were really useful!
📝 For the writing part, key was to get a synthesis that's both short and true. This is not easy to get with LLMs! So they used methods like LLM-based deduplication to shorten the too verbose listings made by LLMs, and RAG to grab original quotes instead of made-up ones.
As a result, their system outperforms previous approaches by far!
As assessed by LLM-judges, the quality score os SurveyX even approaches this of human experts, with 4.59/5 vs 4.75/5 🏆
I advise you to read the paper, it's a great overview of the kind of assistants that we'll get in the short future! 👉 SurveyX: Academic Survey Automation via Large Language Models (2502.14776)
Their website shows examples of generated surveys 👉 http://www.surveyx.cn/
Researchers from Beijing and Shanghai just published the first application of a deep research system to academia: their algorithm, given a question, can give you a survey of all papers on the subject.
To make a research survey, you generally follow two steps, preparation (collect and organize papers) and writing (outline creation, writing, polishing). Researchers followed the same two steps and automated them.
🎯 For the preparation part, a key part is find all the important references on the given subject.
Researchers first cast a wide net of all relevant papers. But then finding the really important ones is like distilling knowledge from a haystack of information. To solve this challenge, they built an “AttributeTree” object that structures key information from citations. Ablating these AttributeTrees significantly decreased structure and synthesis scores, so they were really useful!
📝 For the writing part, key was to get a synthesis that's both short and true. This is not easy to get with LLMs! So they used methods like LLM-based deduplication to shorten the too verbose listings made by LLMs, and RAG to grab original quotes instead of made-up ones.
As a result, their system outperforms previous approaches by far!
As assessed by LLM-judges, the quality score os SurveyX even approaches this of human experts, with 4.59/5 vs 4.75/5 🏆
I advise you to read the paper, it's a great overview of the kind of assistants that we'll get in the short future! 👉 SurveyX: Academic Survey Automation via Large Language Models (2502.14776)
Their website shows examples of generated surveys 👉 http://www.surveyx.cn/
reacted to
their
post with 🔥
5 days ago
Post
3524
🚀🎭🌟 New Research Alert - WACV 2025 (Avatars Collection)! 🌟🎭🚀
📄 Title: EmoVOCA: Speech-Driven Emotional 3D Talking Heads 🔝
📝 Description: EmoVOCA is a data-driven method for generating emotional 3D talking heads by combining speech-driven lip movements with expressive facial dynamics. This method has been developed to overcome the limitations of corpora and to achieve state-of-the-art animation quality.
👥 Authors: @FedeNoce , Claudio Ferrari, and Stefano Berretti
📅 Conference: WACV, 28 Feb – 4 Mar, 2025 | Arizona, USA 🇺🇸
📄 Paper: https://arxiv.org/abs/2403.12886
🌐 Github Page: https://fedenoce.github.io/emovoca/
📁 Repository: https://github.com/miccunifi/EmoVOCA
🚀 CVPR-2023-24-Papers: https://github.com/DmitryRyumin/CVPR-2023-24-Papers
🚀 WACV-2024-Papers: https://github.com/DmitryRyumin/WACV-2024-Papers
🚀 ICCV-2023-Papers: https://github.com/DmitryRyumin/ICCV-2023-Papers
📚 More Papers: more cutting-edge research presented at other conferences in the DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin
🚀 Added to the Avatars Collection: DmitryRyumin/avatars-65df37cdf81fec13d4dbac36
🔍 Keywords: #EmoVOCA #3DAnimation #TalkingHeads #SpeechDriven #FacialExpressions #MachineLearning #ComputerVision #ComputerGraphics #DeepLearning #AI #WACV2024
📄 Title: EmoVOCA: Speech-Driven Emotional 3D Talking Heads 🔝
📝 Description: EmoVOCA is a data-driven method for generating emotional 3D talking heads by combining speech-driven lip movements with expressive facial dynamics. This method has been developed to overcome the limitations of corpora and to achieve state-of-the-art animation quality.
👥 Authors: @FedeNoce , Claudio Ferrari, and Stefano Berretti
📅 Conference: WACV, 28 Feb – 4 Mar, 2025 | Arizona, USA 🇺🇸
📄 Paper: https://arxiv.org/abs/2403.12886
🌐 Github Page: https://fedenoce.github.io/emovoca/
📁 Repository: https://github.com/miccunifi/EmoVOCA
🚀 CVPR-2023-24-Papers: https://github.com/DmitryRyumin/CVPR-2023-24-Papers
🚀 WACV-2024-Papers: https://github.com/DmitryRyumin/WACV-2024-Papers
🚀 ICCV-2023-Papers: https://github.com/DmitryRyumin/ICCV-2023-Papers
📚 More Papers: more cutting-edge research presented at other conferences in the DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin
🚀 Added to the Avatars Collection: DmitryRyumin/avatars-65df37cdf81fec13d4dbac36
🔍 Keywords: #EmoVOCA #3DAnimation #TalkingHeads #SpeechDriven #FacialExpressions #MachineLearning #ComputerVision #ComputerGraphics #DeepLearning #AI #WACV2024
- 1 reply
posted
an
update
5 days ago
Post
3524
🚀🎭🌟 New Research Alert - WACV 2025 (Avatars Collection)! 🌟🎭🚀
📄 Title: EmoVOCA: Speech-Driven Emotional 3D Talking Heads 🔝
📝 Description: EmoVOCA is a data-driven method for generating emotional 3D talking heads by combining speech-driven lip movements with expressive facial dynamics. This method has been developed to overcome the limitations of corpora and to achieve state-of-the-art animation quality.
👥 Authors: @FedeNoce , Claudio Ferrari, and Stefano Berretti
📅 Conference: WACV, 28 Feb – 4 Mar, 2025 | Arizona, USA 🇺🇸
📄 Paper: https://arxiv.org/abs/2403.12886
🌐 Github Page: https://fedenoce.github.io/emovoca/
📁 Repository: https://github.com/miccunifi/EmoVOCA
🚀 CVPR-2023-24-Papers: https://github.com/DmitryRyumin/CVPR-2023-24-Papers
🚀 WACV-2024-Papers: https://github.com/DmitryRyumin/WACV-2024-Papers
🚀 ICCV-2023-Papers: https://github.com/DmitryRyumin/ICCV-2023-Papers
📚 More Papers: more cutting-edge research presented at other conferences in the DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin
🚀 Added to the Avatars Collection: DmitryRyumin/avatars-65df37cdf81fec13d4dbac36
🔍 Keywords: #EmoVOCA #3DAnimation #TalkingHeads #SpeechDriven #FacialExpressions #MachineLearning #ComputerVision #ComputerGraphics #DeepLearning #AI #WACV2024
📄 Title: EmoVOCA: Speech-Driven Emotional 3D Talking Heads 🔝
📝 Description: EmoVOCA is a data-driven method for generating emotional 3D talking heads by combining speech-driven lip movements with expressive facial dynamics. This method has been developed to overcome the limitations of corpora and to achieve state-of-the-art animation quality.
👥 Authors: @FedeNoce , Claudio Ferrari, and Stefano Berretti
📅 Conference: WACV, 28 Feb – 4 Mar, 2025 | Arizona, USA 🇺🇸
📄 Paper: https://arxiv.org/abs/2403.12886
🌐 Github Page: https://fedenoce.github.io/emovoca/
📁 Repository: https://github.com/miccunifi/EmoVOCA
🚀 CVPR-2023-24-Papers: https://github.com/DmitryRyumin/CVPR-2023-24-Papers
🚀 WACV-2024-Papers: https://github.com/DmitryRyumin/WACV-2024-Papers
🚀 ICCV-2023-Papers: https://github.com/DmitryRyumin/ICCV-2023-Papers
📚 More Papers: more cutting-edge research presented at other conferences in the DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin
🚀 Added to the Avatars Collection: DmitryRyumin/avatars-65df37cdf81fec13d4dbac36
🔍 Keywords: #EmoVOCA #3DAnimation #TalkingHeads #SpeechDriven #FacialExpressions #MachineLearning #ComputerVision #ComputerGraphics #DeepLearning #AI #WACV2024
-
- 1 reply
updated
a
collection
5 days ago
Fixed typos
#2 opened 18 days ago
by
markitantov
Reset the tracked ID
#1 opened 18 days ago
by
markitantov
upvoted
a
collection
21 days ago
upvoted
a
paper
23 days ago
liked
a
Space
26 days ago
reacted to
not-lain's
post with 🔥
27 days ago
Post
3957
I have just released a new blogpost about kv caching and its role in inference speedup 🚀
🔗 https://huggingface.co/blog/not-lain/kv-caching/
some takeaways :
🔗 https://huggingface.co/blog/not-lain/kv-caching/
some takeaways :
- 4 replies
liked
2
Spaces
28 days ago
liked
a
Space
about 1 month ago
updated
a
Space
about 2 months ago
updated
a
collection
2 months ago
liked
2
Spaces
2 months ago