eliebak (Elie Bakouch)

upvoted a paper about 19 hours ago

To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning

Paper • 2409.12183 • Published 1 day ago • 14

upvoted an article 1 day ago

Article

Fine-tuning LLMs to 1.58bit: extreme quantization made easy

2 days ago

• 96

upvoted a paper 22 days ago

Enhancing Training Efficiency Using Packing with Flash Attention

Paper • 2407.09105 • Published Jul 12 • 12

upvoted a paper 23 days ago

Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler

Paper • 2408.13359 • Published 27 days ago • 21

upvoted a collection 23 days ago

Power-LM

Collection

Dense & MoE LLMs trained with power learning rate scheduler. • 3 items • Updated 8 days ago • 13

upvoted 2 articles 24 days ago

Article

Introducing AuraFace: Open-Source Face Recognition and Identity Preservation Models

By

•

24 days ago

• 34

Article

MicroJAX

By

•

25 days ago

• 13

upvoted a paper 29 days ago

To Code, or Not To Code? Exploring Impact of Code in Pre-training

Paper • 2408.10914 • Published about 1 month ago • 39

upvoted 2 collections about 1 month ago

💻 Local SmolLMs

Collection

SmolLM models in MLC, ONNX and GGUF format for local applications + in-browser demos • 14 items • Updated about 1 month ago • 40

🪐 SmolLM

Collection

A series of smol LLMs: 135M, 360M and 1.7B. We release base and Instruct models as well as the training corpus and some WebGPU demos • 12 items • Updated Aug 18 • 169

upvoted an article about 1 month ago

Article

A failed experiment: Infini-Attention, and why we should keep trying?

Aug 14

• 40

upvoted a paper about 1 month ago

LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs

Paper • 2408.07055 • Published Aug 13 • 65

upvoted an article about 1 month ago

Article

Introduction to ggml

Aug 13

• 91

upvoted a paper about 1 month ago

Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters

Paper • 2408.04093 • Published Aug 7 • 4

upvoted an article about 2 months ago

Article

The case for specialized pre-training: ultra-fast foundation models for dedicated tasks

By

•

Aug 4

• 24

upvoted 3 papers about 2 months ago

The Llama 3 Herd of Models

Paper • 2407.21783 • Published Jul 31 • 102

Sailor: Open Language Models for South-East Asia

Paper • 2404.03608 • Published Apr 4 • 20

Data Mixture Inference: What do BPE Tokenizers Reveal about their Training Data?

Paper • 2407.16607 • Published Jul 23 • 21

upvoted an article about 2 months ago

Article

LAVE: Zero-shot VQA Evaluation on Docmatix with LLMs - Do We Still Need Fine-Tuning?

Jul 25

• 18

upvoted an article 2 months ago

Article

SmolLM - blazingly fast and remarkably powerful

Jul 16

• 242

upvoted 2 papers 2 months ago

Qwen2 Technical Report

Paper • 2407.10671 • Published Jul 15 • 153

MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases

Paper • 2402.14905 • Published Feb 22 • 107

upvoted 2 papers 3 months ago

Let the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language Models

Paper • 2407.01906 • Published Jul 2 • 33

The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale

Paper • 2406.17557 • Published Jun 25 • 84

upvoted a collection 3 months ago

Nemotron 4 340B

Collection

Nemotron-4: open models for Synthetic Data Generation (SDG). Includes Base, Instruct, and Reward models. • 4 items • Updated Jul 17 • 156

upvoted a paper 4 months ago

Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations

Paper • 2405.18392 • Published May 28 • 12

upvoted 2 collections 5 months ago

🍷 FineWeb datasets

Collection

5 items • Updated Jun 26 • 15

[lecture artifacts] aligning open language models

Collection

artifacts referenced in the talk timeline! Slides: https://docs.google.com/presentation/d/1quMyI4BAx4rvcDfk8jjv063bmHg4RxZd9mhQloXpMn0/edit?usp=sharin • 63 items • Updated Apr 17 • 55

upvoted a paper 8 months ago

Secrets of RLHF in Large Language Models Part II: Reward Modeling

Paper • 2401.06080 • Published Jan 11 • 24

Elie Bakouch

AI & ML interests

Articles

SmolLM - blazingly fast and remarkably powerful

Organizations

eliebak's activity

To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning

Fine-tuning LLMs to 1.58bit: extreme quantization made easy

Enhancing Training Efficiency Using Packing with Flash Attention

Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler

Power-LM

Introducing AuraFace: Open-Source Face Recognition and Identity Preservation Models

MicroJAX

To Code, or Not To Code? Exploring Impact of Code in Pre-training

💻 Local SmolLMs

🪐 SmolLM

A failed experiment: Infini-Attention, and why we should keep trying?

LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs

Introduction to ggml

Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters

The case for specialized pre-training: ultra-fast foundation models for dedicated tasks

The Llama 3 Herd of Models

Sailor: Open Language Models for South-East Asia

Data Mixture Inference: What do BPE Tokenizers Reveal about their Training Data?

LAVE: Zero-shot VQA Evaluation on Docmatix with LLMs - Do We Still Need Fine-Tuning?

SmolLM - blazingly fast and remarkably powerful

Qwen2 Technical Report

MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases

Let the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language Models

The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale

Nemotron 4 340B

Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations

🍷 FineWeb datasets

[lecture artifacts] aligning open language models

Secrets of RLHF in Large Language Models Part II: Reward Modeling