InfiniteHiP: Extending Language Model Context Up to 3 Million Tokens on a Single GPU
Paper
•
2502.08910
•
Published
•
141
Long context
-
-
Cramming 1568 Tokens into a Single Vector and Back Again: Exploring the Limits of Embedding Space Capacity
Paper • 2502.13063 • Published • 62 -
Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
Paper • 2502.11089 • Published • 134 -
LLM Pretraining with Continuous Concepts
Paper • 2502.08524 • Published • 27