Linear Transformers with Learnable Kernel Functions are Better In-Context Models
Paper
•
2402.10644
•
Published
•
80
Transformers & Attention
-
-
Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models
Paper • 2401.04658 • Published • 27 -
KAN: Kolmogorov-Arnold Networks
Paper • 2404.19756 • Published • 109 -
Your Transformer is Secretly Linear
Paper • 2405.12250 • Published • 151 -
Reducing Transformer Key-Value Cache Size with Cross-Layer Attention
Paper • 2405.12981 • Published • 29 -
Block Transformer: Global-to-Local Language Modeling for Fast Inference
Paper • 2406.02657 • Published • 38