3 11

Haoze Wu

WaitHZ

https://waithz.github.io/

AI & ML interests

Modular DL, Complex Reasoning

Recent Activity

commented on a paper 1 day ago

HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization

upvoted a paper 1 day ago

HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization

commented on a paper 1 day ago

HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization

View all activity

Organizations

None yet

WaitHZ's activity

commented a paper 1 day ago

HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization

Paper • 2503.04598 • Published 4 days ago • 16 •

upvoted a paper 1 day ago

HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization

Paper • 2503.04598 • Published 4 days ago • 16

commented a paper 1 day ago

HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization

Paper • 2503.04598 • Published 4 days ago • 16 •

upvoted an article 7 days ago

Article

Open-R1: Update #1

and 7 others •

Feb 2

• 293

upvoted a paper 25 days ago

Mask-Enhanced Autoregressive Prediction: Pay Less Attention to Learn More

Paper • 2502.07490 • Published 27 days ago • 9

upvoted 2 articles about 1 month ago

Article

How to generate text: using different decoding methods for language generation with Transformers

Mar 1, 2020

• 165

Article

You could have designed state of the art positional encoding

Nov 25, 2024

• 179

upvoted a paper about 1 month ago

Sigma: Differential Rescaling of Query, Key and Value for Efficient Language Models

Paper • 2501.13629 • Published Jan 23 • 44

upvoted a paper about 2 months ago

Autonomy-of-Experts Models

Paper • 2501.13074 • Published Jan 22 • 41

commented a paper about 2 months ago

Autonomy-of-Experts Models

Paper • 2501.13074 • Published Jan 22 • 41 •

upvoted 2 papers 6 months ago

Benchmarking Chinese Knowledge Rectification in Large Language Models

Paper • 2409.05806 • Published Sep 9, 2024 • 15

OneGen: Efficient One-Pass Unified Generation and Retrieval for LLMs

Paper • 2409.05152 • Published Sep 8, 2024 • 32

upvoted a paper 8 months ago

GW-MoE: Resolving Uncertainty in MoE Router with Global Workspace Theory

Paper • 2406.12375 • Published Jun 18, 2024 • 1

authored a paper 8 months ago

GW-MoE: Resolving Uncertainty in MoE Router with Global Workspace Theory

Paper • 2406.12375 • Published Jun 18, 2024 • 1

upvoted a paper 11 months ago

Mixture-of-Depths: Dynamically allocating compute in transformer-based language models

Paper • 2404.02258 • Published Apr 2, 2024 • 104

New activity in deepseek-ai/deepseek-moe-16b-base 12 months ago

A little question about aux_loss

#4 opened about 1 year ago by

WaitHZ

A little question about aux_loss

#4 opened about 1 year ago by

WaitHZ

New activity in deepseek-ai/deepseek-moe-16b-base about 1 year ago

A little question about aux_loss

#4 opened about 1 year ago by

WaitHZ