Mike Burnham's picture
3 1

Mike Burnham

mlburnham

AI & ML interests

None yet

Recent Activity

updated a dataset 8 days ago
mlburnham/polnli_hypothesis_stability
published a dataset 8 days ago
mlburnham/polnli_hypothesis_stability
updated a collection 14 days ago
Political DEBATE
View all activity

Organizations

None yet

mlburnham's activity

New activity in answerdotai/ModernBERT-base 17 days ago
replied to MoritzLaurer's post 24 days ago
view reply

Any plans to train a version of your zero-shot models on ModernBERT? I'm finding that ModernBERT is a huge boost in speed, and a slight drop in performance vs. DeBERTa when I tune it. Not sure if the performance drop is because your zero-shot models were such a strong foundation for transfer learning, or the strength of DeBERTa architecture on NLI.

reacted to MoritzLaurer's post with đź‘Ť about 1 month ago
view post
Post
2605
Quite excited by the ModernBERT release! 0.15/0.4B small, 2T modern pre-training data and tokenizer with code, 8k context window, great efficient model for embeddings & classification!

This will probably be the basis for many future SOTA encoders! And I can finally stop using DeBERTav3 from 2021 :D

Congrats @answerdotai , @LightOnIO and collaborators like @tomaarsen !

Paper and models here 👇https://huggingface.co/collections/answerdotai/modernbert-67627ad707a4acbf33c41deb
·
New activity in mlburnham/Pol_NLI 5 months ago
New activity in mlburnham/Pol_NLI 5 months ago