orel lavie

orel12's activity

• All
• Models
• Datasets
• Spaces
• Papers
• Collections
• Community
• Posts
• Upvotes
• Likes

New activity in answerdotai/ModernBERT-base about 2 months ago

How to see which version of Transformers library is needed to get access to this model

16

#3 opened about 2 months ago by

aero-artem

New activity in jarrelscy/Mixtral-8x22B-Instruct-v0.1-GPTQ-4bit 10 months ago

I am running in vllm 0.4.1 with 4 x gpus 24gb (A10G 24gb) = 96gb and eager mode and I am still out of memory, how? it should fit (like 87gb vram)

1

#3 opened 10 months ago by

orel12

KeyError: 'model.layers.45.block_sparse_moe.gate.g_idx'

5

#2 opened 10 months ago by

tutu329