Spaces:

hf-accelerate
/

model-memory-usage

Running on CPU Upgrade

Memory scaling on input length

by marksverdhei - opened Aug 30, 2023

Aug 30, 2023

Will it be possible to add some sort of memory scaling estimate?
either like a widget where you enter n-tokens or a list of typical lengths like 512, 1k, 2k, 4k, 8k

muellerzr

accelerate org Aug 30, 2023

We're looking into this, definitely!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment