How much VRAM is required to run inference?

by valdanito - opened Jul 3, 2023

Jul 3, 2023

I use A100 40G machine still OOM.

Liskie

Sep 11, 2023

I use A100 40G machine still OOM.

I use A100 80G and ~75G are occupied during inference.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment