[Bug] Model Breaking Dynamic Cache

#69

by kylesayrs - opened 13 days ago

13 days ago

The line max_cache_length = past_key_values.get_max_length() will be incompatible with the next release of transformers. The line needs to be changed to use past_key_values.get_max_cache_shape()

https://github.com/huggingface/transformers/issues/36071

kylesayrs changed discussion title from [Bug] Dynamic Cache to [Bug] Model Breaking Dynamic Cache 13 days ago

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment