`use_cache=False`
#11
by
jemoka
- opened
Hi! Thanks so much for the great model! I wonder why the generation_config for this model doesn't use kv caching? (i.e. use_cache=False); is there a particular reason for this?
Thanks!
hmm, that wasn't intentional. Using cache should be fine
Great, will try it out with the KV cache on.