Bad responses for longer context
#6
by
sidhantthole
- opened
Model doesnt match up with 32k promise from the original one. I guess there is some issue with quantization due to sliding window as pointed in another discussion Whie the other one Casperhanssen does well