What does exl2-4bpw-rpcal in the model name mean?
#1
by
BigDeeper
- opened
I don't see anything in the card to explain the difference with the source model.
This is the original model quantized in exllamav2 format up to 4-bit, using the calibration of the rp dataset. and yes, the extended context is achieved there by increasing the rope_theta parameter, I get acceptable results in long rp chats.