Fixing "RuntimeError: expected scalar type Half but found Float" error
#46
by
marygm
- opened
Change in Commit "85c1f1c201273bbfee661d4a2f8307c95f8956c9" is raising an error for me when using the model for 8bit inference.
When I returned back the changes, it fixed the error.
I am encountering this problem and this seems like a solution.
How can I use it before it gets merged?
I tried setting revision="8b2b3a65dea7ccb56b79a2e596aab76a4df4cf42" in AutoModelForCausalLM.from_pretrained() but it gave me no effect.
@Patters
using the previous commit works also.
revision=“ fe31052a3aeb03f1b709c38471605d0af0486fe8”