How to make 8 bit sharded model?
#1
by
a1nkit
- opened
Can you plz share the steps involved in making 8 bit sharded model?
hi! Simply load the model in 8-bit with the bitsandbytes integration as described here and then you can shard/push it as I described in this discussion on another model.
hope that helps and if you have further issues/questions feel free to comment here or reopen as needed.
pszemraj
changed discussion status to
closed