README.md

metadata

license: mit

Bo Li*¹  Yuanhan Zhang*^,1  Liangyu Chen*^,1  Jinghao Wang*^,1  Fanyi Pu*^,1 
Jingkang Yang¹  Chunyuan Li²  Ziwei Liu¹

¹S-Lab, Nanyang Technological University  ²Microsoft Research, Redmond

This weight is for initilizing training for Otter-MPT7B. It's directly converted from Openflamingov2, we added a <answer> token for Otter's downstream instruction tuning.

You can load and try this model using

model = OtterForConditionalGeneration.from_pretrained("luodian/OTTER-MPT7B-Init", device_map="sequential")
model.text_tokenizer.padding_side = "left"
tokenizer = model.text_tokenizer
image_processor = transformers.CLIPImageProcessor()
model.eval()

Leave us a message if you have any error or question. You can follow Otter code (see training section) to further tune your model on top of it.