Unable to deploy
I tried to deploy this on sagemaker using the script provided, but what i'm getting is this error :
2024-09-15T16:43:16,088 [INFO ] W-liuhaotian__llava-v1.5-7b-3-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - KeyError: "Unknown task image-text-to-text, available tasks are ['audio-classification', 'automatic-speech-recognition', 'conversational', 'depth-estimation', 'document-question-answering', 'feature-extraction', 'fill-mask', 'image-classification', 'image-segmentation', 'image-to-image', 'image-to-text', 'mask-generation', 'ner', 'object-detection', 'question-answering', 'sentiment-analysis', 'summarization', 'table-question-answering', 'text-classification', 'text-generation', 'text-to-audio', 'text-to-speech', 'text2text-generation', 'token-classification', 'translation', 'video-classification', 'visual-question-answering', 'vqa', 'zero-shot-audio-classification', 'zero-shot-classification', 'zero-shot-image-classification', 'zero-shot-object-detection', 'translation_XX_to_YY']"
Try updating the version of transformers
huggingface_model = HuggingFaceModel(
transformers_version='4.44.0',
pytorch_version='2.1.0',
py_version='py310',
env=hub,
role=role,
)
You could also consider trying the TGI sagemaker container which can run llava-next (llava 1.6), mllama, and idefics