Adapter selection for training with retrieval data

#71
by hail75 - opened

model = SentenceTransformer("jinaai/jina-embeddings-v3", trust_remote_code=True, model_kwargs={'default_task': 'classification'})
For retrieval task, when initializing can the task be "retrieval" or i must fine tuning one model for query and one for passage, thank you

hail75 changed discussion title from Training for retrieval task to Adapter selection for training with retrieval data

Hi @hail75 , for retrieval, both the query and passage adapters were fine-tuned together, one to encode queries and the other to encode passages. Unfortunately, this kind of fine-tuning isn't currently supported in SentenceTransformers. So I would recommend selecting either the query or passage adapter and fine-tuning just that adapter for your retrieval task.

So you mean i need fine tune two seperate adapter for each task when infer right?. Thanks

No, fine-tuning both adapters for retrieval isn't possible without custom training code, so you might consider tuning just one adapter for both queries and passages. This approach should work well, our experiments showed that the two-adapter setup was only slightly better than using a single adapter.

Sign up or log in to comment