Getting error while using the model for OnnxScoringModel in java
I am getting an error :ai.onnxruntime.OrtException: Error code - ORT_RUNTIME_EXCEPTION - message: Non-zero status code returned while running Shape node. Name:'/model/rotary_emb/Shape' Status Message: /Users/runner/work/1/s/include/onnxruntime/core/framework/op_kernel_context.h:42 const T *onnxruntime::OpKernelContext::Input(int) const [T = onnxruntime::Tensor] Missing Input: position_ids
when used in java springboot project which uses langchain4j as one of the depedencies,
the piece of code is :
import dev.langchain4j.data.segment.TextSegment;
import dev.langchain4j.model.output.Response;
import dev.langchain4j.model.scoring.ScoringModel;
import dev.langchain4j.model.scoring.onnx.OnnxScoringModel;
ScoringModel scoringModel = new OnnxScoringModel("pathToModel.onnx", "pathTotokenizer.json");
Response<List<Double>> response = scoringModel.scoreAll(TextSegments, "Sample String");```
I have also saved the model.onnx_data in the same path,
My use case it to compare 2 strings and get the similarity score of them.
when using the model https://huggingface.co/BAAI/bge-reranker-base, this one works fine where it has only the model.onnx i am trying to figure it out how the granite onnx model with model data can work
As the error suggests, it is missing position_ids, which is a 2D tensor specifying the positions of the input ids.
However, a bigger problem is that the two models have different architectures and output signatures, and I would imagine that OnnxScoringModel is not compatible with causal language models.
Thanks
@Xenova
for the reply,
can you guide me about how i can use the granite onnx model in any springboot java project in which i would pass 2 args (list string, string) to the scoring model or any other such implementations which would give me the similarity score? any references you might be aware of?
Any leads would help,
I am trying to implement this model.onnx in springboot java project to cover this use case.
I would recommend asking the langchain4j team, as they are far more familiar with this than I am :)