ydshieh

/

kosmos-2-patch14-224

like 55

Is there a way to get likelihoods from the model for a pre-defined answer? Something like this:

inputs = processor(text=prompt, images=image, return_tensors="pt")
targets = processor(text=answer, return_tensors="pt")
output = model(
    pixel_values=inputs["pixel_values"],
    input_ids=inputs["input_ids"][:, :-1],
    attention_mask=inputs["attention_mask"][:, :-1],
    img_features=None,
    img_attn_mask=inputs["img_attn_mask"][:, :-1],
    use_cache=False,
    labels=targets
)
nll_loss = output[0]