dimension error
When I use gemma-2-2b-it to generate, I met this error:
padding_mask = causal_mask[:, :, :, :mask_length] + attention_mask[:, None, None, :]
[rank0]: RuntimeError: The size of tensor a (120) must match the size of tensor b (121) at non-singleton dimension 3
my transformers version is: transformers 4.44.1
Hi @Yingshu ,
Could you please check the shapes of both tensors and ensure that their shapes match so the addition can be performed correctly. The issue arises because the third dimension of the tensors at index 0 corresponds to [dim0, dim1, dim2, dim3], and in this case, the third dimensions don't match. Specifically, the third dimension of causal_mask[3] is 120, while the third dimension of attention_mask is 121.
To resolve this error, you can try using the following code:
padding_mask = causal_mask[:, :, :, :(mask_length+1)] + attention_mask[:, None, None, :]
Thank you.