 In the self-attention module, the word  correlation and positional correlation are computed separately with different parameterizations and then added together.
<meaning-changed>  In the self-attention module, the word  correlation and positional correlation are computed separately with different parameterizations and then added together.
 In the self-attention module, the word contextual correlation and positional correlation are computed separately with different parameterizations and then added together.