Saved model is missing the pooler.dense.weight
#2
by
AngledLuffa
- opened
The model saved on HF is missing pooler.dense.weight and pooler.dense.bias. The weight gets initialized randomly, and the bias to 0, which presumably hurts performance compared to having the actual pooler saved. Also, the random initialization means that if the transformer is used as an encoder without finetuning, the model produces non-deterministic results.
https://github.com/stanfordnlp/stanza/issues/1301
It would be great if this could be updated with a fixed version of the model that includes all of the parameters relevant to inference. Thanks!