 Extensive experiments on our generated probe sets  show that PTLMs perform no better than random guessing (even with fine-tuning) , are heavily impacted by statistical biases, and are not robust to perturbation attacks.
<meaning-changed>  Extensive experiments on our generated probe sets  show that PTLMs perform no better than random guessing (even with fine-tuning) , are heavily impacted by statistical biases, and are not robust to perturbation attacks.
 Extensive experiments on our generated probe sets with more than 10k statements show that PTLMs perform no better than random guessing (even with fine-tuning) , are heavily impacted by statistical biases, and are not robust to perturbation attacks.