Update README.md
Browse files
README.md
CHANGED
|
@@ -84,7 +84,7 @@ Average MSE of last hidden state: 2.46e-09
|
|
| 84 |
You can load the weights from the ESM package instead of transformers by replacing .from_pretrained(...) to .from_pretrained_esm('esmc_600m')
|
| 85 |
|
| 86 |
## Model probes
|
| 87 |
-
We employ linear probing techniques on various PLMs and standard datasets, similar our previous [paper](https://www.biorxiv.org/content/10.1101/2024.07.30.605924v1), to
|
| 88 |
|
| 89 |
The plot below showcases performance normalized between the negative control (random vector embeddings) and the best performer. Classification task scores are averaged between MCC and F1 (or F1max for multilabel) and regression tasks are averaged between Spearman rho and R2.
|
| 90 |
|
|
|
|
| 84 |
You can load the weights from the ESM package instead of transformers by replacing .from_pretrained(...) to .from_pretrained_esm('esmc_600m')
|
| 85 |
|
| 86 |
## Model probes
|
| 87 |
+
We employ linear probing techniques on various PLMs and standard datasets, similar our previous [paper](https://www.biorxiv.org/content/10.1101/2024.07.30.605924v1), to assess the intrinsic correlation between pooled hidden states and valuable properties. ESMC (and thus ESM++) perform very well.
|
| 88 |
|
| 89 |
The plot below showcases performance normalized between the negative control (random vector embeddings) and the best performer. Classification task scores are averaged between MCC and F1 (or F1max for multilabel) and regression tasks are averaged between Spearman rho and R2.
|
| 90 |
|