metadata

license: mit
datasets:
  - sagawa/ZINC-canonicalized
metrics:
  - accuracy
model-index:
  - name: ZINC-deberta
    results:
      - task:
          name: Masked Language Modeling
          type: fill-mask
        dataset:
          name: sagawa/ZINC-canonicalized
          type: sagawa/ZINC-canonicalized
        metrics:
          - name: Accuracy
            type: accuracy
            value: 0.9475839734077454

ZINC-t5

This model is a fine-tuned version of google/t5-v1_1-base on the sagawa/ZINC-canonicalized dataset. It achieves the following results on the evaluation set:

Loss: 0.1228
Accuracy: 0.9476

Model description

We trained t5 on SMILES from ZINC using the task of masked-language modeling (MLM). Compared to ZINC-t5, ZINC-t5-v2 uses a character-level tokenizer, and it was also trained on ZINC.

Intended uses & limitations

This model can be used for the prediction of molecules' properties, reactions, or interactions with proteins by changing the way of finetuning. As an example, We finetuned this model to predict products. The model is here, and you can use the demo here. Using its encoder, we trained a regression model to predict a reaction yield. You can use this demo here.

Training and evaluation data

We downloaded ZINC data and canonicalized them using RDKit. Then, we dropped duplicates. The total number of data is 22992522, and they were randomly split into train:validation=10:1.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-03
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 10.0

Training results

Training Loss	Step	Accuracy	Validation Loss
0.2090	100000	0.9264	0.1860
0.1628	200000	0.9349	0.1613
0.1632	300000	0.9395	0.1467
0.1451	400000	0.9435	0.1345
0.1311	500000	0.9465	0.1261