PromoGen_K562_2080Ti_restart

This model is a fine-tuned version of on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4624

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 1000
• num_epochs: 25
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.7676	0.49	2500	0.7383
0.7121	0.97	5000	0.6867
0.6914	1.46	7500	0.6705
0.6837	1.95	10000	0.6622
0.6778	2.44	12500	0.6558
0.6748	2.92	15000	0.6517
0.6676	3.41	17500	0.6433
0.6593	3.9	20000	0.6358
0.6584	4.38	22500	0.6320
0.6557	4.87	25000	0.6301
0.6523	5.36	27500	0.6257
0.6478	5.84	30000	0.6236
0.6393	6.33	32500	0.6145
0.6039	6.82	35000	0.5658
0.5616	7.31	37500	0.5376
0.5518	7.79	40000	0.5310
0.5509	8.28	42500	0.5273
0.5487	8.77	45000	0.5261
0.5479	9.25	47500	0.5249
0.546	9.74	50000	0.5242
0.5447	10.23	52500	0.5229
0.5439	10.71	55000	0.5220
0.5433	11.2	57500	0.5209
0.5394	11.69	60000	0.5162
0.5153	12.18	62500	0.4944
0.5137	12.66	65000	0.4932
0.514	13.15	67500	0.4924
0.5131	13.64	70000	0.4919
0.5104	14.12	72500	0.4914
0.5122	14.61	75000	0.4906
0.5089	15.1	77500	0.4901
0.5076	15.59	80000	0.4891
0.4986	16.07	82500	0.4721
0.4875	16.56	85000	0.4672
0.4887	17.05	87500	0.4669
0.4839	17.53	90000	0.4661
0.4849	18.02	92500	0.4654
0.4848	18.51	95000	0.4649
0.4831	18.99	97500	0.4646
0.4816	19.48	100000	0.4644
0.4808	19.97	102500	0.4637
0.4812	20.46	105000	0.4634
0.4813	20.94	107500	0.4633
0.4818	21.43	110000	0.4631
0.4813	21.92	112500	0.4629
0.4782	22.4	115000	0.4628
0.4804	22.89	117500	0.4626
0.4815	23.38	120000	0.4625
0.4812	23.87	122500	0.4625
0.4785	24.35	125000	0.4624
0.4795	24.84	127500	0.4624

Framework versions

• Transformers 4.24.0
• Pytorch 1.13.0
• Datasets 2.7.0
• Tokenizers 0.13.0.dev0