MIDICausalFinetuningFromFolder

This model was trained from scratch on the generator dataset. It achieves the following results on the evaluation set:

• Loss: 2.1183

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
2.6992	1.0	959	2.6253
2.6438	2.0	1918	2.5918
2.6098	3.0	2877	2.5617
2.5847	4.0	3836	2.5325
2.5604	5.0	4795	2.5040
2.5433	6.0	5754	2.4798
2.5203	7.0	6713	2.4534
2.4956	8.0	7672	2.4253
2.4787	9.0	8631	2.4038
2.4582	10.0	9590	2.3793
2.4419	11.0	10549	2.3574
2.4246	12.0	11508	2.3354
2.4114	13.0	12467	2.3144
2.3916	14.0	13426	2.2958
2.3882	15.0	14385	2.2784
2.364	16.0	15344	2.2607
2.3512	17.0	16303	2.2440
2.3444	18.0	17262	2.2292
2.3261	19.0	18221	2.2123
2.3165	20.0	19180	2.1988
2.3071	21.0	20139	2.1844
2.2968	22.0	21098	2.1729
2.2868	23.0	22057	2.1625
2.2777	24.0	23016	2.1522
2.2666	25.0	23975	2.1428
2.2619	26.0	24934	2.1365
2.2616	27.0	25893	2.1291
2.2563	28.0	26852	2.1228
2.2482	29.0	27811	2.1197
2.2493	30.0	28770	2.1183

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1