S1_InstructionGeneratorGamma

This model is a fine-tuned version of distilgpt2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0763

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 48
• eval_batch_size: 48
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 24

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	230	0.0822
No log	2.0	460	0.0809
0.0914	3.0	690	0.0801
0.0914	4.0	920	0.0788
0.0865	5.0	1150	0.0785
0.0865	6.0	1380	0.0780
0.0841	7.0	1610	0.0781
0.0841	8.0	1840	0.0778
0.0824	9.0	2070	0.0774
0.0824	10.0	2300	0.0770
0.0817	11.0	2530	0.0771
0.0817	12.0	2760	0.0769
0.0817	13.0	2990	0.0769
0.0806	14.0	3220	0.0766
0.0806	15.0	3450	0.0765
0.0799	16.0	3680	0.0765
0.0799	17.0	3910	0.0765
0.0793	18.0	4140	0.0764
0.0793	19.0	4370	0.0763
0.0789	20.0	4600	0.0763
0.0789	21.0	4830	0.0762
0.0786	22.0	5060	0.0763
0.0786	23.0	5290	0.0763
0.0784	24.0	5520	0.0763

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2