outputs

This model is a fine-tuned version of openai-community/gpt2 on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.0494

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 1
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 4
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 0.03
  • num_epochs: 100

Training results

Training Loss Epoch Step Validation Loss
2.9872 4.0 184 1.7517
1.769 8.0 368 1.3586
1.5071 12.0 552 1.2644
1.4096 16.0 736 1.2088
1.3405 20.0 920 1.1738
1.2971 24.0 1104 1.1478
1.2609 28.0 1288 1.1241
1.2328 32.0 1472 1.1112
1.2087 36.0 1656 1.0996
1.1919 40.0 1840 1.0918
1.1687 44.0 2024 1.0868
1.157 48.0 2208 1.0768
1.1436 52.0 2392 1.0724
1.1296 56.0 2576 1.0684
1.1147 60.0 2760 1.0657
1.1082 64.0 2944 1.0622
1.102 68.0 3128 1.0578
1.0925 72.0 3312 1.0573
1.0865 76.0 3496 1.0533
1.0786 80.0 3680 1.0534
1.076 84.0 3864 1.0523
1.0764 88.0 4048 1.0505
1.0685 92.0 4232 1.0495
1.0684 96.0 4416 1.0489
1.0662 100.0 4600 1.0494

Framework versions

  • PEFT 0.10.0
  • Transformers 4.39.3
  • Pytorch 2.2.1+cu121
  • Datasets 2.18.0
  • Tokenizers 0.15.2
Downloads last month
7
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for Ikeofai/outputs

Adapter
(1609)
this model