metadata

license: apache-2.0
base_model: google/electra-small-discriminator
tags:
  - generated_from_keras_callback
model-index:
  - name: nguyennghia0902/electra-small-discriminator_0.0001_16_15e
    results: []

nguyennghia0902/electra-small-discriminator_0.0001_16_15e

This model is a fine-tuned version of google/electra-small-discriminator on an unknown dataset. It achieves the following results on the evaluation set:

Train Loss: 0.4315
Train End Logits Accuracy: 0.8714
Train Start Logits Accuracy: 0.8580
Validation Loss: 0.1470
Validation End Logits Accuracy: 0.9577
Validation Start Logits Accuracy: 0.9542
Epoch: 14

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 0.0001, 'decay_steps': 46905, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
training_precision: float32

Training results

Train Loss	Train End Logits Accuracy	Train Start Logits Accuracy	Validation Loss	Validation End Logits Accuracy	Validation Start Logits Accuracy	Epoch
2.9418	0.3441	0.3115	2.1831	0.4777	0.4649	0
2.2767	0.4696	0.4357	1.7802	0.5643	0.5481	1
1.9907	0.5234	0.4941	1.5055	0.6229	0.6068	2
1.7630	0.5690	0.5440	1.2348	0.6824	0.6708	3
1.5637	0.6086	0.5842	1.0345	0.7291	0.7190	4
1.3785	0.6500	0.6241	0.8309	0.7823	0.7724	5
1.2118	0.6880	0.6604	0.6918	0.8105	0.8116	6
1.0610	0.7222	0.6963	0.5471	0.8490	0.8476	7
0.9249	0.7495	0.7272	0.4426	0.8770	0.8763	8
0.8085	0.7777	0.7585	0.3695	0.8919	0.8908	9
0.7062	0.8018	0.7843	0.2773	0.9194	0.9198	10
0.6182	0.8232	0.8043	0.2323	0.9343	0.9302	11
0.5422	0.8414	0.8267	0.1807	0.9470	0.9470	12
0.4797	0.8588	0.8443	0.1570	0.9530	0.9515	13
0.4315	0.8714	0.8580	0.1470	0.9577	0.9542	14

Framework versions

Transformers 4.39.3
TensorFlow 2.15.0
Datasets 2.18.0
Tokenizers 0.15.2