---
license: mit
tags:
- generated_from_trainer
base_model: facebook/w2v-bert-2.0
metrics:
- wer
model-index:
- name: w2v-bert-2.0-tamil-gpu-custom_clean_v2
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# w2v-bert-2.0-tamil-gpu-custom_clean_v2

This model is a fine-tuned version of [facebook/w2v-bert-2.0](https://huggingface.co/facebook/w2v-bert-2.0) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1042
- Wer: 0.1892

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2.5356e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 10
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Wer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| 2.9949        | 0.25  | 300   | 0.5158          | 0.6736 |
| 0.4051        | 0.5   | 600   | 0.1858          | 0.3035 |
| 0.2789        | 0.76  | 900   | 0.1670          | 0.2730 |
| 0.2352        | 1.01  | 1200  | 0.1479          | 0.2594 |
| 0.1988        | 1.26  | 1500  | 0.1428          | 0.2464 |
| 0.1879        | 1.51  | 1800  | 0.1388          | 0.2391 |
| 0.1757        | 1.76  | 2100  | 0.1244          | 0.2412 |
| 0.1683        | 2.01  | 2400  | 0.1278          | 0.2231 |
| 0.1475        | 2.27  | 2700  | 0.1231          | 0.2240 |
| 0.1438        | 2.52  | 3000  | 0.1219          | 0.2192 |
| 0.1422        | 2.77  | 3300  | 0.1216          | 0.2128 |
| 0.1337        | 3.02  | 3600  | 0.1145          | 0.2087 |
| 0.1212        | 3.27  | 3900  | 0.1131          | 0.2061 |
| 0.1195        | 3.52  | 4200  | 0.1159          | 0.2147 |
| 0.1169        | 3.78  | 4500  | 0.1111          | 0.2083 |
| 0.1162        | 4.03  | 4800  | 0.1116          | 0.2058 |
| 0.1073        | 4.28  | 5100  | 0.1138          | 0.2114 |
| 0.1011        | 4.53  | 5400  | 0.1103          | 0.2057 |
| 0.1008        | 4.78  | 5700  | 0.1096          | 0.2018 |
| 0.1016        | 5.03  | 6000  | 0.1045          | 0.2008 |
| 0.092         | 5.29  | 6300  | 0.1104          | 0.2008 |
| 0.0889        | 5.54  | 6600  | 0.1079          | 0.2005 |
| 0.0936        | 5.79  | 6900  | 0.1036          | 0.2026 |
| 0.0888        | 6.04  | 7200  | 0.1106          | 0.2109 |
| 0.0836        | 6.29  | 7500  | 0.1115          | 0.2103 |
| 0.0807        | 6.54  | 7800  | 0.1104          | 0.2045 |
| 0.0807        | 6.8   | 8100  | 0.1051          | 0.2039 |
| 0.0784        | 7.05  | 8400  | 0.1067          | 0.1947 |
| 0.0719        | 7.3   | 8700  | 0.1051          | 0.1957 |
| 0.0735        | 7.55  | 9000  | 0.1084          | 0.1894 |
| 0.0715        | 7.8   | 9300  | 0.1029          | 0.1916 |
| 0.0732        | 8.05  | 9600  | 0.1059          | 0.1894 |
| 0.0673        | 8.31  | 9900  | 0.1053          | 0.1890 |
| 0.0642        | 8.56  | 10200 | 0.1042          | 0.1879 |
| 0.0669        | 8.81  | 10500 | 0.1039          | 0.1877 |
| 0.0665        | 9.06  | 10800 | 0.1043          | 0.1881 |
| 0.0606        | 9.31  | 11100 | 0.1027          | 0.1870 |
| 0.0615        | 9.56  | 11400 | 0.1046          | 0.1887 |
| 0.0602        | 9.82  | 11700 | 0.1042          | 0.1892 |


### Framework versions

- Transformers 4.39.3
- Pytorch 2.1.2+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2