PATTARA TIPAKSORN
commited on
Commit
•
4b56629
1
Parent(s):
0656ce9
Update README.md
Browse files
README.md
CHANGED
@@ -63,7 +63,12 @@ with torch.no_grad():
|
|
63 |
print(response[0])
|
64 |
```
|
65 |
## Evaluation Performance
|
66 |
-
|
|
|
|
|
|
|
|
|
|
|
67 |
|
68 |
## Limitations and Future Work
|
69 |
At present, our model remains in the experimental research phase and is not yet fully suitable for practical applications as an assistant. Future work will focus on upgrading the language model to a newer version [Pathumma-llm-text-1.0.0](https://huggingface.co/nectec/Pathumma-llm-text-1.0.0), and curating more refined and robust datasets to improve performance. Additionally, we aim to address and prioritize the safety and reliability of the model's outputs.
|
|
|
63 |
print(response[0])
|
64 |
```
|
65 |
## Evaluation Performance
|
66 |
+
| Model | ASR-th CV18 th (WER↓) | ASR-en CV18 En (WER↓) | ASR-en Librispeech En (WER↓) | ThaiSER Emotion (Acc↑, F1↑)| ThaiSER Gender (Acc↑, F1↑) |
|
67 |
+
|:----------------------------:|:------------------------:|:------------------------:|:------------------------------:|:------------------:|:--------------------:|
|
68 |
+
| Typhoon-Audio-Preview | 13.26 | 13.34 (partial result) | 5.07 (partial result) | 41.50, 33.48 | 96.20, 96.69 |
|
69 |
+
| DIVA | 69.15 (partial result) | 37.40 | 49.06 | 18.64, 8.16 | 47.50, 35.90 |
|
70 |
+
| Gemini-1.5-Pro | 16.49 | 12.94 | 25.83 | 26.00, 18.26 | 79.66, 77.32 |
|
71 |
+
| Pathumma-llm-audio-1.0.0 | 12.03 | 12.20 | 11.36 | 42.30, 36.88 | 90.30, 92.07 |
|
72 |
|
73 |
## Limitations and Future Work
|
74 |
At present, our model remains in the experimental research phase and is not yet fully suitable for practical applications as an assistant. Future work will focus on upgrading the language model to a newer version [Pathumma-llm-text-1.0.0](https://huggingface.co/nectec/Pathumma-llm-text-1.0.0), and curating more refined and robust datasets to improve performance. Additionally, we aim to address and prioritize the safety and reliability of the model's outputs.
|