metadata
license: apache-2.0
metrics:
- cer
Belle-whisper-large-v2-zh
Fine tune whisper-large-v2 to improve Chinese speech recognition, Belle-whisper-large-v2-zh has 30-70% relative improvements on Chinese ASR benchmark(AISHELL1, AISHELL2, WENETSPEECH, HKUST).
Usage
from transformers import pipeline
transcriber = pipeline(
"automatic-speech-recognition",
model="BELLE-2/Belle-whisper-large-v2-zh"
)
transcriber.model.config.forced_decoder_ids = (
transcriber.tokenizer.get_decoder_prompt_ids(
language="zh",
task="transcribe"
)
)
transcription = transcriber("my_audio.wav")
Fine-tuning
| Model | (Re)Sample Rate | Train Datasets | Fine-tuning (full or peft) |
|---|---|---|---|
| Belle-whisper-large-v2-zh | 16KHz | AISHELL-1 AISHELL-2 WenetSpeech HKUST | full fine-tuning |
CER
| Model | Language Tag | aishell_1_test | aishell_2_test | wenetspeech_net | wenetspeech_meeting | HKUST_dev |
|---|---|---|---|---|---|---|
| whisper-large-v2 | Chinese | 0.08818 | 0.06183 | 0.12343 | 0.26413 | 0.31917 |
| Belle-whisper-large-v2-zh | Chinese | 0.02549 | 0.03746 | 0.08503 | 0.14598 | 0.16289 |