Spaces:
Sleeping
Sleeping
added app.py and readme
Browse files- README.md +36 -5
- app.py +53 -0
- requirements.txt +1 -0
README.md
CHANGED
@@ -1,12 +1,43 @@
|
|
1 |
---
|
2 |
-
title:
|
3 |
-
emoji:
|
4 |
colorFrom: blue
|
5 |
-
colorTo:
|
6 |
sdk: gradio
|
7 |
-
sdk_version: 3.
|
8 |
app_file: app.py
|
9 |
pinned: false
|
|
|
|
|
|
|
|
|
|
|
10 |
---
|
11 |
|
12 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
+
title: WER
|
3 |
+
emoji: 🖩
|
4 |
colorFrom: blue
|
5 |
+
colorTo: red
|
6 |
sdk: gradio
|
7 |
+
sdk_version: 3.19.1
|
8 |
app_file: app.py
|
9 |
pinned: false
|
10 |
+
tags:
|
11 |
+
- evaluate
|
12 |
+
- wer
|
13 |
+
- neuralspace
|
14 |
+
- STT
|
15 |
---
|
16 |
|
17 |
+
## About this Demo
|
18 |
+
|
19 |
+
This demo was built as a part of [NeuralSpace](https://neuralspace.ai/)'s [VoiceAI](https://voice.neuralspace.ai/) blog on [Word Error Rate 101: Your Guide to STT Vendor Evaluation](voice.neuralspace.ai).
|
20 |
+
|
21 |
+
## What is WER?
|
22 |
+
|
23 |
+
WER or Word Error Rate is a metric used primarily in the field of speech recognition to measure the performance of an automatic speech recognition (ASR) system. WER calculates the minimum number of operations (substitutions, deletions, and insertions) required to change the system's transcription (prediction) into the reference transcription (truth), divided by the number of words in the reference.
|
24 |
+
|
25 |
+
```WER = (Substitutions + Insertions + Deletions)/Total number of words in truth```
|
26 |
+
|
27 |
+
WER can range from 0 to infinite. The closer the WER is to 0, the better. WER is often also represented as a percentage. It is usually calculated by just multiplying 100 to it. For example, a WER of 0.15 might also be represented as 15%.
|
28 |
+
|
29 |
+
WER is important because it provides:
|
30 |
+
* **Performance Measure**: It gives an objective measure of how well an ASR system is transcribing speech into text.
|
31 |
+
* **Comparison**: It allows for comparison between different ASR systems or versions of a system.
|
32 |
+
|
33 |
+
|
34 |
+
## References
|
35 |
+
|
36 |
+
* Python package to calculate WER: [jiwer](https://jitsi.github.io/jiwer/)
|
37 |
+
|
38 |
+
|
39 |
+
## FAQ
|
40 |
+
|
41 |
+
Have any questions or comments? Reach out to NeuralSpace at [email protected]. Interested to try out [NeuralSpace VoiceAI](https://voice.neuralspace.ai/) for your enterprise? Book time with our expert [here]().
|
42 |
+
|
43 |
+
Space authored by : [Aditya Dalmia](https://www.linkedin.com/in/dalmeow/)
|
app.py
ADDED
@@ -0,0 +1,53 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
import gradio as gr
|
2 |
+
import pandas as pd
|
3 |
+
from jiwer import wer
|
4 |
+
import re
|
5 |
+
import os
|
6 |
+
|
7 |
+
REGEX_YAML_BLOCK = re.compile(r"---[\n\r]+([\S\s]*?)[\n\r]+---[\n\r]")
|
8 |
+
|
9 |
+
|
10 |
+
def parse_readme(filepath):
|
11 |
+
"""Parses a repositories README and removes"""
|
12 |
+
if not os.path.exists(filepath):
|
13 |
+
return "No README.md found."
|
14 |
+
with open(filepath, "r") as f:
|
15 |
+
text = f.read()
|
16 |
+
match = REGEX_YAML_BLOCK.search(text)
|
17 |
+
if match:
|
18 |
+
text = text[match.end() :]
|
19 |
+
return text
|
20 |
+
|
21 |
+
|
22 |
+
def compute(input):
|
23 |
+
preds = input['prediction'].tolist()
|
24 |
+
truths = input['truth'].tolist()
|
25 |
+
print(truths, preds, type(truths))
|
26 |
+
err = wer(truths, preds)
|
27 |
+
print(err)
|
28 |
+
return err
|
29 |
+
|
30 |
+
|
31 |
+
description = """
|
32 |
+
To calculate WER:
|
33 |
+
|
34 |
+
* Type the `prediction` and the `truth` in the respective columns in the below calculator.
|
35 |
+
* You can insert multiple predictions and truths by clicking on the `New row` button.
|
36 |
+
* To calculate the WER after inserting all the texts, click on `Submit`.
|
37 |
+
"""
|
38 |
+
|
39 |
+
demo = gr.Interface(
|
40 |
+
fn=compute,
|
41 |
+
inputs=gr.components.Dataframe(
|
42 |
+
headers=["prediction", "truth"],
|
43 |
+
col_count=2,
|
44 |
+
row_count=1,
|
45 |
+
label="Input"
|
46 |
+
),
|
47 |
+
outputs=gr.components.Textbox(label="WER"),
|
48 |
+
description=description,
|
49 |
+
title="WER Calculator",
|
50 |
+
article=parse_readme("README.md")
|
51 |
+
)
|
52 |
+
|
53 |
+
demo.launch()
|
requirements.txt
ADDED
@@ -0,0 +1 @@
|
|
|
|
|
1 |
+
jiwer
|