Spaces:
Runtime error
Runtime error
l1006986533
commited on
Commit
·
2d4b3fd
1
Parent(s):
f3c59e1
change model location and rewrite readme
Browse files- .gitignore +1 -0
- README.md +10 -89
- app.py +1 -1
.gitignore
CHANGED
@@ -1 +1,2 @@
|
|
1 |
.DS_Store
|
|
|
|
1 |
.DS_Store
|
2 |
+
G_98000.pth
|
README.md
CHANGED
@@ -1,89 +1,10 @@
|
|
1 |
-
|
2 |
-
|
3 |
-
|
4 |
-
|
5 |
-
|
6 |
-
|
7 |
-
|
8 |
-
|
9 |
-
|
10 |
-
|
11 |
-
`chinese_mode = True`
|
12 |
-
in ./text/__init__.py
|
13 |
-
|
14 |
-
## data example
|
15 |
-
copy **24000hz** data-baker datasets to ./test/csmsc
|
16 |
-
|
17 |
-
**For copyright reasons, you can only download it yourself**
|
18 |
-
|
19 |
-
## models
|
20 |
-
you can get example model in ./logs/woman_csmsc/G*.pth
|
21 |
-
|
22 |
-
## Prosody model
|
23 |
-
you can try chinese Prosody model in this repo.
|
24 |
-
|
25 |
-
|
26 |
-
|
27 |
-
======================================================================================================
|
28 |
-
|
29 |
-
# VITS: Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech
|
30 |
-
|
31 |
-
### Jaehyeon Kim, Jungil Kong, and Juhee Son
|
32 |
-
|
33 |
-
In our recent [paper](https://arxiv.org/abs/2106.06103), we propose VITS: Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech.
|
34 |
-
|
35 |
-
Several recent end-to-end text-to-speech (TTS) models enabling single-stage training and parallel sampling have been proposed, but their sample quality does not match that of two-stage TTS systems. In this work, we present a parallel end-to-end TTS method that generates more natural sounding audio than current two-stage models. Our method adopts variational inference augmented with normalizing flows and an adversarial training process, which improves the expressive power of generative modeling. We also propose a stochastic duration predictor to synthesize speech with diverse rhythms from input text. With the uncertainty modeling over latent variables and the stochastic duration predictor, our method expresses the natural one-to-many relationship in which a text input can be spoken in multiple ways with different pitches and rhythms. A subjective human evaluation (mean opinion score, or MOS) on the LJ Speech, a single speaker dataset, shows that our method outperforms the best publicly available TTS systems and achieves a MOS comparable to ground truth.
|
36 |
-
|
37 |
-
Visit our [demo](https://jaywalnut310.github.io/vits-demo/index.html) for audio samples.
|
38 |
-
|
39 |
-
We also provide the [pretrained models](https://drive.google.com/drive/folders/1ksarh-cJf3F5eKJjLVWY0X1j1qsQqiS2?usp=sharing).
|
40 |
-
|
41 |
-
** Update note: Thanks to [Rishikesh (ऋषिकेश)](https://github.com/jaywalnut310/vits/issues/1), our interactive TTS demo is now available on [Colab Notebook](https://colab.research.google.com/drive/1CO61pZizDj7en71NQG_aqqKdGaA_SaBf?usp=sharing).
|
42 |
-
|
43 |
-
<table style="width:100%">
|
44 |
-
<tr>
|
45 |
-
<th>VITS at training</th>
|
46 |
-
<th>VITS at inference</th>
|
47 |
-
</tr>
|
48 |
-
<tr>
|
49 |
-
<td><img src="resources/fig_1a.png" alt="VITS at training" height="400"></td>
|
50 |
-
<td><img src="resources/fig_1b.png" alt="VITS at inference" height="400"></td>
|
51 |
-
</tr>
|
52 |
-
</table>
|
53 |
-
|
54 |
-
|
55 |
-
## Pre-requisites
|
56 |
-
0. Python >= 3.6
|
57 |
-
0. Clone this repository
|
58 |
-
0. Install python requirements. Please refer [requirements.txt](requirements.txt)
|
59 |
-
1. You may need to install espeak first: `apt-get install espeak`
|
60 |
-
0. Download datasets
|
61 |
-
1. Download and extract the LJ Speech dataset, then rename or create a link to the dataset folder: `ln -s /path/to/LJSpeech-1.1/wavs DUMMY1`
|
62 |
-
1. For mult-speaker setting, download and extract the VCTK dataset, and downsample wav files to 22050 Hz. Then rename or create a link to the dataset folder: `ln -s /path/to/VCTK-Corpus/downsampled_wavs DUMMY2`
|
63 |
-
0. Build Monotonic Alignment Search and run preprocessing if you use your own datasets.
|
64 |
-
```sh
|
65 |
-
# Cython-version Monotonoic Alignment Search
|
66 |
-
cd monotonic_align
|
67 |
-
python setup.py build_ext --inplace
|
68 |
-
|
69 |
-
# Preprocessing (g2p) for your own datasets. Preprocessed phonemes for LJ Speech and VCTK have been already provided.
|
70 |
-
# python preprocess.py --text_index 1 --filelists filelists/ljs_audio_text_train_filelist.txt filelists/ljs_audio_text_val_filelist.txt filelists/ljs_audio_text_test_filelist.txt
|
71 |
-
# python preprocess.py --text_index 2 --filelists filelists/vctk_audio_sid_text_train_filelist.txt filelists/vctk_audio_sid_text_val_filelist.txt filelists/vctk_audio_sid_text_test_filelist.txt
|
72 |
-
```
|
73 |
-
|
74 |
-
|
75 |
-
## Training Exmaple
|
76 |
-
```sh
|
77 |
-
# LJ Speech
|
78 |
-
python train.py -c configs/ljs_base.json -m ljs_base
|
79 |
-
|
80 |
-
# VCTK
|
81 |
-
python train_ms.py -c configs/vctk_base.json -m vctk_base
|
82 |
-
|
83 |
-
|
84 |
-
|
85 |
-
```
|
86 |
-
|
87 |
-
|
88 |
-
## Inference Example
|
89 |
-
See [inference.ipynb](inference.ipynb)
|
|
|
1 |
+
---
|
2 |
+
title: Vits Chinese
|
3 |
+
emoji: 🏃
|
4 |
+
colorFrom: red
|
5 |
+
colorTo: yellow
|
6 |
+
sdk: gradio
|
7 |
+
sdk_version: 3.9
|
8 |
+
app_file: app.py
|
9 |
+
pinned: false
|
10 |
+
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
app.py
CHANGED
@@ -28,7 +28,7 @@ net_g = SynthesizerTrn(
|
|
28 |
_ = net_g.eval()
|
29 |
|
30 |
# _ = utils.load_checkpoint("logs/woman_csmsc/G_100000.pth", net_g, None)
|
31 |
-
_ = utils.load_checkpoint("
|
32 |
|
33 |
def vc_fn(input):
|
34 |
stn_tst = get_text(input, hps)
|
|
|
28 |
_ = net_g.eval()
|
29 |
|
30 |
# _ = utils.load_checkpoint("logs/woman_csmsc/G_100000.pth", net_g, None)
|
31 |
+
_ = utils.load_checkpoint("G_98000.pth", net_g, None)
|
32 |
|
33 |
def vc_fn(input):
|
34 |
stn_tst = get_text(input, hps)
|