Spaces:
Runtime error
Runtime error
File size: 1,051 Bytes
12caa44 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
# Tutorial for WebUI 1.6 Version
## We have added one new feature
- We have added an audio-grounding feature that tracks the sound-making object within the video's soundtrack.
## audio-grounding
### 1. Clone the audio-spectrum transformer (AST) model to `./ast_master` and download the pretrained model
```
git clone https://github.com/YuanGongND/ast.git ast_master
wget -O ./ast_master/pretrained_models/audio_mdl.pth https://www.dropbox.com/s/cv4knew8mvbrnvq/audioset_0.4593.pth?dl=1
```
### 2. Switch to Audio-Tab by clicking the `audio-grounding` Tab
<p align="center">
<img src="./img/audio_tab.jpg" height="600">
</p>
### 3. Upload video or use example directly
### 4. Alter the number of labels and the confidence interval of the resulting labels according to your personal preference
step 1: detect the label of the sound making object
step 2: ground the sound making object
<p align="center">
<img src="./img/grounding-tab.jpg" height="400">
</p>
### 5. Get mask of selected object by clicking `Detect` button
### 6. Track in video |