Update README.md
Browse files
README.md
CHANGED
|
@@ -39,7 +39,7 @@ To start using this model, run the following:
|
|
| 39 |
from transformers import AutoProcessor, PaliGemmaForConditionalGeneration
|
| 40 |
|
| 41 |
model = PaliGemmaForConditionalGeneration.from_pretrained("agentsea/paligemma-3b-ft-waveui-896").eval()
|
| 42 |
-
processor = AutoProcessor.from_pretrained("
|
| 43 |
```
|
| 44 |
|
| 45 |
## Data
|
|
@@ -49,4 +49,13 @@ We used the [WaveUI](https://huggingface.co/datasets/agentsea/wave-ui) dataset f
|
|
| 49 |
|
| 50 |
## Evaluation
|
| 51 |
|
| 52 |
-
We
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
from transformers import AutoProcessor, PaliGemmaForConditionalGeneration
|
| 40 |
|
| 41 |
model = PaliGemmaForConditionalGeneration.from_pretrained("agentsea/paligemma-3b-ft-waveui-896").eval()
|
| 42 |
+
processor = AutoProcessor.from_pretrained("google/paligemma-3b-pt-896")
|
| 43 |
```
|
| 44 |
|
| 45 |
## Data
|
|
|
|
| 49 |
|
| 50 |
## Evaluation
|
| 51 |
|
| 52 |
+
We calculated the mean IoU over 1024 examples of the test set using 3 different closed-source models: Gemini Pro 1.5, Claude Sonnet 3.5 and GPT 4o. We also ran this same calculation using the PaliGemma WaveUI fine-tunes. We obtained the following values:
|
| 53 |
+
|
| 54 |
+
- Gemini 1.5: 0.12
|
| 55 |
+
- Claude: 0.05
|
| 56 |
+
- GPT: 0.05
|
| 57 |
+
- PaliGemma Widgetcap+WaveUI 448: 0.40
|
| 58 |
+
- **PaliGemma WaveUI 896: 0.49**
|
| 59 |
+
|
| 60 |
+
|
| 61 |
+
|