File size: 979 Bytes
d8e4650 fa818e2 fce8f6a d8e4650 fce8f6a d8e4650 23d9120 d8e4650 862d0ff |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
---
license: apache-2.0
---
## Cat Embeddings
A set of embedding model trained for study embedding quality vs model architecture (width/depth) given a size constraint (12M params).
- **cat-emb-2-128**: 2 layers/hidden size 128/4.4m
- **cat-emb-4-128**: 4 layers/H 128/4.8m
- **cat-emb-8-128**: 8 layers/H 128/5.6m
- **cat-emb-12-128**: 12 layers/H 128/6.4m
- **cat-emb-2-256**: 2 layers/H 256/9.7m
- **cat-emb-4-256**: 4 layers/H 256/11.3m
### Perf
| MRL dim\Task | BIOSSES | SICK-R | STS12 | STS13 | STS14 | STS15 | STS16 | STSB | SummEval |
|-----|---------|--------|-------|-------|-------|-------|-------|------|----------|
| 128 | 0.7107 | 0.7126 | 0.6815| 0.7343| 0.7038| 0.8163| 0.7495| 0.7652| 0.2958 |
| 64 | 0.713 | 0.7123 | 0.6829| 0.7348| 0.7008| 0.813 | 0.7475| 0.7609| 0.2861 |
| 32 | 0.6714 | 0.7094 | 0.6847| 0.7345| 0.6911| 0.7989| 0.7385| 0.7545| 0.3106 |
| 16 | 0.6637 | 0.697 | 0.669 | 0.7096| 0.6665| 0.7589| 0.7183| 0.7307| 0.3164 |
|