File size: 528 Bytes
d8e4650 fa818e2 fce8f6a d8e4650 fce8f6a d8e4650 23d9120 d8e4650 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
---
license: apache-2.0
---
## Cat Embeddings
A set of embedding model trained for study embedding quality vs model architecture (width/depth) given a size constraint (12M params).
- **cat-emb-2-128**: 2 layers/hidden size 128/4.4m
- **cat-emb-4-128**: 4 layers/H 128/4.8m
- **cat-emb-6-128**: 6 layers/H 128/5.2m
- **cat-emb-8-128**: 8 layers/H 128/5.6m
- **cat-emb-10-128**: 10 layers/H 128/6.0m
- **cat-emb-12-128**: 12 layers/H 128/6.4m
- **cat-emb-2-256**: 2 layers/H 256/9.7m
- **cat-emb-4-256**: 4 layers/H 256/11.3m
|