Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,16 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
---
|
| 4 |
+
|
| 5 |
+
## Cat Embeddings
|
| 6 |
+
|
| 7 |
+
A set of embedding model trained for study embedding quality vs model architecture (width/depth) given a size constraint (12M params).
|
| 8 |
+
|
| 9 |
+
- **cat-emb-2-128**: 2 layers/hidden size 128/4.4m
|
| 10 |
+
- **cat-emb-4-128**: 4 layers/H 128/4.8m
|
| 11 |
+
- **cat-emb-6-128**: 6 layers/H 128/5.2m
|
| 12 |
+
- **cat-emb-8-128**: 8 layers/H 128/5.6m
|
| 13 |
+
- **cat-emb-10-128**: 10 layers/H 128/6.0m
|
| 14 |
+
- **cat-emb-12-128**: 12 layers/H 128/6.4m
|
| 15 |
+
- **cat-emb-2-256**: 2 layers/H 256/9.7m
|
| 16 |
+
- **cat-emb-4-256**: 4 layers/H 256/11.3m
|