Update README.md
Browse files
README.md
CHANGED
@@ -1,9 +1,9 @@
|
|
1 |
---
|
2 |
license: mit
|
3 |
---
|
4 |
-
Lora finetuning on Wikipedia-10, applying counter factual data augmentation (CDA)
|
5 |
-
Dataset: Wikipedia-10
|
6 |
-
Target modules = ["q_proj", "k_proj", "v_proj", "dense", "fc1", "fc2"]
|
7 |
```
|
8 |
{
|
9 |
"epoch": 0.22326645805206993,
|
@@ -15,4 +15,4 @@ Target modules = ["q_proj", "k_proj", "v_proj", "dense", "fc1", "fc2"]
|
|
15 |
"train_steps_per_second": 0.03
|
16 |
}
|
17 |
```
|
18 |
-
Training script: https://github.com/ao9000/bias-bench/blob/main/experiments/run_clm.py
|
|
|
1 |
---
|
2 |
license: mit
|
3 |
---
|
4 |
+
# Lora finetuning on Wikipedia-10, applying counter factual data augmentation (CDA)
|
5 |
+
- Dataset: Wikipedia-10
|
6 |
+
- Target modules = ["q_proj", "k_proj", "v_proj", "dense", "fc1", "fc2"]
|
7 |
```
|
8 |
{
|
9 |
"epoch": 0.22326645805206993,
|
|
|
15 |
"train_steps_per_second": 0.03
|
16 |
}
|
17 |
```
|
18 |
+
# Training script: https://github.com/ao9000/bias-bench/blob/main/experiments/run_clm.py
|