Update README.md

README.md

@@ -1,3 +1,10 @@
+---
+license: mit
+datasets:
+- yahma/alpaca-cleaned
+language:
+- en
+---
 This repo contains a low-rank adapter for LLaMA-13b fit on the Stanford Alpaca dataset.
 
 This version of the weights was trained on dual RTX3090 with the following hyperparameters:
@@ -21,4 +28,8 @@ OMP_NUM_THREADS=4 WORLD_SIZE=2 CUDA_VISIBLE_DEVICES=0,1 torchrun --nproc_per_nod
     --val_set_size=0 \
     --micro_batch_size=32
 
-Instructions for running it can be found at https://github.com/tloen/alpaca-lora.
+LR warmup was tuned to fit the first epoch.
+
+Instructions for running it can be found at https://github.com/tloen/alpaca-lora.
+
+![10 epochs](alpaca13b.png)