Update README.md

README.md

@@ -11,12 +11,65 @@ tags:
 - trl
 ---
 
-# Uploaded  model
+# Uploaded Model: LORA Adapter
 
 - **Developed by:** Mubin1917
 - **License:** apache-2.0
-- **Finetuned from model :** unsloth/meta-llama-3.1-8b-instruct-bnb-4bit
+- **Finetuned from model:** unsloth/meta-llama-3.1-8b-instruct-bnb-4bit
 
-This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
+This LORA adapter is based on the `unsloth/meta-llama-3.1-8b-instruct-bnb-4bit` model and has been fine-tuned on the [**Lamini_docs QnA**](https://huggingface.co/datasets/lamini/lamini_docs) dataset. The fine-tuning process was optimized using [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library, resulting in a 2x faster training time.
+
+### Training Configuration
+
+The model was trained with the following configuration:
+
+```python
+training_args = TrainingArguments(
+    num_train_epochs=6,
+    per_device_train_batch_size=4,
+    gradient_accumulation_steps=4,
+    per_device_eval_batch_size=4,
+    eval_accumulation_steps=4,
+    warmup_steps=50,
+    learning_rate=2e-4,
+    fp16=not torch.cuda.is_bf16_supported(),
+    bf16=torch.cuda.is_bf16_supported(),
+    eval_steps=25,  # Evaluate every 25 steps
+    logging_steps=25,
+    optim="adamw_8bit",
+    weight_decay=0.01,
+    lr_scheduler_type="linear",
+    seed=3407,
+    output_dir="/kaggle/temp/results",
+    report_to="wandb",
+    save_total_limit=1,  # Save the best one and the last one
+    metric_for_best_model="val_loss",
+    eval_strategy="steps",
+    load_best_model_at_end=True,
+)
+```
+
+### Evaluation Results
+
+- **SacreBLEU Test:**  
+  Score: **73.55**  
+  Detailed Metrics:  
+  - Counts: [20894, 19191, 18504, 18029]  
+  - Totals: [26214, 26074, 25934, 25794]  
+  - Precisions: [79.71%, 73.60%, 71.35%, 69.90%]  
+  - Brevity Penalty: **1.0**  
+  - System Length: **26214**  
+  - Reference Length: **24955**
+
+- **BLEU Test:**  
+  BLEU Score: **0.767**  
+  Detailed Metrics:  
+  - Precisions: [79.71%, 73.73%]  
+  - Brevity Penalty: **1.0**  
+  - Length Ratio: **1.05**  
+  - Translation Length: **26299**  
+  - Reference Length: **24955**
+
+For a detailed comparison between the predicted and actual QnA responses on the test dataset, please visit the [evaluation dataset](https://huggingface.co/datasets/Mubin1917/lamini_docs_evaluation).
 
 [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)