Update README.md

README.md

@@ -10,19 +10,19 @@ tags:
 - LoRA
 ---
 
-# Llama-2-7b-hf 모델을 이용한 LoRA 파인 튜닝 (서울사이버대학교 TREX-Lab)
+# LoRA fine tuning using the Llama-2-7b-hf model (Seoul Cyber University TREX-Lab)
 
 <!-- Provide a quick summary of what the model is/does. -->
 
-## 파인 튜닝 개요
-  - 기본 모델 : meta-llama/Llama-2-7b-hf
-  - 데이터셋 : heegyu/open-korean-instructions (10%)
-  - 튜닝 방법
+## Summary
+  - Base Model : meta-llama/Llama-2-7b-hf
+  - Dataset : heegyu/open-korean-instructions (10%)
+  - Tuning Method
     - PEFT(Parameter Efficient Fine-Tuning)
     - LoRA(Low-Rank Adaptation of Large Language Models)
-  - 관련 논문 : https://arxiv.org/abs/2106.09685
-  - Llama2 모델을 한국어 챗봇 데이터(open korean instructions)의 임의 10%로 파인 튜닝
-  - 대규모 언어 모델을 A30 GPU*1 에서 파인 튜닝이 가능한지 테스트 (성공)
+  - Related Articles : https://arxiv.org/abs/2106.09685
+  - Fine-tuning the Llama2 model with a random 10% of Korean chatbot data (open Korean instructions)
+  - Test whether fine tuning of a large language model is possible on A30 GPU*1 (successful)
 
 <!-- Provide a longer summary of what this model is. -->
 
@@ -30,10 +30,10 @@ tags:
 - **Language(s) (NLP):** [Korean]
 - **Finetuned from model :** [meta-llama/Llama-2-7b-hf]
 
-## 파인 튜닝 상세
+## Fine Tuning Detail
 
-- 알파값을 16으로 설정
-- r 값을 64로 설정 (좀 큰 듯...ㅠㅠ)
+- alpha value 16
+- r value 64 (it seems a bit big...@@)
 ```
 peft_config = LoraConfig(
     lora_alpha=16,
@@ -44,7 +44,7 @@ peft_config = LoraConfig(
 )
 ```
 
-- Mixed precision 설정 : 4bit 정밀도 (bnb_4bit_use_double_quant로 중복 양자 설정)
+- Mixed precision : 4bit (bnb_4bit_use_double_quant)
 ```
 bnb_config = BitsAndBytesConfig(
     load_in_4bit=True,
@@ -54,30 +54,7 @@ bnb_config = BitsAndBytesConfig(
 )
 ```
 
-- 3 epoch 수행
-```
-training_args = TrainingArguments(
-    output_dir=fine_tuning_model_name,
-    num_train_epochs=3,
-    per_device_train_batch_size=4,
-    gradient_accumulation_steps=2,
-    gradient_checkpointing=True,
-    optim='paged_adamw_32bit',
-    logging_steps=5,
-    save_strategy='epoch',
-    learning_rate=2e-4,
-    weight_decay=0.001,
-    max_grad_norm=0.3,
-    warmup_ratio=0.03,
-    group_by_length=False,
-    lr_scheduler_type='cosine',
-    disable_tqdm=True,
-    report_to='wandb',
-    seed=42
-)
-```
-
-- 기본 Trainer대신 SFT트레이너 사용 (https://huggingface.co/docs/trl/sft_trainer)
+- Use SFT trainer (https://huggingface.co/docs/trl/sft_trainer)
 ```
 trainer = SFTTrainer(
     model=peft_model,
@@ -90,10 +67,10 @@ trainer = SFTTrainer(
 )
 ```
 
-### 학습 결과
+### Train Result
 
 ```
-소요시간 : executed in 2d 0h 17m
+time taken : executed in 2d 0h 17m
 ```
 
 ```