LEESM commited on
Commit
a3cea18
β€’
1 Parent(s): 691c197

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -0
README.md CHANGED
@@ -16,6 +16,7 @@ tags:
16
 
17
  ## 파인 νŠœλ‹ κ°œμš”
18
  - λŒ€μƒ λͺ¨λΈ : meta-llama/Llama-2-7b-hf
 
19
  - νŠœλ‹ 방법 : PEFT(Parameter Efficient Fine-Tuning) - LoRA(Low-Rank Adaptation of Large Language Models)
20
  - κ΄€λ ¨ λ…Όλ¬Έ : https://arxiv.org/abs/2106.09685
21
  - Llama2 λͺ¨λΈμ„ ν•œκ΅­μ–΄ 챗봇 데이터(open korean instructions)의 일뢀λ₯Ό μ΄μš©ν•˜μ—¬ 파인 νŠœλ‹ν•œ λͺ¨λΈ
@@ -30,6 +31,7 @@ tags:
30
  ## 파인 νŠœλ‹ 상세
31
 
32
  - μ•ŒνŒŒκ°’μ„ 16으둜 μ„€μ •
 
33
  ```
34
  peft_config = LoraConfig(
35
  lora_alpha=16,
 
16
 
17
  ## 파인 νŠœλ‹ κ°œμš”
18
  - λŒ€μƒ λͺ¨λΈ : meta-llama/Llama-2-7b-hf
19
+ - 데이터셋 : heegyu/open-korean-instructions (10%)
20
  - νŠœλ‹ 방법 : PEFT(Parameter Efficient Fine-Tuning) - LoRA(Low-Rank Adaptation of Large Language Models)
21
  - κ΄€λ ¨ λ…Όλ¬Έ : https://arxiv.org/abs/2106.09685
22
  - Llama2 λͺ¨λΈμ„ ν•œκ΅­μ–΄ 챗봇 데이터(open korean instructions)의 일뢀λ₯Ό μ΄μš©ν•˜μ—¬ 파인 νŠœλ‹ν•œ λͺ¨λΈ
 
31
  ## 파인 νŠœλ‹ 상세
32
 
33
  - μ•ŒνŒŒκ°’μ„ 16으둜 μ„€μ •
34
+ - r 값을 64둜 μ„€μ • (μ’€ 큰 λ“―...γ… γ… )
35
  ```
36
  peft_config = LoraConfig(
37
  lora_alpha=16,