yuringu commited on
Commit
1e75918
ยท
verified ยท
1 Parent(s): f48cbb6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +34 -3
README.md CHANGED
@@ -82,9 +82,40 @@ Llama-3-Luxia-Ko-8B ๋ชจ๋ธ ํ•™์Šต์„ ์œ„ํ•ด 1TB ์ˆ˜์ค€์˜ ํ•œ๊ตญ์–ด ์ฝ”ํผ์Šค์˜
82
  NVIDIA H100 80GB * 8EA์„ ํ™œ์šฉํ•˜์—ฌ ๋ชจ๋ธ ์‚ฌ์ „ํ•™์Šต์„ ์ง„ํ–‰ํ•˜์˜€์Šต๋‹ˆ๋‹ค.
83
 
84
  #### Training Hyperparameters
85
- |Model|Params|Context length|GQA|Learning rate|Batch|Precision|
86
- |---|---|---|---|---|---|---|
87
- |Llama-3-Luxia-Ko|8B|8k|Yes|1e-5|128|bf16|
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
88
 
89
  ### Tokenizer
90
  Llama-3-Tokenizer๋ฅผ ํ•œ๊ตญ์–ด ํŠนํ™”ํ•˜๊ธฐ ์œ„ํ•ด ํ•œ๊ตญ์–ด ํ† ํฐ 17,536๊ฐœ๋ฅผ ์ถ”๊ฐ€ํ•˜๊ณ  ํ™œ์šฉํ•˜์˜€์Šต๋‹ˆ๋‹ค.
 
82
  NVIDIA H100 80GB * 8EA์„ ํ™œ์šฉํ•˜์—ฌ ๋ชจ๋ธ ์‚ฌ์ „ํ•™์Šต์„ ์ง„ํ–‰ํ•˜์˜€์Šต๋‹ˆ๋‹ค.
83
 
84
  #### Training Hyperparameters
85
+ <table>
86
+ <tr>
87
+ <td><strong>Model</strong>
88
+ </td>
89
+ <td><strong>Params</strong>
90
+ </td>
91
+ <td><strong>Context length</strong>
92
+ </td>
93
+ <td><strong>GQA</strong>
94
+ </td>
95
+ <td><strong>Learning rate</strong>
96
+ </td>
97
+ <td><strong>Batch</strong>
98
+ </td>
99
+ <td><strong>Precision</strong>
100
+ </td>
101
+ </tr>
102
+ <tr>
103
+ <td>Llama-3-Luxia-Ko
104
+ </td>
105
+ <td>8B
106
+ </td>
107
+ <td>8k
108
+ </td>
109
+ <td>yes
110
+ </td>
111
+ <td>1e-5
112
+ </td>
113
+ <td>128
114
+ </td>
115
+ <td>bf16
116
+ </td>
117
+ </tr>
118
+ </table>
119
 
120
  ### Tokenizer
121
  Llama-3-Tokenizer๋ฅผ ํ•œ๊ตญ์–ด ํŠนํ™”ํ•˜๊ธฐ ์œ„ํ•ด ํ•œ๊ตญ์–ด ํ† ํฐ 17,536๊ฐœ๋ฅผ ์ถ”๊ฐ€ํ•˜๊ณ  ํ™œ์šฉํ•˜์˜€์Šต๋‹ˆ๋‹ค.