s-mizuki-nlp
commited on
Liked to instruction datasets, updted metadata.
Browse files
README.md
CHANGED
@@ -9,9 +9,12 @@ license:
|
|
9 |
- gemma
|
10 |
model_type: llama
|
11 |
datasets:
|
12 |
-
- lmsys/lmsys-chat-1m
|
13 |
- tokyotech-llm/lmsys-chat-1m-synth
|
|
|
|
|
|
|
14 |
- argilla/magpie-ultra-v0.1
|
|
|
15 |
---
|
16 |
|
17 |
# Llama 3.1 Swallow - Built with Llama
|
@@ -200,19 +203,17 @@ print(output[0].outputs[0].text)
|
|
200 |
|
201 |
The following datasets were used for the instruction tuning.
|
202 |
|
203 |
-
-
|
204 |
- Multi-turn Japanese instruction dataset synthesized and derived from [lmsys-chat-1m](https://huggingface.co/datasets/lmsys/lmsys-chat-1m) [\[Zhang+, ICLR24\]](https://openreview.net/forum?id=BOfDKxfwt0)).
|
205 |
- First-turn user instructions were translated into Japanese via DeepL (machine translation), and assistant responses were generated using [gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it). The same model, i.e., [gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it) served as a judge for rejection sampling (n=6).
|
206 |
- Second-turn user instructions and responses were synthesized using [gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it). The same model scores the quality of the second-turn response with a range of 1-10. Second-turn responses with scores lower than 9 were rejected, along with their corresponding instructions.
|
207 |
Conversations containing personally identifiable information (PII) and template-based user instructions were removed. Duplicate instructions were removed.
|
208 |
-
|
209 |
-
- `filtered-magpie-ultra-ja`
|
210 |
- A Japanese variant of the `filtered-magpie-ultra-en` dataset, translated into Japanese by [gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it).
|
211 |
-
-
|
212 |
-
- A Japanese synthetic
|
213 |
- The conversations were heuristically filtered for quality and length. Then, [gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it) was applied to score the quality of each of the conversation with a range of 1-10. Conversations with scores <= 7 were rejected.
|
214 |
|
215 |
-
|
216 |
## Risks and Limitations
|
217 |
|
218 |
The models released here are still in the early stages of our research and development and have not been tuned to ensure outputs align with human intent and safety considerations.
|
|
|
9 |
- gemma
|
10 |
model_type: llama
|
11 |
datasets:
|
|
|
12 |
- tokyotech-llm/lmsys-chat-1m-synth
|
13 |
+
- tokyotech-llm/swallow-magpie-ultra-v0.1
|
14 |
+
- tokyotech-llm/swallow-gemma-magpie-v0.1
|
15 |
+
- lmsys/lmsys-chat-1m
|
16 |
- argilla/magpie-ultra-v0.1
|
17 |
+
|
18 |
---
|
19 |
|
20 |
# Llama 3.1 Swallow - Built with Llama
|
|
|
203 |
|
204 |
The following datasets were used for the instruction tuning.
|
205 |
|
206 |
+
- [Gemma-2-LMSYS-Chat-1M-Synth](https://huggingface.co/datasets/tokyotech-llm/lmsys-chat-1m-synth)
|
207 |
- Multi-turn Japanese instruction dataset synthesized and derived from [lmsys-chat-1m](https://huggingface.co/datasets/lmsys/lmsys-chat-1m) [\[Zhang+, ICLR24\]](https://openreview.net/forum?id=BOfDKxfwt0)).
|
208 |
- First-turn user instructions were translated into Japanese via DeepL (machine translation), and assistant responses were generated using [gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it). The same model, i.e., [gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it) served as a judge for rejection sampling (n=6).
|
209 |
- Second-turn user instructions and responses were synthesized using [gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it). The same model scores the quality of the second-turn response with a range of 1-10. Second-turn responses with scores lower than 9 were rejected, along with their corresponding instructions.
|
210 |
Conversations containing personally identifiable information (PII) and template-based user instructions were removed. Duplicate instructions were removed.
|
211 |
+
- [Swallow-Magpie-Ultra-v0.1](https://huggingface.co/datasets/tokyotech-llm/swallow-magpie-ultra-v0.1)
|
|
|
212 |
- A Japanese variant of the `filtered-magpie-ultra-en` dataset, translated into Japanese by [gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it).
|
213 |
+
- [Swallow-Gemma-Magpie-v0.1](https://huggingface.co/datasets/tokyotech-llm/swallow-gemma-magpie-v0.1)
|
214 |
+
- A Japanese synthetic instruction tuning dataset from scratch, generated by [gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it). User instructions were created with prompts specific to each topic, and assistant responses were generated for these instructions.
|
215 |
- The conversations were heuristically filtered for quality and length. Then, [gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it) was applied to score the quality of each of the conversation with a range of 1-10. Conversations with scores <= 7 were rejected.
|
216 |
|
|
|
217 |
## Risks and Limitations
|
218 |
|
219 |
The models released here are still in the early stages of our research and development and have not been tuned to ensure outputs align with human intent and safety considerations.
|