s-mizuki-nlp commited on
Commit
982c4f1
·
verified ·
1 Parent(s): ffb8ea1

Liked to instruction datasets, updted metadata.

Browse files
Files changed (1) hide show
  1. README.md +8 -7
README.md CHANGED
@@ -9,9 +9,12 @@ license:
9
  - gemma
10
  model_type: llama
11
  datasets:
12
- - lmsys/lmsys-chat-1m
13
  - tokyotech-llm/lmsys-chat-1m-synth
 
 
 
14
  - argilla/magpie-ultra-v0.1
 
15
  ---
16
 
17
  # Llama 3.1 Swallow - Built with Llama
@@ -200,19 +203,17 @@ print(output[0].outputs[0].text)
200
 
201
  The following datasets were used for the instruction tuning.
202
 
203
- - `lmsys-chat-1m-synth-gemma2-2turns-ja-wo-pii-and-template-instructions`
204
  - Multi-turn Japanese instruction dataset synthesized and derived from [lmsys-chat-1m](https://huggingface.co/datasets/lmsys/lmsys-chat-1m) [\[Zhang+, ICLR24\]](https://openreview.net/forum?id=BOfDKxfwt0)).
205
  - First-turn user instructions were translated into Japanese via DeepL (machine translation), and assistant responses were generated using [gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it). The same model, i.e., [gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it) served as a judge for rejection sampling (n=6).
206
  - Second-turn user instructions and responses were synthesized using [gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it). The same model scores the quality of the second-turn response with a range of 1-10. Second-turn responses with scores lower than 9 were rejected, along with their corresponding instructions.
207
  Conversations containing personally identifiable information (PII) and template-based user instructions were removed. Duplicate instructions were removed.
208
- - The dataset will be available at [tokyotech-llm/lmsys-chat-1m-synth](https://huggingface.co/datasets/tokyotech-llm/lmsys-chat-1m-synth).
209
- - `filtered-magpie-ultra-ja`
210
  - A Japanese variant of the `filtered-magpie-ultra-en` dataset, translated into Japanese by [gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it).
211
- - `gemma-magpie`
212
- - A Japanese synthetic Q&A dataset from scratch, generated by [gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it). User instructions were created with prompts specific to each topic, and assistant responses were generated for these instructions.
213
  - The conversations were heuristically filtered for quality and length. Then, [gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it) was applied to score the quality of each of the conversation with a range of 1-10. Conversations with scores <= 7 were rejected.
214
 
215
-
216
  ## Risks and Limitations
217
 
218
  The models released here are still in the early stages of our research and development and have not been tuned to ensure outputs align with human intent and safety considerations.
 
9
  - gemma
10
  model_type: llama
11
  datasets:
 
12
  - tokyotech-llm/lmsys-chat-1m-synth
13
+ - tokyotech-llm/swallow-magpie-ultra-v0.1
14
+ - tokyotech-llm/swallow-gemma-magpie-v0.1
15
+ - lmsys/lmsys-chat-1m
16
  - argilla/magpie-ultra-v0.1
17
+
18
  ---
19
 
20
  # Llama 3.1 Swallow - Built with Llama
 
203
 
204
  The following datasets were used for the instruction tuning.
205
 
206
+ - [Gemma-2-LMSYS-Chat-1M-Synth](https://huggingface.co/datasets/tokyotech-llm/lmsys-chat-1m-synth)
207
  - Multi-turn Japanese instruction dataset synthesized and derived from [lmsys-chat-1m](https://huggingface.co/datasets/lmsys/lmsys-chat-1m) [\[Zhang+, ICLR24\]](https://openreview.net/forum?id=BOfDKxfwt0)).
208
  - First-turn user instructions were translated into Japanese via DeepL (machine translation), and assistant responses were generated using [gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it). The same model, i.e., [gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it) served as a judge for rejection sampling (n=6).
209
  - Second-turn user instructions and responses were synthesized using [gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it). The same model scores the quality of the second-turn response with a range of 1-10. Second-turn responses with scores lower than 9 were rejected, along with their corresponding instructions.
210
  Conversations containing personally identifiable information (PII) and template-based user instructions were removed. Duplicate instructions were removed.
211
+ - [Swallow-Magpie-Ultra-v0.1](https://huggingface.co/datasets/tokyotech-llm/swallow-magpie-ultra-v0.1)
 
212
  - A Japanese variant of the `filtered-magpie-ultra-en` dataset, translated into Japanese by [gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it).
213
+ - [Swallow-Gemma-Magpie-v0.1](https://huggingface.co/datasets/tokyotech-llm/swallow-gemma-magpie-v0.1)
214
+ - A Japanese synthetic instruction tuning dataset from scratch, generated by [gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it). User instructions were created with prompts specific to each topic, and assistant responses were generated for these instructions.
215
  - The conversations were heuristically filtered for quality and length. Then, [gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it) was applied to score the quality of each of the conversation with a range of 1-10. Conversations with scores <= 7 were rejected.
216
 
 
217
  ## Risks and Limitations
218
 
219
  The models released here are still in the early stages of our research and development and have not been tuned to ensure outputs align with human intent and safety considerations.