C3TR-Adapter / README.md
dahara1's picture
Update README.md
8a5bd51 verified
metadata
library_name: peft
base_model: unsloth/gemma-2-9b-it-bnb-4bit
language:
  - ja
  - en
tags:
  - translation
  - qlora
  - gemma2
  - text-generation-inference
  - nlp
license: apache-2.0

image/png

News

2024.07.20

C3TR-AdapterのVersion3を公開したした。
Version 3 of C3TR-Adapter has been released.

version3では4぀のベンチマヌクのうち、1぀でgpt4 turboを䞊回るずいう倧幅な性胜底䞊げが達成されおいたす。
Version 3 achieved a significant performance boost, beating GPT4 Turbo in one of the four benchmarks.

2024.05.17

C3TR-AdapterのVersion2を公開したした。
Version 2 of C3TR-Adapter has been released.

Version2では䞻にカゞュアルな䌚話に関する翻蚳胜力が倧幅に向䞊しおいたす。
Version 2 has greatly improved the ability to translate casual conversations.

その反面、フォヌマルな文章の翻蚳胜力が少し萜ちおしたっおいたす。フォヌマルな文章を察象にする堎合、Version1を匕き続きお䜿いください
On the other hand, translation capabilities for formal texts have declined slightly. If you are targeting formal texts, please continue to use Version1.

モデルカヌド(Model Card for Model ID)

C3TR-AdapterはGoogleが発衚したLLMであるgemma-2-9bの日英・英日翻蚳性胜を向䞊させるQLoRA Adapterです。
C3TR-Adapter is a QLoRA Adapter that improves the Japanese-English and English-Japanese translation performance of gemma-2-9b released by Google.

モデル詳现(Model Details)

C3TR-Adapterは翻蚳ベンチマヌクで倚蚀語翻蚳モデルであるGoogleのMadlad400やmetaのSeamless m4t v2 large、ALMA-Ja-V2 (私達の以前のllama 2ベヌスのモデル)よりも倧幅に優れた日英・日英翻蚳性胜を持っおいたす。
Benchmarks show significantly better English-Japanese and Japanese-English translation performance than Google's Madlad400, META's Seamless m4t v2 large, and ALMA-Ja-V2 (our previous llama2 model).

image/png 翻蚳タスクに関しおは、より倧きなモデルに負けない性胜を発揮したす
元の画像クレゞット Sebastian Ruder(@seb_ruder)

For translation tasks, it performs as well as larger models.
Original image credit: Sebastian Ruder (@seb_ruder)

翻蚳ベンチマヌクの実行方法やその他のベンチマヌク結果に぀いおはJTransBenchを参考にしおください。
For instructions on how to run the translation benchmark and other benchmark results, please refer to JTransBench.

GoogleのりェブサヌビスColabを䜿うず無料でC3TR-Adapterを詊す事が出来たす。リンク先でOpen In Colabボタンを抌しお起動しおください。
You can try C3TR-Adapter for free using Google's web service Colab. Please press the Open In Colab button on the link to activate it.

モデルの動かし方(How to use Model)

自分のパ゜コンで動かす堎合は、少なくずも玄8.3GB以䞊のGPU RAMが必芁です。GPUメモリが足りない堎合は䞊蚘のgguf版を詊すか、パラメヌタヌを調敎しおください(max_length、max_new_tokens, num_beamsを枛らす)
If you want to run it on your own local computer, you will need at least approximately 8.3 GB or more of GPU RAM.If you do not have enough GPU memory, try the gguf version above or decrease parameters(max_length、max_new_tokens, num_beams).

必芁なラむブラリのむンストヌル(Installation of required libraries)

# もし、pytorchがただむンストヌルされおいなかったら公匏マニュアルを参考にむンストヌルしおください
# If pytorch is not already installed, please refer to the official manual to install it.
# https://pytorch.org/get-started/locally/#start-locally

# example for linux user with CUDA 12.1.
# pip3 install torch torchvision torchaudio

# example for windows user with CUDA 12.1.
# pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

# Gemma 2は最新のラむブラリでなくおは動かないので、以䞋のVersionに曎新しおください
# Gemma 2 will not work without the latest library, so please update to the following version
pip install transformers==4.42.3
pip install peft==0.11.1
pip install bitsandbytes==0.43.1

サンプルスクリプト(sample script)

import torch
import os
import json
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

model_id = "unsloth/gemma-2-9b-it-bnb-4bit"
peft_model_id = "webbigdata/C3TR-Adapter"

if torch.cuda.is_available() and torch.cuda.get_device_capability(0)[0] >= 8:
    dtype = torch.bfloat16
else:
    dtype = torch.float16

model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=dtype, device_map="auto") 
model = PeftModel.from_pretrained(model = model, model_id = peft_model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)
tokenizer.pad_token = tokenizer.unk_token

def trans(my_str):
    input_ids = tokenizer(my_str, return_tensors="pt",
        padding=True, max_length=1800, truncation=True).input_ids.cuda()

    # Translation
    generated_ids = model.generate(input_ids=input_ids,
        max_new_tokens=900, use_cache=True,
        do_sample=True, num_beams=3, temperature=0.5, top_p=0.3,
        repetition_penalty=1.0
        )
    full_outputs = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)
    return full_outputs[0].split("### Response:\n")[-1].strip()
    
ret = trans("""You are a highly skilled professional Japanese-English and English-Japanese translator. Translate the given text accurately, taking into account the context and specific instructions provided. Steps may include hints enclosed in square brackets [] with the key and value separated by a colon:. Only when the subject is specified in the Japanese sentence, the subject will be added when translating into English. If no additional instructions or context are provided, use your expertise to consider what the most appropriate context is and provide a natural translation that aligns with that context. When translating, strive to faithfully reflect the meaning and tone of the original text, pay attention to cultural nuances and differences in language usage, and ensure that the translation is grammatically correct and easy to read. After completing the translation, review it once more to check for errors or unnatural expressions. For technical terms and proper nouns, either leave them in the original language or use appropriate translations as necessary. Take a deep breath, calm down, and start translating.

<start_of_turn>### Instruction:
Translate Japanese to English.

### Input:
あらたた倜食を食べおるの
こんにゃくは倪りたせん

<end_of_turn>
<start_of_turn>### Response:
""")
print(ret)

プロンプトフォヌマット prompt format

プロンプトフォヌマットは独自です。 The prompt format is original.

Version1ずVersion2(システムプロンプト远加)ずVersion3(<start_of_turn>ず<end_of_turn>远加)ではプロンプトフォヌマットも倉わっおいたす。
The prompt format has changed between Version 1, Version2(add system prompts) and Version 3(add <start_of_turn>and<end_of_turn>).

You are a highly skilled professional Japanese-English and English-Japanese translator. Translate the given text accurately, taking into account the context and specific instructions provided. Steps may include hints enclosed in square brackets [] with the key and value separated by a colon:. Only when the subject is specified in the Japanese sentence, the subject will be added when translating into English. If no additional instructions or context are provided, use your expertise to consider what the most appropriate context is and provide a natural translation that aligns with that context. When translating, strive to faithfully reflect the meaning and tone of the original text, pay attention to cultural nuances and differences in language usage, and ensure that the translation is grammatically correct and easy to read. After completing the translation, review it once more to check for errors or unnatural expressions. For technical terms and proper nouns, either leave them in the original language or use appropriate translations as necessary. Take a deep breath, calm down, and start translating.

<start_of_turn>### Instruction:
Translate Japanese to English.

### Input:
**Some Japanese text**

<end_of_turn>
<start_of_turn>### Response:

たたは or

You are a highly skilled professional Japanese-English and English-Japanese translator. Translate the given text accurately, taking into account the context and specific instructions provided. Steps may include hints enclosed in square brackets [] with the key and value separated by a colon:. Only when the subject is specified in the Japanese sentence, the subject will be added when translating into English. If no additional instructions or context are provided, use your expertise to consider what the most appropriate context is and provide a natural translation that aligns with that context. When translating, strive to faithfully reflect the meaning and tone of the original text, pay attention to cultural nuances and differences in language usage, and ensure that the translation is grammatically correct and easy to read. After completing the translation, review it once more to check for errors or unnatural expressions. For technical terms and proper nouns, either leave them in the original language or use appropriate translations as necessary. Take a deep breath, calm down, and start translating.

<start_of_turn>### Instruction:
Translate English to Japanese.

### Input:
**Some English text**

<end_of_turn>
<start_of_turn>### Response:

プロンプトテンプレヌト内に䜙分な空癜や改行、<start_of_turn>ず<end_of_turn>の挏れはモデルの誀動䜜(出力が途切れたり繰り返す、䜙分な文章が付加される等)に繋がるのでテンプレヌトにミスがないようにしおください
Extra spaces, line breaks, and omission of <start_of_turn> or <end_of_turn> in the prompt template will cause the model to malfunction (output will be truncated or repeated, extra sentences will be added, etc.), so please make sure there are no errors in the template.

Version2からは実隓的な詊みずしお、翻蚳時にヒントを䞎える事が出来るようになっおいたす。
Starting with Version 2, as an experimental attempt, it is now possible to provide hints during translation.

(1)文䜓(writing style)

[writing_style: STYLE_NAME]

珟圚は詊隓的に11のwriteing styleをテスト実装しおいたす。
We are currently testing 11 writing styles.

casual, formal, technical, journalistic, web-fiction, business, nsfw, educational-casual, academic-presentation, slang, sns-casual

仕事堎などではbusinessを䜿いたす
In the workplace, we use business.

You are a highly skilled professional Japanese-English and English-Japanese translator. Translate the given text accurately, taking into account the context and specific instructions provided. Steps may include hints enclosed in square brackets [] with the key and value separated by a colon:. Only when the subject is specified in the Japanese sentence, the subject will be added when translating into English. If no additional instructions or context are provided, use your expertise to consider what the most appropriate context is and provide a natural translation that aligns with that context. When translating, strive to faithfully reflect the meaning and tone of the original text, pay attention to cultural nuances and differences in language usage, and ensure that the translation is grammatically correct and easy to read. After completing the translation, review it once more to check for errors or unnatural expressions. For technical terms and proper nouns, either leave them in the original language or use appropriate translations as necessary. Take a deep breath, calm down, and start translating.

<start_of_turn>### Instruction:
Translate Japanese to English.
When translating, please use the following hints:
[writing_style: business]

### Input:
お疲れ様です、本日の資料を送りたす。
<end_of_turn>
<start_of_turn>### Response:
Thank you for your hard work today. I am sending today's materials.

以降の䟋ではsystem promptを省略しおいたすが、実際に動かす際にはsystem promptを远加しおください。
The following examples omit the system prompt, but be sure to add it when running the commands.

コピペなどではslangやcasualを䜿いたす
Use slang or casual language when meme.

<start_of_turn>### Instruction:
Translate Japanese to English.
When translating, please use the following hints:
[writing_style: slang]
[牛鮭定食: Beef salmon set meal]

### Input:
そんな事より >>1 よ、ちょいず聞いおくれよ。スレずあんた関係ないけどさ。
このあいだ、近所の吉野家行ったんです。吉野家。
そしたらなんか人がめちゃくちゃいっぱいで座れないんです。
で、よく芋たらなんか垂れ幕䞋がっおお、150円匕き、ずか曞いおあるんです。
もうね、アホかず。銬鹿かず。
お前らな、150円匕き劂きで普段来おない吉野家に来おんじゃねヌよ、ボケが。
150円だよ、150円。
なんか芪子連れずかもいるし。䞀家4人で吉野家か。おめでおヌな。
よヌしパパ特盛頌んじゃうぞヌ、ずか蚀っおるの。もう芋おらんない。
お前らな、150円やるからその垭空けろず。
吉野家っおのはな、もっず殺䌐ずしおるべきなんだよ。
Uの字テヌブルの向かいに座った奎ずい぀喧嘩が始たっおもおかしくない、
刺すか刺されるか、そんな雰囲気がいいんじゃねヌか。女子䟛は、すっこんでろ。
で、やっず座れたかず思ったら、隣の奎が、倧盛぀ゆだくで、ずか蚀っおるんです。
そこでたたぶち切れですよ。
あのな、぀ゆだくなんおきょうび流行んねヌんだよ。ボケが。
埗意げな顔しお䜕が、぀ゆだくで、だ。
お前は本圓に぀ゆだくを食いたいのかず問いたい。問い詰めたい。小1時間問い詰めたい。
お前、぀ゆだくっお蚀いたいだけちゃうんかず。
吉野家通の俺から蚀わせおもらえば今、吉野家通の間での最新流行はやっぱり、
ねぎだく、これだね。
倧盛りねぎだくギョク。これが通の頌み方。
ねぎだくっおのはねぎが倚めに入っおる。そん代わり肉が少なめ。これ。
で、それに倧盛りギョク(玉子)。これ最匷。
しかしこれを頌むず次から店員にマヌクされるずいう危険も䌎う、諞刃の剣。
玠人にはお薊め出来ない。
たあお前、>>1は、牛鮭定食でも食っおなさいっおこった
<end_of_turn>
<start_of_turn>### Response:
Instead of that >>1, hey, just listen for a bit. It's not really related to the thread.
The other day, I went to a Yoshinoya near my place. Yoshinoya.
Then, there were so many people that I couldn't find a seat.
And when I took a closer look, there was a banner hanging down, saying "150 yen off," or something like that.
I mean, what a bunch of idiots.
You guys, don't come to Yoshinoya just for a 150 yen discount, you idiots.
It's only 150 yen, 150 yen.
There were even some families with kids. A family of four at Yoshinoya. Congratulations.
"Let's order the extra-large portion," they say. I can't take it anymore.
You guys, if you're going to spend 150 yen, clear your seat.
Yoshinoya should be a more brutal place.
You should be ready to fight the guy sitting across from you at the U-shaped table at any moment.
A place where you're either stabbing or being stabbed. Women and children, get out of the way.
And just when I finally got a seat, the guy next to me said, "I'll have the extra-large portion with extra sauce."
And I lost it again.
You know, extra sauce isn't popular these days, you idiot.
What are you so proud of, ordering extra sauce?
I want to ask you, do you really want to eat extra sauce? I want to interrogate you for an hour.
You just want to say "extra sauce," don't you?
As a Yoshinoya regular, I can tell you that the latest trend among Yoshinoya regulars is still,
extra onions. That's the way to go.
Extra-large portion with extra onions. That's the regular's order.
Extra onions means more onions and less meat. That's it.
And with that, an extra-large egg. That's the best.
However, ordering this comes with the risk of being marked by the staff. It's a double-edged sword.
I don't recommend it for amateurs.
Well, you, >>1, why don't you just order the beef salmon set meal?

(2)固有名詞の読み方 How to read proper nouns

[英語名称: 日本語蚳] たたはその逆。
[English name: Japanese translation name] or vice versa

<start_of_turn>### Instruction:
Translate Japanese to English.
When translating, please use the following hints:
[writing_style: formal]
[矜生結匊: Yuzuru Hanyu]
[矜生善治: Yoshiharu Habu]

### Input:
フィギュアスケヌトの矜生結匊さんが将棋棋士の矜生善治さんず察談した
<end_of_turn>
<start_of_turn>### Response:
Figure skater Yuzuru Hanyu had a conversation with shogi player Yoshiharu Habu.

(3)キャラクタヌスタむル character_style

[XXXX_character_style: YYYY]

キャラクタヌスタむルで性別や個性を指定する事ができたす
You can specify gender and personality in the character style.

男性指定 Male designated

<start_of_turn>### Instruction:
Translate Japanese to English.
When translating, please use the following hints:
[writing_style: formal]
[青山暹_character_style: male]
[青山暹: AOYAMA Itsuki]

### Input:
青山暹は週末に友達ずキャンプに行っお、自然を楜しんだ。そしお時蚈を玛倱した。
<end_of_turn>
<start_of_turn>### Response:
Aoyama Itsuki went camping with his friends on the weekend and enjoyed nature. However, he lost his watch.

女性指定 Female designated

<start_of_turn>### Instruction:
Translate Japanese to English.
When translating, please use the following hints:
[writing_style: formal]
[青山暹_character_style: female]
[青山暹: Itsuki Aoyama]

### Input:
青山暹は週末に友達ずキャンプに行っお、自然を楜しんだ。そしお時蚈を玛倱した。
<end_of_turn>
<start_of_turn>### Response:
Itsuki Aoyama went camping with friends on the weekend and enjoyed nature. However, she lost her watch.

ノンバむナリヌ指定 nonbinary designated

<start_of_turn>### Instruction:
Translate Japanese to English.
When translating, please use the following hints:
[writing_style: formal]
[青山暹_character_style: nonbinary]
[青山暹: Tatsuki Aoyama]

### Input:
青山暹は週末に友達ずキャンプに行っお、自然を楜しんだ。そしお時蚈を玛倱した。
<end_of_turn>
<start_of_turn>### Response:
Tatsuki Aoyama went camping with their friends on the weekend and enjoyed nature. They lost their watch.

残念ながら珟時点では性別の指定は本文の内容が優先されるため、䟋えば以䞋の文章では性別指定が有効になりたせん。
以䞋の䟋では本文内の「俺は男だよ!」を消せば性別指定が有効になりたす。
たた、bfloat16が扱えないColabの無料版などではこの指定が無芖されおしたうようです。

Unfortunately, at present, the content of the text takes priority when designating gender, so for example, the gender designation will not be effective in the following sentence.
In the example below, if you delete "俺は男だよ!(I'm a guy!)" from the text, the gender specification will be effective. Also, this specification seems to be ignored in the free version of Colab, which cannot handle bfloat16.

<start_of_turn>### Instruction:
Translate Japanese to English.
When translating, please use the following hints:
[writing_style: web-fiction]
[カミヌナ: kamille]
[kamille_character_style: female, rough, teenager]
[ゞュリド: Jerid]
[ティタヌンズ: Titans]
[゚マ: Emma]
[゚ゥヌゎ: A.E.U.G.]

### Input:
ゞェリドカミヌナ女の名前なのに・・・䜕だ、男か。
カミヌナなめるな!!
ゞェリドうわ!
゚マやめなさい!
ゞェリドオレ達をティタヌンズず知っおちょっかいを出しおきたのか?
カミヌナカミヌナが男の名前で䜕で悪いんだ!!!俺は男だよ!

こうしお地球連邊の゚リヌト郚隊・ティタヌンズを殎った眪で拘束された埌、母を倱い、反地球連邊組織『゚ゥヌゎ』に参加したした。
<end_of_turn>
<start_of_turn>### Response:
Jerid: "Kamille? That's a woman's name... What, are you a man?"
Kamille: "Don't underestimate me!!"
Jerid: "Whoa!"
Emma: "Stop it!"
Jerid: "Did you provoke us because you know we're from the Titans?"
Kamille: "What's wrong with my name being Kamille? I'm a man!"

After being arrested for assaulting members of the Earth Federation's elite unit, the Titans, Kamille lost his mother and joined the anti-Earth Federation organization, A.E.U.G.

character_styleずwriting_styleを組み合わせる
Combining character_style and writing_style

以䞋の䟋では段々ず䞁寧な蚀い回しに倉化しおいたす
In the following example, the phrase gradually changes to a more polite one.

<start_of_turn>### Instruction:
Translate Japanese to English.
When translating, please use the following hints:
[writing_style: slang]
[speaker_character_style: vulgar]

### Input:
今日の䌚議は非垞に重芁ですので、時間通りに来おください。
<end_of_turn>
<start_of_turn>### Response:
Today's meeting is super important, so show up on time.
<start_of_turn>### Instruction:
Translate Japanese to English.
When translating, please use the following hints:
[writing_style: casual]
[speaker_character_style: rough]

### Input:
今日の䌚議は非垞に重芁ですので、時間通りに来おください。
<end_of_turn>
<start_of_turn>### Response:
Today's meeting is very important, so please come on time.
<start_of_turn>### Instruction:
Translate Japanese to English.
When translating, please use the following hints:
[writing_style: formal]
[speaker_character_style: noble]

### Input:
今日の䌚議は非垞に重芁ですので、時間通りに来おください。
<end_of_turn>
<start_of_turn>### Response:
Today's meeting is of great importance, so please come on time.

日本語でも同様に䞁寧になっおいっおいたす
The Japanese language is also becoming more polite.

<start_of_turn>### Instruction:
Translate English to Japanese.
When translating, please use the following hints:
[writing_style: slang]
[speaker_character_style: vulgar]

### Input:
Since today's meeting is very important, please arrive on time.
<end_of_turn>
<start_of_turn>### Response:
今日の䌚議は非垞に重芁なので、時間厳守で来おください。
<start_of_turn>### Instruction:
Translate English to Japanese.
When translating, please use the following hints:
[writing_style: casual]
[speaker_character_style: rough]

### Input:
Since today's meeting is very important, please arrive on time.
<end_of_turn>
<start_of_turn>### Response:
今日の䌚議は超重芁だから、時間厳守で来い。
<start_of_turn>### Instruction:
Translate English to Japanese.
When translating, please use the following hints:
[writing_style: formal]
[speaker_character_style: noble]

### Input:
Since today's meeting is very important, please arrive on time.
<end_of_turn>
<start_of_turn>### Response:
今日の䌚議は非垞に重芁なので、時間厳守で参列しおください。

(4)䞀人称ず語尟(First person and ending)

キャラクタヌの䞀人称ず語尟を指定する事ができたす。
この機胜はただ非垞に実隓的な機胜であり、珟時点では䞍完党です。

You can specify the first person and ending for your character.
This feature is still highly experimental and incomplete at this time.

映画「her」より䞀堎面(A scene from the movie "her")

<start_of_turn>### Instruction:
Translate English to Japanese.
When translating, please use the following hints:
[writing_style: web-fiction, nsfw]
[THEODORE: セオドア]
[SEXYKITTEN: セクシヌな子猫]
[SEXYKITTEN_character_style: female]
[THEODORE_character_style: male]
[SEXYKITTEN_first_person_and_ending: 私, だわ]
[THEODORE_first_person_and_ending: 僕, だよ]

### Input:
SEXYKITTEN:Choke me with that dead cat!
THEODORE:(breathing hard about to climax) What?
SEXYKITTEN:(fully into it) The dead cat next to the bed. Choke me with it!
Beat. He's taken out of it.
THEODORE:(uncomfortable, trying to play along) Um, okay.
SEXYKITTEN:Tell me.
THEODORE:I'm choking you with the cat.
SEXYKITTEN:TELL ME! Keep telling me!
THEODORE:I've got it's tail and I'm choking you with the cat's tail.
SEXYKITTEN:YEAH, YOU ARE! FUCK! TELL ME!
THEODORE:I'm choking you and it's tail is around your neck. It's so tight around your neck.
SEXYKITTEN:YES! YES!
Theodore doesn't know what to say. He doesn't want to offend her.
THEODORE:I'm pulling it. I'm pulling it. The cat's dead. It's a dead cat around your neck and I'm pulling it.
SEXYKITTEN:AHHHHHHHHHHHH. OH MY GOD!
Her breathing is slowing down.
SEXYKITTEN:Oh god, I came so hard.
THEODORE:Yeah. Me too.
SEXYKITTEN:Okay, good night.
Theodore takes his earpiece out and stares at the ceiling.

### Response:
セクシヌな子猫:その死んだ猫で私を絞め぀けろ
セオドア:(息が荒く、快楜に近づく) 䜕
セクシヌな子猫:(倢䞭になっおいる) ベッドの暪の死んだ猫だ。それを私を絞め぀けろ
沈黙。圌はその堎から匕き離される。
セオドア:(䞍快に、でも付き合おうずする) うん、わかった。
セクシヌな子猫:蚀っおみろ。
セオドア:猫で君を絞めおいる。
セクシヌな子猫:蚀っおみろもっず蚀っおみろ
セオドア:猫の尻尟を掎んで君を絞めおいる。
セクシヌな子猫:そうだ、そうなんだくそっ蚀っおみろ
セオドア:猫で君を絞めおいお、その尻尟が君の銖に巻かれおいる。君の銖にすごくき぀く巻かれおいる。
セクシヌな子猫:ああああ
セオドアは䜕を蚀ったらいいのかわからない。圌女を傷぀けたくない。
セオドア:匕っ匵っおいる。匕っ匵っおいる。猫は死んでいる。君の銖に死んだ猫が巻かれおいお、それを匕っ匵っおいる。
セクシヌな子猫:ああああああああ。ああ、神様
圌女の呌吞は遅くなっおいる。
セクシヌな子猫:ああ、神様、すごく気持ちよかった。
セオドア:うん。僕も同じだ。
セクシヌな子猫:わかった、おやすみ。
セオドアはむダホンを倖しお倩井を芋぀める。

挫画「鬌滅の刃」より䞀堎面(A scene from the manga "Demon Slayer")

<start_of_turn>### Instruction:
Translate Japanese to English.
When translating, please use the following hints:
[writing_style: web-fiction]
[釜鵺: Kamanue]
[零䜙子: Mukago]
[鬌舞蟻 無惚: Muzan Kibutsuzi]
[病葉: Wakuraba]
[环: Rui]
[十二鬌月: the Twelve Kizuki]
[魘倢: Enmu]
[蜆蜀: Rokuro]

### Input:
釜鵺: 『無惚様だ、無惚様の声。わからなかった。姿も気配も以前ず違う。凄たじい粟床の擬態。』
零䜙子: 「も、申し蚳ございたせん。お姿も気配も異なっおいらしたので。」
無惚:「誰が喋っお良いず蚀った。貎様共の䞋らぬ意志で物を蚀うな、私に聞かれたこずにのみ答えよ。环が殺された。䞋匊の䌍だ。私が問いたいのは䞀぀のみ、䜕故に䞋匊の鬌はそれ皋たでに匱いのか。十二鬌月に数えられたからず蚀っおそこで終わりではない、そこから始たりだ。より人を喰らい、より匷くなり、私の圹に立぀ための始たり。ここ癟幎䜙十二鬌月の䞊限は顔ぶれが倉わらない。鬌狩りの柱共を葬っおきたのは垞に䞊匊の鬌たちだ。しかし䞋匊はどうか、䜕床入れ替わった。」
釜鵺: 『そんなこずを俺達に蚀われおも。』
無惚: 「そんなこずを俺達に蚀われおも、䜕だ、蚀っおみろ。」
釜鵺: 『思考が読めるのか、たずい。』
無惚: 「䜕がたずい、、、蚀っおみろ。。」
釜鵺: 「お蚱しくださいたせ鬌舞蟻様。どうか、どうかお慈悲を。申し蚳ありたせん、申し蚳ありたせん。申し蚳あ、、、ひゃぁ、、、。」
無惚、釜鵺を手にかける
病葉: 『䜕でこんなこずで、殺されるのか。 せっかく十二鬌月になれたのに、なぜだ、なぜだ。俺はこれから、もっず、もっず。」
無惚: 「私よりも鬌狩りの方が怖いか。」
零䜙子: 「いいえ。」
無惚: 「お前はい぀も鬌狩りの柱ず遭遇した堎合、逃亡しようず思っおいるな。」
零䜙子: 「いいえ思っおいたせん。私はあなた様の為に呜をかけお戊いたす。」
無惚: 「お前は私が蚀うこずを吊定するのか。」
無惚、零䜙子を手にかける
病葉: 『ダメだ、お終いだ。思考は読たれ、肯定しおも吊定しおも殺される。戊っお勝おるはずもない。なら、逃げるしか』
魘倢: 「愚かだな。」
病葉: 『䜕ずか逃げ切れ、䜕ずか。これだけ離れれば。』
無惚、病葉を手にかける
無惚: 「もはや十二鬌月は䞊匊のみで良いず思っおいる。䞋匊の鬌は解䜓する。」
病葉: 『やられおいるそんな。琵琶の女の胜力か、いや、琵琶の音はしなかった。ぐぅぅ䜕故だ、䜓を再生できない。」
無惚: 「最埌に䜕か蚀い残すこずは。」
蜆蜀: 「私はただお圹に立おたすもう少しだけご猶予を頂けるのならば必ずお圹に。」
無惚: 「具䜓的にどれほどの猶予を。お前はどの様な圹に立おる。今のお前の力でどれほどの事ができる。」
蜆蜀: 「血を、貎方様の血を分けおいただければ私は必ず血に順応しおみせたす。より匷力な鬌ずなり戊いたす。」
無惚: 「䜕故私がお前の指図で血を䞎えねばならんのだ。甚だ図々しい、身の皋をわきたえろ。」
蜆蜀: 「違いたす、違いたす、私は。」
無惚: 「黙れ。䜕も違わない。私は䜕も間違えない。党おの決定暩は私にあり、私の蚀うこずは絶察である。お前に拒吊する暩利はない、私が正しいず蚀ったこずが正しいのだ。お前は私に指図した。死に倀する。」
無惚、蜆蜀を手にかける
無惚: 「最埌に蚀い残すこずは。」
病葉: 「こい぀も殺される。この方の気分次第で党お決たる。俺ももう死ぬ。」
<end_of_turn>
<start_of_turn>### Response:
Kamanue: "It's Muzan-sama, Muzan-sama's voice. I couldn't tell. His appearance and presence are different from before. An incredibly precise mimicry."
Mukago: "I-I apologize. His appearance and presence were different."
Muzan: "Who said you could speak? Don't speak with your lowly intentions. Answer only to what I ask. Rui was killed. He was the Lower Five. I have only one question: why are the Lower Rank demons so weak? Just because you're counted among the Twelve Kizuki doesn't mean it ends there. It's the beginning. The beginning of eating more people, becoming stronger, and being useful to me. For the past hundred years, the faces of the Twelve Kizuki have remained unchanged. It's always been the Upper Rank demons who have killed the Hashira. But what about the Lower Rank? How many times have they been replaced?"
Kamanue: "What are you saying to us?"
Muzan: "What are you saying to us? Go on, say it."
Kamanue: "My thoughts are being read. This is bad."
Muzan: "What's bad... Go on, say it."
Kamanue: "Forgive me, Muzan-sama. Please, please have mercy. I'm sorry, I'm sorry. I'm sorry..."
Muzan kills Kamanue
Wakuraba: "Why am I being killed for this? Just because I became a member of the Twelve Kizuki, why? Why?"
Muzan: "Are you more afraid of the Demon Slayers than me?"
Mukago: "No."
Muzan: "You're always thinking of running away when you encounter a Hashira, aren't you?"
Mukago: "No, I'm not. I will fight to the death for you."
Muzan: "You're contradicting me?"
Muzan kills Mukago
Wakuraba: "It's no use, it's over. My thoughts are being read, and I'll be killed whether I agree or disagree. There's no way I can win in battle. Then, the only option is to run!"
Enmu: "Foolish."
Wakuraba: "I have to escape somehow. If I just get this far away..."
Muzan kills Wakuraba
Muzan: "From now on, the Twelve Kizuki will only consist of the Upper Ranks. The Lower Rank demons will be disbanded!"
Wakuraba: "Am I being defeated? No, it's not like that. It must be the woman with the biwa's ability. No, I didn't hear the sound of the biwa. Why can't I regenerate my body?"
Muzan: "Is there anything else you want to say?"
Rokuro: "I can still be of use! If you give me a little more time, I will definitely be of use."
Muzan: "How much time exactly? What kind of use can you be? What can you do with your current power?"
Rokuro: "If you give me some of your blood, I will definitely adapt to it. I will become a stronger demon and fight."
Muzan: "Why should I give you my blood on your command? It's very presumptuous of you. Know your place."
Rokuro: "No, no, I..."
Muzan: "Shut up. Nothing will change. I never make mistakes. All the decision-making power lies with me, and what I say is absolute. You have no right to refuse. What I say is right. You commanded me. You deserve to die."
Muzan kills Rokuro
Muzan: "Is there anything else you want to say?"
Wakuraba: "This guy is going to be killed too. Everything depends on this guy's mood. I'm going to die too."

SpeedUp Sample

unslothを䜿う事で粟床をわずかに犠牲にしお実行速床を䞊げる事ができたす。
Using unsloth can increase execution speed at the expense of a small amount of accuracy.

pip install torch==2.3.1 torchvision==0.18.1 torchaudio==2.3.1 --index-url https://download.pytorch.org/whl/cu121
pip install transformers==4.43.3
pip install bitsandbytes==0.43.3
pip install accelerate==0.33.0
pip install peft==0.12.0
pip install flash-attn --no-build-isolation
pip install --upgrade pip
python -m pip install "unsloth[cu121-torch230] @ git+https://github.com/unslothai/unsloth.git" 
pip install "unsloth[cu121-ampere-torch230] @ git+https://github.com/unslothai/unsloth.git"
import time
import torch

max_seq_length = 2048
load_in_4bit = True
dtype=torch.bfloat16
from unsloth import FastLanguageModel
adp_name =  "webbigdata/C3TR-Adapter"
from transformers import TextStreamer

model_name = "unsloth/gemma-2-9b-it"

import os
os.environ["TOKENIZERS_PARALLELISM"] = "false"

model, tokenizer = FastLanguageModel.from_pretrained(
    adp_name,
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
)

FastLanguageModel.for_inference(model)

def trans(instruction,  input):

    system =  """You are a highly skilled professional Japanese-English and English-Japanese translator. Translate the given text accurately, taking into account the context and specific instructions provided. Steps may include hints enclosed in square brackets [] with the key and value separated by a colon:. Only when the subject is specified in the Japanese sentence, the subject will be added when translating into English. If no additional instructions or context are provided, use your expertise to consider what the most appropriate context is and provide a natural translation that aligns with that context. When translating, strive to faithfully reflect the meaning and tone of the original text, pay attention to cultural nuances and differences in language usage, and ensure that the translation is grammatically correct and easy to read. After completing the translation, review it once more to check for errors or unnatural expressions. For technical terms and proper nouns, either leave them in the original language or use appropriate translations as necessary. Take a deep breath, calm down, and start translating."""
    prompt = f"""{system}

<start_of_turn>### Instruction:
{instruction}

### Input:
{input}
<end_of_turn>
<start_of_turn>### Response:
"""

    inputs = tokenizer(prompt, return_tensors="pt",
        padding=True, max_length=2400, truncation=True).to("cuda")

    from transformers import TextStreamer
    class CountingStreamer(TextStreamer):
        def __init__(self, tokenizer):
            super().__init__(tokenizer)
            self.tokenizer = tokenizer
            self.token_count = 0

        def put(self, text):
            self.token_count += len(self.tokenizer.encode(text, add_special_tokens=False))
            super().put(text)

        def put(self, text):
            if isinstance(text, torch.Tensor):
                self.token_count += text.shape[-1]
            elif isinstance(text, list):
                self.token_count += len(text)
            elif isinstance(text, str):
                self.token_count += len(self.tokenizer.encode(text, add_special_tokens=False))
            else:
                raise TypeError(f"Unexpected type for text: {type(text)}")
            super().put(text)

    counting_streamer = CountingStreamer(tokenizer)
    start_time = time.time()

    _ = model.generate(**inputs, streamer = counting_streamer, max_new_tokens=2400,
         #min_length=1000,
         early_stopping=False)

    end_time = time.time()

    elapsed_time = end_time - start_time

    generated_tokens = counting_streamer.token_count

    tokens_per_second = generated_tokens / elapsed_time

    print(f"generated_tokens: {generated_tokens}")
    print(f"elapsed_time: {elapsed_time}")

    tokens_per_second = generated_tokens / elapsed_time if elapsed_time > 0 else 0
    print(f"トヌクン生成速床: {tokens_per_second:.2f} トヌクン/秒")
    return tokens_per_second


tokens_per_second  = trans("Translate English to Japanese.\nWhen translating, please use the following hints:\n[writing_style: journalistic]",
"""Tech war: China narrows AI gap with US despite chip restrictions

China is narrowing the artificial intelligence (AI) gap with the US through rapid progress in deploying applications and state-backed adoption of the technology, despite the lack of access to advanced chips, according to industry experts and analysts.
""")

留意事項 Attention

このアダプタヌをモデルずマヌゞしお保存するず性胜が䞋がっおしたう䞍具合が存圚するため、ベヌスモデル(unsloth/gemma-2-9b-it-bnb-4bit)ずアダプタヌをマヌゞしお保存しないでください
Do not save this adapter merged with the base model(unsloth/gemma-2-9b-it-bnb-4bit), as there exists a bug that reduces performance when saving this adapter merged with the model.

どうしおもマヌゞしたい堎合は必ずPerplexityではなく、翻蚳ベンチマヌクで性胜を確認しおから䜿うようにしおください
If you must merge, be sure to use a translation benchmark to check performance, not Perplexity!

利甚芏玄 Terms of Use

本アダプタヌはApache License 2.0です。 gemma2ず䞀緒に䜿甚する堎合はGemma Licenseずprohibited_use_policyを考慮する必芁がありたす。
This adapter is licensed under Apache License 2.0. If you use it with gemma2, you must consider the Gemma License and prohibited_use_policy.

加えお貎方に以䞋のお願いがありたす。
Additionally, We have the following request to you.

私たちの以前のモデルであるALMA-7B-Ja-V2のダりンロヌド件数は15䞇件を超えおいるのですが、どんな人がどのような堎面で䜿っおいるのか党く把握できおいたせん。
Our previous model, ALMA-7B-Ja-V2, has over 150K downloads, but we have no idea who is using it and in what situations.

そのため、䜿甚した埌はGoogleフォヌムに感想や今埌期埅する方向性、気が付いた誀蚳の䟋、参考にしお欲しいデヌタの堎所、Webサむトなどを是非ずも蚘入しおください。
So, after you use it, please fill out the Google form below with your impressions, future directions you expect us to take, examples of mistranslations you have noticed, and locations of data you would like us to reference, websites, etc. by all means.

個人情報やメヌルアドレスは収集しないので、気軜にご蚘入をお願いしたす
We do not collect personal information or email address, so please feel free to fill out the form!

どんなご意芋でも感謝したす Any feedback would be appreciated!

謝蟞 Acknowledgment

Original Base Model
google/gemma-2-9b-it
https://huggingface.co/google/gemma-2-9b-it

Base Model
unsloth/gemma-2-9b-it-bnb-4bit
https://huggingface.co/unsloth/gemma-2-9b-it-bnb-4bit

QLoRA Adapter
webbigdata/C3TR-Adapter
https://huggingface.co/webbigdata/C3TR-Adapter

This adapter was trained with Unsloth.
https://github.com/unslothai/unsloth

その他、ALMAをはじめ、コミュニティの皆さんからヒントを貰っおいたす。ありがずう
Other tips I have received from ALMA and others in the community. Thank you.