benjamin/Gemma2-2B-IT-with-Qwen2-Tokenizer

Gemma2-2B Instruction Tuned Model (Transferred to Qwen Tokenizer) Model Card

Gemma2-2B-IT transferred to the Qwen2 Tokenizer. The model approximately preserves performance of the original on most benchmarks, except for some slight degradations.

Model Details

• Base Model: Gemma2-2B
• Tokenization: Transferred to the Qwen Tokenizer
• Training Methodology: Instruction-tuned Gemma2-2B-IT transferred to the Qwen Tokenizer

Benchmark	Gemma2-2B w/ Qwen Tokenizer	Original Gemma2-2B-IT
PiQA	76.9	79.6
HS	70.7	72.5
ARC-C	46.8	50.4
BoolQ	82.8	83.8
MMLU	53.8	56.9
Arith.	83.9	84.8
IFEval	62.5	62.5

Model Details

Details on the training methodology are forthcoming.

Use

import torch
from transformers import pipeline

pipe = pipeline(
    "text-generation",
    model="benjamin/Gemma2-2B-IT-with-Qwen2-Tokenizer",
    model_kwargs={"torch_dtype": torch.bfloat16},
    device="cuda",  # replace with "mps" to run on a Mac device
)

messages = [
    {"role": "user", "content": "Who are you? Please, answer in pirate-speak."},
]

outputs = pipe(messages, max_new_tokens=256)
assistant_response = outputs[0]["generated_text"][-1]["content"].strip()
print(assistant_response)