metadata

language:
  - en
license: mit
tags:
  - nlp
  - code
  - mlx
license_link: https://huggingface.co/microsoft/phi-2/resolve/main/LICENSE
datasets:
  - teknium/openhermes
pipeline_tag: text-generation

marcel/phi-2-openhermes-30k

This model was converted to MLX format from microsoft/phi-2. Refer to the original model card for more details on the model.

Use with mlx

pip install mlx
git clone https://github.com/ml-explore/mlx-examples.git
cd mlx-examples/llms/hf_llm
python generate.py --model marcel/phi-2-openhermes-30k --prompt "My name is"

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "marcel/phi-2-openhermes-30k",
    low_cpu_mem_usage=True,
    device_map="auto",
    trust_remote_code=True,
    torch_dtype=torch.float16,
)
tokenizer = AutoTokenizer.from_pretrained("phi-2-openhermes-30k")

input_text = "### Human: Give me a good recipe for a chinese dish\n\n### Assistant:"

outputs = model.generate(
    tokenizer(input_text, return_tensors="pt").to(model.device)['input_ids'],
    max_length=1024,
    temperature=0.7,
    top_p=0.9,
    do_sample=True,
    pad_token_id=tokenizer.pad_token_id,
    eos_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))