Load adapters from the Hub

You can also directly load adapters from the Hub using the commands below:

import torch
from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer

peft_model_id = f"{HUGGING_FACE_USER_NAME}/{model_name}"
config = PeftConfig.from_pretrained(peft_model_id)
model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path, return_dict=True, load_in_8bit=False, device_map='auto')
tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)

# Load the Lora model
model = PeftModel.from_pretrained(model, peft_model_id)

Inference

You can then directly use the trained model or the model that you have loaded from the 🤗 Hub for inference as you would do it usually in transformers.

from IPython.display import display, Markdown

def make_inference(product, description):
  batch = tokenizer(f"### INSTRUCTION\nBelow is a product and description,\
                    please write a marketing email for this product.\
                    \n\n### Product:\n{product}\n### Description:\n{description}\n\n### Marketing Email:\n",
                    return_tensors='pt')

  with torch.cuda.amp.autocast():
    output_tokens = model.generate(**batch, max_new_tokens=200)

  display(Markdown((tokenizer.decode(output_tokens[0], skip_special_tokens=True))))

# Example 
your_product_name_here = "Campfortable chair"
your_product_description_here = "A lightweight camping chair known for its comfort"

make_inference(your_product_name_here, your_product_description_here)

Executing the code above then yields the following

INSTRUCTION
Below is a product and description, please write a marketing email for this product.

Product:
Campfortable chair

Description:
A lightweight camping chair known for its comfort

Marketing Email:
Subject: 🏖️🌞 Get Relaxed incampfortable! ✨

Hey there, Thirsty Traveler! 😎

Imagine being able to lounge in your camping chair all day long, wave goodbye to friends, and return to camp with a refreshed, energy-filled mind and body? 🌴

That's what you’ll get with our revolutionary Campfortable Chair! 🚀

🌱 Say Goodbye to Fears of Inflatable Chairs Our revolutionary design eliminates the worries of bulky, heavy chairs. With just a few simple touches, you’ll feel like you are cradling the world in your arms! 💫

🌺 Flip through Days with Campfortable Chair When you bring Campfortable Chair with you, you’ll have the power to adjust its comfort level based on the demands of your day. Say goodbye to sore backs and headaches, and welcome to relaxed, full-body fun