Linaqruf/sketch-style-xl-lora

Sketch Style XL LoRA

---

Overview

Sketch Style XL LoRA stands as an innovative, high-resolution LoRA adapter tailored for Animagine XL 2.0, focusing on sketch-style and monochromatic image generation. This model is adept at transforming text prompts into detailed sketches and grayscale images, capturing the essence of traditional media in digital form.

Notable supported tags for this model include monochrome, greyscale, sketch, traditional media among others, facilitating a wide range of artistic expressions.

---

Model Details

• Developed by: Linaqruf
• Model type: LoRA adapter for Stable Diffusion XL
• Model Description: Sketch Style XL LoRA is a specialized adapter designed to enhance Animagine XL 2.0's capability in creating sketch-like, monochromatic images. This model excels in interpreting text prompts to generate images that resemble hand-drawn sketches or traditional grayscale artworks. It is a versatile tool for artists and creators who seek to explore the interplay between text and sketch-style imagery.
• License: CreativeML Open RAIL++-M License
• Finetuned from model: Animagine XL 2.0

---

Usage

The Sketch Style XL LoRA model effectively interprets and generates images based on the following keywords:

• Monochrome: Generates images in shades of a single color, typically black and white.
• Greyscale: Produces images using varying shades of grey, simulating the look of classic black and white photography or artwork.
• Sketch: Creates images that resemble hand-drawn sketches, emphasizing outlines and shading without color.
• Traditional Media: Imitates the style of artwork created with traditional physical media like pencils, charcoal, or ink.

---

🧨 Diffusers Installation

Ensure the installation of the latest diffusers library, along with other essential packages:

pip install diffusers --upgrade
pip install transformers accelerate safetensors

The following Python script demonstrates how to utilize the LoRA with Animagine XL 2.0. The default scheduler is EulerAncestralDiscreteScheduler, but it can be explicitly defined for clarity.

import torch
from diffusers import (
    StableDiffusionXLPipeline, 
    EulerAncestralDiscreteScheduler,
    AutoencoderKL
)

# Initialize LoRA model and weights
lora_model_id = "Linaqruf/sketch-style-xl-lora"
lora_filename = "sketch-style-xl.safetensors"

# Load VAE component
vae = AutoencoderKL.from_pretrained(
    "madebyollin/sdxl-vae-fp16-fix", 
    torch_dtype=torch.float16
)

# Configure the pipeline
pipe = StableDiffusionXLPipeline.from_pretrained(
    "Linaqruf/animagine-xl-2.0", 
    vae=vae,
    torch_dtype=torch.float16, 
    use_safetensors=True, 
    variant="fp16"
)
pipe.scheduler = EulerAncestralDiscreteScheduler.from_config(pipe.scheduler.config)
pipe.to('cuda')

# Load and fuse LoRA weights
pipe.load_lora_weights(lora_model_id, weight_name=lora_filename)
pipe.fuse_lora(lora_scale=0.6)

# Define prompts and generate image
prompt = "face focus, cute, masterpiece, best quality, 1girl, sketch, monochrome, greyscale, green hair, sweater, looking at viewer, upper body, beanie, outdoors, night, turtleneck"
negative_prompt = "lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry"

image = pipe(
    prompt, 
    negative_prompt=negative_prompt, 
    width=1024,
    height=1024,
    guidance_scale=12,
    num_inference_steps=50
).images[0]

# Unfuse LoRA before saving the image
pipe.unfuse_lora()
image.save("anime_girl.png")