README.md

metadata

base_model: genmo/mochi-1-preview
library_name: diffusers
license: apache-2.0
instance_prompt: >-
  A pristine snowglobe featuring a winter scene sits peacefully. The globe
  violently explodes, sending glass, water, and glittering fake snow in all
  directions. The scene is captured with high-speed photography.
widget:
  - text: >-
      A pristine snowglobe featuring a winter scene sits peacefully. The globe
      violently explodes, sending glass, water, and glittering fake snow in all
      directions. The scene is captured with high-speed photography.
    output:
      url: final_video_0.mp4
tags:
  - text-to-video
  - diffusers-training
  - diffusers
  - lora
  - mochi-1-preview
  - mochi-1-preview-diffusers
  - template:sd-lora

Mochi-1 Preview LoRA Finetune

Prompt

A pristine snowglobe featuring a winter scene sits peacefully. The globe violently explodes, sending glass, water, and glittering fake snow in all directions. The scene is captured with high-speed photography.

Model description

This is a lora finetune of the Mochi-1 preview model genmo/mochi-1-preview.

The model was trained using CogVideoX Factory - a repository containing memory-optimized training scripts for the CogVideoX and Mochi family of models using TorchAO and DeepSpeed. The scripts were adopted from CogVideoX Diffusers trainer.

Download model

Download LoRA in the Files & Versions tab.

Usage

Requires the 🧨 Diffusers library installed.

from diffusers import MochiPipeline
from diffusers.utils import export_to_video
import torch 

pipe = MochiPipeline.from_pretrained("genmo/mochi-1-preview")
pipe.load_lora_weights("CHANGE_ME")
pipe.enable_model_cpu_offload()

with torch.autocast("cuda", torch.bfloat16):
    video = pipe(
        prompt="CHANGE_ME",
        guidance_scale=6.0,
        num_inference_steps=64,
        height=480,
        width=848,
        max_sequence_length=256,
        output_type="np"
    ).frames[0]
export_to_video(video)

For more details, including weighting, merging and fusing LoRAs, check the documentation on loading LoRAs in diffusers.

Intended uses & limitations

How to use

# TODO: add an example code snippet for running this diffusion pipeline

Limitations and bias

[TODO: provide examples of latent issues and potential remediations]

Training details

[TODO: describe the data used to train the model]