metadata

title: Fumo Classifier
emoji: 🧸
colorFrom: blue
colorTo: pink
sdk: gradio
sdk_version: 5.9.1
app_file: app.py
pinned: false

Fumo Image Classifier

Overview

A LoRA for google/siglip-so400m-patch14-384 trained to identify Fumo dolls (Gift) in images. This model achieves 99.28% accuracy in fumo detection with a bias towards false negatives.

Model Details

Base Model: google/siglip-so400m-patch14-384
Training Method: LoRA fine-tuning (r=16, alpha=32)
Target Modules: Attention (q_proj, v_proj) and FFN (fc1, fc2)
Input: Images (384x384 pixels)
Output: Binary classification (fumo/not fumo)
Accuracy: 99.28%

Training Specifications

Training Device: NVIDIA RTX 3050 Ti
Training Time: ~2 hours
Epochs: 15
Batch Size: 10
Optimizer: AdamW (lr=1.3e-5, weight_decay=0.001)
Data Augmentation:

Random vertical flip (p=0.5)
Color jitter (hue=0.5)

Usage

from transformers import AutoProcessor, AutoModel
import torch
from peft import PeftModel, PeftConfig

# Load the base model and processor
processor = AutoProcessor.from_pretrained("google/siglip-so400m-patch14-384")
model = AutoModel.from_pretrained("google/siglip-so400m-patch14-384")

# Load LoRA weights
model = PeftModel.from_pretrained(model, "houraisan/fumo-classifier")

def classify_image(image_path):
    # Preprocess image
    inputs = processor(images=image_path, return_tensors="pt")

    # Get prediction
    with torch.no_grad():
        outputs = model(**inputs)

    # Process outputs
    # Classification logic here

Model Architecture

LoraConfig(
    r=16,
    lora_alpha=32,
    target_modules=["q_proj", "v_proj", "fc1", "fc2"],
    lora_dropout=0.1,
    bias="none",
)

Citations

@misc{houraisan2024fumo,
  author = {Houraisan, Kaguya},
  title = {Fumo Image Classifier},
  year = {2024},
  publisher = {Hugging Face Spaces},
  journal = {Imperial Lunar Institute Research Repository}
}

License

MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Acknowledgments

Google for the base SigLIP model
Hugging Face for hosting infrastructure