---
license: mit
datasets:
- Qingyun/lmmrotate-sft-data
language:
- en
base_model:
- microsoft/Florence-2-large
pipeline_tag: image-text-to-text
tags:
- aerial
- geoscience
- remotesensing
---
LMMRotate 🎮: A Simple Aerial Detection Baseline of Multimodal Language Models
Qingyun Li
Yushi Chen
Xinya Shu
Dong Chen
Xin He
Yi Yu
Xue Yang
If you find our work helpful, please consider giving us a ⭐!
- ArXiv Paper: https://arxiv.org/abs/2501.09720
- GitHub Repo: https://github.com/Li-Qingyun/mllm-mmrotate
- HuggingFace Page: https://huggingface.co/collections/Qingyun/lmmrotate-6780cabaf49c4e705023b8df
This repo hosts all the available checkpoints of Florence-2 trained for aerial detection with LMMRotate in [our paper](https://arxiv.org/abs/2501.09720).
LMMRotate is a technical practice to fine-tune Large Multimodal language Models for oriented object detection as in MMRotate and hosts the official implementation of the paper: A Simple Aerial Detection Baseline of Multimodal Language Models.
See the list of available checkpoint [here](https://huggingface.co/Qingyun/Florence-2-models-lmmrotate/tree/main).
The folder is named `{base_model}_vis{vision_input_size}-lang{max_language_input_length}_{dataset_name}-{annotation_version}_b{samples_per_gpu}x{num_gpus}-{num_epoch}e-{note}`
For example:
> `florence-2-b_vis1024-lang2048_dota1-train-v2_b2x16-100e-slurm-zero2`:
> - **base_model**: Microsoft/Florence-2-base
> - **vision input size**: 1024 \times 1024
> - **max language input length**: 2048
> - **aerial detection source dataset name**: dota-train (`train` split of `split_ss_dota`)
> - **annotation version**: v2 (the users should ignore this)
> - **batch size and resources**: 2x16gpus = 32
> - **schedule**: 100 epochs
> - **note**: the model is trained on a slurm cluster and accelerated with DeepSpeed ZeRO2
## Downloading Guide
You can download with your web browser on [the file page](https://huggingface.co/datasets/Qingyun/Florence-2-models-lmmrotate/tree/main).
We recommand downloading in terminal using huggingface-cli (`pip install --upgrade huggingface_cli`). You can refer to [the document](https://huggingface.co/docs/huggingface_hub/guides/download) for more usages.
```
# Set Huggingface Mirror for Chinese users (if required):
export HF_ENDPOINT=https://hf-mirror.com
# Download a certain checkpoint:
huggingface-cli download Qingyun/Florence-2-models-lmmrotate --repo-type model --local-dir checkpoint/
# If any error (such as network error) interrupts the downloading, you just need to execute the same command, the latest huggingface_hub will resume downloading.
```
## Detection Performance
## Cite
LMMRotate paper:
```
@article{li2025lmmrotate,
title={A Simple Aerial Detection Baseline of Multimodal Language Models},
author={Li, Qingyun and Chen, Yushi and Shu, Xinya and Chen, Dong and He, Xin and Yu Yi and Yang, Xue },
journal={arXiv preprint arXiv:2501.09720},
year={2025}
}
```