Update README.md
Browse files
README.md
CHANGED
@@ -40,6 +40,46 @@ LMMRotate is a technical practice to fine-tune Large Multimodal language Models
|
|
40 |
|
41 |
<img src="https://github.com/user-attachments/assets/d34e4c0c-9e04-446e-a511-2e7005e32074" alt="framework" width="100%" />
|
42 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
43 |
## Detection Performance
|
44 |
|
45 |
-

|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
40 |
|
41 |
<img src="https://github.com/user-attachments/assets/d34e4c0c-9e04-446e-a511-2e7005e32074" alt="framework" width="100%" />
|
42 |
|
43 |
+
The folder is named `{base_model}_vis{vision_input_size}-lang{max_language_input_length}_{dataset_name}-{annotation_version}_b{samples_per_gpu}x{num_gpus}-{num_epoch}e-{note}`
|
44 |
+
|
45 |
+
For example:
|
46 |
+
|
47 |
+
> `florence-2-b_vis1024-lang2048_dota1-train-v2_b2x16-100e-slurm-zero2`:
|
48 |
+
> - **base_model**: Microsoft/Florence-2-base
|
49 |
+
> - **vision input size**: 1024 \times 1024
|
50 |
+
> - **max language input length**: 2048
|
51 |
+
> - **aerial detection source dataset name**: dota-train (`train` split of `split_ss_dota`)
|
52 |
+
> - **annotation version**: v2 (the users should ignore this)
|
53 |
+
> - **batch size and resources**: 2x16gpus = 32
|
54 |
+
> - **schedule**: 100 epochs
|
55 |
+
> - **note**: the model is trained on a slurm cluster and accelerated with DeepSpeed ZeRO2
|
56 |
+
|
57 |
+
## Downloading Guide
|
58 |
+
|
59 |
+
You can download with your web browser on [the file page](https://huggingface.co/datasets/Qingyun/Florence-2-models-lmmrotate/tree/main).
|
60 |
+
|
61 |
+
We recommand downloading in terminal using huggingface-cli (`pip install --upgrade huggingface_cli`). You can refer to [the document](https://huggingface.co/docs/huggingface_hub/guides/download) for more usages.
|
62 |
+
|
63 |
+
```
|
64 |
+
# Set Huggingface Mirror for Chinese users (if required):
|
65 |
+
export HF_ENDPOINT=https://hf-mirror.com
|
66 |
+
# Download a certain checkpoint:
|
67 |
+
huggingface-cli download Qingyun/Florence-2-models-lmmrotate <checkpoint_folder_name> --repo-type model --local-dir checkpoint/
|
68 |
+
# If any error (such as network error) interrupts the downloading, you just need to execute the same command, the latest huggingface_hub will resume downloading.
|
69 |
+
```
|
70 |
+
|
71 |
## Detection Performance
|
72 |
|
73 |
+

|
74 |
+
|
75 |
+
## Cite
|
76 |
+
|
77 |
+
LMMRotate paper:
|
78 |
+
```
|
79 |
+
@article{li2025lmmrotate,
|
80 |
+
title={A Simple Aerial Detection Baseline of Multimodal Language Models},
|
81 |
+
author={Li, Qingyun and Chen, Yushi and Shu, Xinya and Chen, Dong and He, Xin and Yu Yi and Yang, Xue },
|
82 |
+
journal={arXiv preprint arXiv:2501.09720},
|
83 |
+
year={2025}
|
84 |
+
}
|
85 |
+
```
|