Qwen

/

Qwen2-VL-7B-Instruct

like 602

There are some issues in the https://github.com/hiyouga/LLaMA-Factory ; the full fine-tuning YAML file and related modifications have not been updated to the latest version.

Hi @YangJiassh , we would be appreciated if you can provide some details

https://github.com/hiyouga/LLaMA-Factory/commit/727e1848401d306274fb60ba78f66fed577b7b55

These modifications are removed in the latest version.

examples/train_full/qwen2vl_full_sft.yaml

@YangJiassh Thanks! You can try this recipe for full tuning:

### model
model_name_or_path: Qwen/Qwen2-VL-7B-Instruct

### method
stage: sft
do_train: true
finetuning_type: full
deepspeed: examples/deepspeed/ds_z3_config.json

### dataset
dataset: mllm_demo
template: qwen2_vl
cutoff_len: 1024
max_samples: 1000
overwrite_cache: true
preprocessing_num_workers: 16

### output
output_dir: saves/qwen2_vl-7b/full/sft
logging_steps: 10
save_steps: 500
plot_loss: true
overwrite_output_dir: true

### train
per_device_train_batch_size: 1
gradient_accumulation_steps: 8
learning_rate: 1.0e-4
num_train_epochs: 3.0
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: true
ddp_timeout: 180000000

### eval
val_size: 0.1
per_device_eval_batch_size: 1
eval_strategy: steps
eval_steps: 500

OK thanks I'll try it now

LLaMA Factory has integrated Qwen2-VL models for SFT and DPO, try our training recipes and webUI🚀

LoRA repices: https://github.com/hiyouga/LLaMA-Factory/blob/main/examples/train_lora/qwen2vl_lora_sft.yaml

Damn you were fast XD, is there an example for DPO tho ? I don't see any

Both this tool and the model are amazing! SFT with multimedia has never been so easy and it's interesting to see how well the image training translates to the video mode. 加油！

@nicolollo Yeah! We have just supported Qwen2-VL DPO training, try this example: https://github.com/hiyouga/LLaMA-Factory/blob/main/examples/train_lora/qwen2vl_lora_dpo.yaml

@hiyouga
Amazing stuff.
Can you kindly add a config for Qwen Audio 2 as well? both fft and lora. this model is essential for those of us in the Audio realm