VideoLLaMA-7B-Charades-VTune Model

Model details

We trained Video-LLaMA using VTune, a developed instruction-tuning method specifically designed to account for consistency.

For the tuning, we utilized 5K training videos from Charades-STA with 99K automatically generated annotations.

Evaluation

We evaluated the model on Charades-CON and Charades-STA.

• Charades-CON

  Metric	Value
  Ground	54.4
  R-Ground	38.2 (70.3)
  S-Ground	10.9 (20.0)
  H-Verify	30.7 (56.6)
  C-Verify	30.0 (55.2)

• Charades-STA

  Metric	Value
  R@1 IoU=0.3	51.18
  R@1 IoU=0.5	37.15
  R@1 IoU=0.7	20.11
  mIoU	35.29

Paper and Code for more information: Paper, Code

Citation

If you find our research and codes useful, please consider starring our repository and citing our paper:

@article{jung2024consistency,
  title={On the Consistency of Video Large Language Models in Temporal Comprehension},
  author={Jung, Minjoon and Xiao, Junbin and Zhang, Byoung-Tak and Yao, Angela},
  journal={arXiv preprint arXiv:2411.12951},
  year={2024}
}