WildVision: Evaluating Vision-Language Models in the Wild with Human Preferences

NeurIPS'24 Datasets&Benchmarks 🤗 Demo | 🤗 Paper | 📖 arXiv | GitHub

We are a team from AI2, UCSB, UWaterloo, UPenn, NTU, UWM, and UCSC, and we are working on benchmarking vision language models.

Team Member: Yujie Lu, Dongfu Jiang, Xingyu Fu, Hui Chen, Yingzi Ma, Jing Gu, Michael Saxon

Advisor: Bill Yuchen Lin, Wenhu Chen, Chaowei Xiao, Yejin Choi, Miguel Eckstein, William Yang Wang

Compare VLMs at WildVision-Arena and WildVision-Bench.

More chat and vote data will be updated reguarly. Eval script is released here WildVision-Bench

Contact: Bill Yuchen Lin ([email protected]) and Yujie Lu ([email protected])

Citation: If you found this huggingface space useful, please consider cite us:

@article{lu2024wildvision,
  title={WildVision: Evaluating Vision-Language Models in the Wild with Human Preferences},
  author={Lu, Yujie and Jiang, Dongfu and Chen, Wenhu and Wang, William Yang and Choi, Yejin and Lin, Bill Yuchen},
  publisher={NeurIPS},
  year={2024}
}
@misc{yujie2024wildvisionarena,
    title={WildVision Arena: Benchmarking Multimodal LLMs in the Wild},
    url={https://huggingface.co/spaces/WildVision/vision-arena/},
    author={Lu, Yujie and Jiang, Dongfu and Chen, Hui and Ma, Yingzi and Gu, Jing and Xiao, Chaowei and Chen, Wenhu and Wang, William and Choi, Yejin and Lin, Bill Yuchen},
    year={2024}
}
@misc{yujie2024wildvisionv2,
    title={WildVision Data and Model},
    url={https://huggingface.co/WildVision},
    author={Lu, Yujie* and Jiang, Dongfu* and Chen, Hui* and Fu, Xingyu and Ma, Yingzi and Gu, Jing and Saxon, Michael and Xiao, Chaowei and Chen, Wenhu and Choi, Yejin and Lin, Bill Yuchen and Eckstein, Miguel and Wang, William},
    year={2024}
}