Xmodel-VLM: A Simple Baseline for Multimodal Vision Language Model

Wanting Xu, Yang Liu, Langping He, Xucheng Huang, Ling Jiang
5/15/2024
cs.CVcs.AI

Official code: XiaoduoAILab/XmodelVLM (68 stars, Apache-2.0, last commit 2024-06-20).

Abstract

We introduce Xmodel-VLM, a cutting-edge multimodal vision language model. It is designed for efficient deployment on consumer GPU servers. Our work directly confronts a pivotal industry issue by grappling with the prohibitive service costs that hinder the broad adoption of large-scale multimodal systems. Through rigorous training, we have developed a 1B-scale language model from the ground up, employing the LLaVA paradigm for modal alignment. The result, which we call Xmodel-VLM, is a lightweight yet powerful multimodal vision language model. Extensive testing across numerous classic multimodal benchmarks has revealed that despite its smaller size and faster execution, Xmodel-VLM delivers performance comparable to that of larger models. Our model checkpoints and code are publicly available on GitHub at https://github.com/XiaoduoAILab/XmodelVLM.

View on arXivView PDF

Code Implementations(1)

Apache-2.0
687Shell, PythonMay 8, 20242 years agoApache-2.0

Similar Papers

Cite this paper

@article{xu2024xmodelvlm,
  title  = {Xmodel-VLM: A Simple Baseline for Multimodal Vision Language Model},
  author = {Wanting Xu and Yang Liu and Langping He and Xucheng Huang and Ling Jiang},
  year   = {2024},
  eprint = {2405.09215},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2405.09215v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2405.09215.svg)](https://codeofpaper.com/paper/2405.09215)

Discussion