MedM-VL: What Makes a Good Medical LVLM?

Yiming Shi, Shaoshuai Yang, Xun Zhu, Haoyu Wang, Xiangling Fu +2 more
4/6/2025
cs.CV

Official code: MSIIP/MedM-VL (57 stars, Apache-2.0, last commit 2025-12-23).

Abstract

Medical image analysis is essential in modern healthcare. Deep learning has redirected research focus toward complex medical multimodal tasks, including report generation and visual question answering. Traditional task-specific models often fall short in handling these challenges. Large vision-language models (LVLMs) offer new solutions for solving such tasks. In this study, we build on the popular LLaVA framework to systematically explore model architectures and training strategies for both 2D and 3D medical LVLMs. We present extensive empirical findings and practical guidance. To support reproducibility and future research, we release a modular codebase, MedM-VL, and two pre-trained models: MedM-VL-2D for 2D medical image analysis and MedM-VL-CT-Chest for 3D CT-based applications. The code is available at: https://github.com/MSIIP/MedM-VL

View on arXivView PDF

Code Implementations(1)

MSIIP/MedM-VLOfficial100%
Apache-2.0

MedM-VL is a modular, LLaVA-based codebase for medical LVLMs.

579PythonDec 19, 202410 months agoApache-2.0
large-vision-language-modelsmedical-image-analysis

Similar Papers

GMAI-VL & GMAI-VL-5.5M: A Large Vision-Language Model and A Comprehensive Multimodal Dataset Towards General Medical AI

90% match1 repo1022024

A Survey of Medical Vision-and-Language Applications and Their Techniques

89% match1 repo352024

Envisioning MedCLIP: A Deep Dive into Explainability for Medical Vision-Language Models

89% match2024

VILA-M3: Enhancing Vision-Language Models with Medical Expert Knowledge

88% match2026

The Quest for Visual Understanding: A Journey Through the Evolution of Visual Question Answering

88% match2025

MoME: Mixture of Visual Language Medical Experts for Medical Imaging Segmentation

88% match2025

Cite this paper

@article{shi2025medmvl,
  title  = {MedM-VL: What Makes a Good Medical LVLM?},
  author = {Yiming Shi and Shaoshuai Yang and Xun Zhu and Haoyu Wang and Xiangling Fu and Miao Li and Ji Wu},
  year   = {2025},
  eprint = {2504.04323},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2504.04323v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2504.04323.svg)](https://codeofpaper.com/paper/2504.04323)

Discussion