Prompting Medical Vision-Language Models to Mitigate Diagnosis Bias by Generating Realistic Dermoscopic Images

Nusrat Munia, Abdullah-Al-Zubaer Imran
4/2/2025
cs.CV

Official code: Munia03/DermDiT (1 star, last commit 2025-05-22).

Abstract

Artificial Intelligence (AI) in skin disease diagnosis has improved significantly, but a major concern is that these models frequently show biased performance across subgroups, especially regarding sensitive attributes such as skin color. To address these issues, we propose a novel generative AI-based framework, namely, Dermatology Diffusion Transformer (DermDiT), which leverages text prompts generated via Vision Language Models and multimodal text-image learning to generate new dermoscopic images. We utilize large vision language models to generate accurate and proper prompts for each dermoscopic image which helps to generate synthetic images to improve the representation of underrepresented groups (patient, disease, etc.) in highly imbalanced datasets for clinical diagnoses. Our extensive experimentation showcases the large vision language models providing much more insightful representations, that enable DermDiT to generate high-quality images. Our code is available at https://github.com/Munia03/DermDiT

View on arXivView PDF

Code Implementations(1)

Munia03/DermDiTOfficial100%

DermDiT: Prompting Medical Vision-Language Models to Mitigate Diagnosis Bias by Generating Realistic Dermoscopic Images (ISBI 2025)

10PythonJan 12, 20251 years agoNOASSERTION

Similar Papers

DermDiff: Generative Diffusion Model for Mitigating Racial Biases in Dermatology Diagnosis

93% match1 repo32025

DermaFlux: Synthetic Skin Lesion Generation with Rectified Flows for Enhanced Image Classification

88% match2026

SkinGEN: an Explainable Dermatology Diagnosis-to-Generation Framework with Interactive Vision-Language Models

88% match2024

SkinCaRe: A Multimodal Dermatology Dataset Annotated with Medical Caption and Chain-of-Thought Reasoning

87% match2024

Adapting Large Language Models to Mitigate Skin Tone Biases in Clinical Dermatology Tasks: A Mixed-Methods Study

87% match2025

A Multimodal Vision Foundation Model for Clinical Dermatology

86% match1 repo1712024

Cite this paper

@article{munia2025prompting,
  title  = {Prompting Medical Vision-Language Models to Mitigate Diagnosis Bias by Generating Realistic Dermoscopic Images},
  author = {Nusrat Munia and Abdullah-Al-Zubaer Imran},
  year   = {2025},
  eprint = {2504.01838},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2504.01838v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2504.01838.svg)](https://codeofpaper.com/paper/2504.01838)

Discussion