Synthetic Boost: Leveraging Synthetic Data for Enhanced Vision-Language Segmentation in Echocardiography

Rabin Adhikari, Manish Dhakal, Safal Thapaliya, Kanchan Poudel, Prasiddha Bhandari +1 more
9/22/2023
cs.CVcs.AIcs.CL

Official code: naamiinepal/synthetic-boost (3 stars, MIT, last commit 2024-08-30).

Abstract

Accurate segmentation is essential for echocardiography-based assessment of cardiovascular diseases (CVDs). However, the variability among sonographers and the inherent challenges of ultrasound images hinder precise segmentation. By leveraging the joint representation of image and text modalities, Vision-Language Segmentation Models (VLSMs) can incorporate rich contextual information, potentially aiding in accurate and explainable segmentation. However, the lack of readily available data in echocardiography hampers the training of VLSMs. In this study, we explore using synthetic datasets from Semantic Diffusion Models (SDMs) to enhance VLSMs for echocardiography segmentation. We evaluate results for two popular VLSMs (CLIPSeg and CRIS) using seven different kinds of language prompts derived from several attributes, automatically extracted from echocardiography images, segmentation masks, and their metadata. Our results show improved metrics and faster convergence when pretraining VLSMs on SDM-generated synthetic images before finetuning on real images. The code, configs, and prompts are available at https://github.com/naamiinepal/synthetic-boost.

View on arXivView PDF

Code Implementations(1)

naamiinepal/synthetic-boostOfficial100%
MIT

[ASMUS 2023] Synthetic Boost: Leveraging Synthetic Data for Enhanced Vision-Language Segmentation in Echocardiography

30TeX, ShellAug 14, 20232 years agoMIT
cardiac-segmentationechocardiographymlpaper

Similar Papers

Vision-Language Synthetic Data Enhances Echocardiography Downstream Tasks

87% match1 repo132024

Echo from noise: synthetic ultrasound image generation using diffusion models for real image segmentation

86% match1 repo542023

Efficient Semantic Diffusion Architectures for Model Training on Synthetic Echocardiograms

84% match1 repo62024

Generative augmentations for improved cardiac ultrasound segmentation using diffusion models

83% match1 repo112025

Prompt-driven Universal Model for View-Agnostic Echocardiography Analysis

82% match2024

From Transthoracic to Transesophageal: Cross-Modality Generation using LoRA Diffusion

82% match2025

Cite this paper

@article{adhikari2023synthetic,
  title  = {Synthetic Boost: Leveraging Synthetic Data for Enhanced Vision-Language Segmentation in Echocardiography},
  author = {Rabin Adhikari and Manish Dhakal and Safal Thapaliya and Kanchan Poudel and Prasiddha Bhandari and Bishesh Khanal},
  year   = {2023},
  eprint = {2309.12829},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2309.12829v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2309.12829.svg)](https://codeofpaper.com/paper/2309.12829)

Discussion