ProtoSAM: One-Shot Medical Image Segmentation With Foundational Models

Lev Ayzenberg, Raja Giryes, Hayit Greenspan
7/9/2024
cs.CVcs.AI

Official code: levayz/ProtoSAM (49 stars, GPL-3.0, last commit 2024-10-10).

Abstract

This work introduces a new framework, ProtoSAM, for one-shot medical image segmentation. It combines the use of prototypical networks, known for few-shot segmentation, with SAM - a natural image foundation model. The method proposed creates an initial coarse segmentation mask using the ALPnet prototypical network, augmented with a DINOv2 encoder. Following the extraction of an initial mask, prompts are extracted, such as points and bounding boxes, which are then input into the Segment Anything Model (SAM). State-of-the-art results are shown on several medical image datasets and demonstrate automated segmentation capabilities using a single image example (one shot) with no need for fine-tuning of the foundation model. Our code is available at: https://github.com/levayz/ProtoSAM

View on arXivView PDF

Code Implementations(1)

levayz/ProtoSAMOfficial100%
GPL-3.0

Code implementation of ProtoSAM - One Shot Medical Image Segmentation with Foundationl Models

494Shell, PythonJun 29, 20242 years agoGPL-3.0
computer-visionmedicalsamsegmentation

Similar Papers

One-Prompt to Segment All Medical Images

87% match2 repos1452026

Distillation Learning Guided by Image Reconstruction for One-Shot Medical Image Segmentation

86% match1 repo12024

Few-Shot Adaptation of Training-Free Foundation Model for 3D Medical Image Segmentation

85% match1 repo172025

MedDINOv3: How to adapt vision foundation models for medical image segmentation?

85% match1 repo2282025

Improved Baselines with Synchronized Encoding for Universal Medical Image Segmentation

84% match1 repo52024

VISTA3D: A Unified Segmentation Foundation Model For 3D Medical Imaging

84% match1 repo2942026

Cite this paper

@article{ayzenberg2024protosam,
  title  = {ProtoSAM: One-Shot Medical Image Segmentation With Foundational Models},
  author = {Lev Ayzenberg and Raja Giryes and Hayit Greenspan},
  year   = {2024},
  eprint = {2407.07042},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2407.07042v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2407.07042.svg)](https://codeofpaper.com/paper/2407.07042)

Discussion