Cheap Lunch for Medical Image Segmentation by Fine-tuning SAM on Few Exemplars

by   Weijia Feng, et al.

The Segment Anything Model (SAM) has demonstrated remarkable capabilities of scaled-up segmentation models, enabling zero-shot generalization across a variety of domains. By leveraging large-scale foundational models as pre-trained models, it is a natural progression to fine-tune SAM for specific domains to further enhance performances. However, the adoption of foundational models in the medical domain presents a challenge due to the difficulty and expense of labeling sufficient data for adaptation within hospital systems. In this paper, we introduce an efficient and practical approach for fine-tuning SAM using a limited number of exemplars, making it suitable for such scenarios. Our approach combines two established techniques from the literature: an exemplar-guided synthesis module and the widely recognized Low-Rank Adaptation (LoRA) fine-tuning strategy, serving as data-level and model-level attempts respectively. Interestingly, our empirical findings suggest that SAM can be effectively aligned within the medical domain even with few labeled data. We validate our approach through experiments on brain tumor segmentation (BraTS) and multi-organ CT segmentation (Synapse). The comprehensive results underscore the feasibility and effectiveness of such an approach, paving the way for the practical application of SAM in the medical domain.


page 6

page 7


Transductive few-shot adapters for medical image segmentation

With the recent raise of foundation models in computer vision and NLP, t...

DVPT: Dynamic Visual Prompt Tuning of Large Pre-trained Models for Medical Image Analysis

Limited labeled data makes it hard to train models from scratch in medic...

3DSAM-adapter: Holistic Adaptation of SAM from 2D to 3D for Promptable Medical Image Segmentation

Despite that the segment anything model (SAM) achieved impressive result...

Large-scale Generative Modeling to Improve Automated Veterinary Disease Coding

Supervised learning is limited both by the quantity and quality of the l...

When SAM Meets Sonar Images

Segment Anything Model (SAM) has revolutionized the way of segmentation....

Prompt-based Tuning of Transformer Models for Multi-Center Medical Image Segmentation

Medical image segmentation is a vital healthcare endeavor requiring prec...

SLPT: Selective Labeling Meets Prompt Tuning on Label-Limited Lesion Segmentation

Medical image analysis using deep learning is often challenged by limite...

Please sign up or login with your details

Forgot password? Click here to reset