English
Related papers

Related papers: Segment Anything with Multiple Modalities

200 papers

Foundation models have taken over natural language processing and image generation domains due to the flexibility of prompting. With the recent introduction of the Segment Anything Model (SAM), this prompt-driven paradigm has entered image…

Image and Video Processing · Electrical Eng. & Systems 2023-04-13 Saikat Roy , Tassilo Wald , Gregor Koehler , Maximilian R. Rokuss , Nico Disch , Julius Holzschuh , David Zimmerer , Klaus H. Maier-Hein

This paper presents the Autonomous Driving Segment Anything Model (AD-SAM), a fine-tuned vision foundation model for semantic segmentation in autonomous driving (AD). AD-SAM extends the Segment Anything Model (SAM) with a dual-encoder and…

Computer Vision and Pattern Recognition · Computer Science 2025-11-03 Mario Camarena , Het Patel , Fatemeh Nazari , Evangelos Papalexakis , Mohamadhossein Noruzoliaee , Jia Chen

Recent segmentation methods, which adopt large-scale data training and transformer architecture, aim to create one foundation model that can perform multiple tasks. However, most of these methods rely on heavy encoder and decoder…

Computer Vision and Pattern Recognition · Computer Science 2025-03-03 Shilin Xu , Haobo Yuan , Qingyu Shi , Lu Qi , Jingbo Wang , Yibo Yang , Yining Li , Kai Chen , Yunhai Tong , Bernard Ghanem , Xiangtai Li , Ming-Hsuan Yang

Segment Anything Model (SAM) is an advanced foundational model for image segmentation, which is gradually being applied to remote sensing images (RSIs). Due to the domain gap between RSIs and natural images, traditional methods typically…

Computer Vision and Pattern Recognition · Computer Science 2025-01-14 Nanqing Liu , Xun Xu , Yongyi Su , Haojie Zhang , Heng-Chao Li

Image segmentation remains a pivotal component in medical image analysis, aiding in the extraction of critical information for precise diagnostic practices. With the advent of deep learning, automated image segmentation methods have risen…

Image and Video Processing · Electrical Eng. & Systems 2024-03-07 Nhat-Tan Bui , Dinh-Hieu Hoang , Minh-Triet Tran , Gianfranco Doretto , Donald Adjeroh , Brijesh Patel , Arabinda Choudhary , Ngan Le

The Segment Anything Model (SAM) demonstrates impressive zero-shot segmentation ability on natural images but encounters difficulties in medical imaging due to domain shifts, anatomical variability, and its reliance on user-provided…

Computer Vision and Pattern Recognition · Computer Science 2025-09-24 Yingzhen Hu , Yiheng Zhong , Ruobing Li , Yingxue Su , Jiabao An , Feilong Tang , Jionglong Su , Imran Razzak

The success of large language models has inspired the computer vision community to explore image segmentation foundation model that is able to zero/few-shot generalize through prompt engineering. Segment-Anything(SAM), among others, is the…

Computer Vision and Pattern Recognition · Computer Science 2024-04-11 Haojie Zhang , Yongyi Su , Xun Xu , Kui Jia

This paper introduces a new Segment Anything Model with Depth Perception (DSAM) for Camouflaged Object Detection (COD). DSAM exploits the zero-shot capability of SAM to realize precise segmentation in the RGB-D domain. It consists of the…

Computer Vision and Pattern Recognition · Computer Science 2024-07-18 Zhenni Yu , Xiaoqin Zhang , Li Zhao , Yi Bin , Guobao Xiao

In this paper, we address the challenge of image resolution variation for the Segment Anything Model (SAM). SAM, known for its zero-shot generalizability, exhibits a performance degradation when faced with datasets with varying image sizes.…

Computer Vision and Pattern Recognition · Computer Science 2024-03-21 Yiran Song , Qianyu Zhou , Xiangtai Li , Deng-Ping Fan , Xuequan Lu , Lizhuang Ma

Segment Anything Model (SAM), a new AI model from Meta AI released in April 2023, is an ambitious tool designed to identify and separate individual objects within a given image through semantic interpretation. The advanced capabilities of…

Image and Video Processing · Electrical Eng. & Systems 2024-11-06 Gabriel Bellon de Carvalho , Jurandy Almeida

Semantic segmentation of remote sensing imagery plays a pivotal role in extracting precise information for diverse down-stream applications. Recent development of the Segment Anything Model (SAM), an advanced general-purpose segmentation…

Computer Vision and Pattern Recognition · Computer Science 2023-12-21 Xianping Ma , Qianqian Wu , Xingyu Zhao , Xiaokang Zhang , Man-On Pun , Bo Huang

The Segment Anything Model (SAM) exhibits remarkable versatility and zero-shot learning abilities, owing largely to its extensive training data (SA-1B). Recognizing SAM's dependency on manual guidance given its category-agnostic nature, we…

Computer Vision and Pattern Recognition · Computer Science 2023-11-23 Xiyu Qi , Yifan Wu , Yongqiang Mao , Wenhui Zhang , Yidan Zhang

Vision foundation models like the Segment Anything Model (SAM), pretrained on large-scale natural image datasets, often struggle in medical image segmentation due to a lack of domain-specific adaptation. In clinical practice, fine-tuning…

Computer Vision and Pattern Recognition · Computer Science 2025-11-03 Zelin Liu , Sicheng Dong , Bocheng Li , Yixuan Yang , Jiacheng Ruan , Chenxu Zhou , Suncheng Xiang

Background: The segment-anything model (SAM), introduced in April 2023, shows promise as a benchmark model and a universal solution to segment various natural images. It comes without previously-required re-training or fine-tuning specific…

Image and Video Processing · Electrical Eng. & Systems 2023-05-09 Sheng He , Rina Bao , Jingpeng Li , Jeffrey Stout , Atle Bjornerud , P. Ellen Grant , Yangming Ou

Video segmentation is essential for advancing robotics and autonomous driving, particularly in open-world settings where continuous perception and object association across video frames are critical. While the Segment Anything Model (SAM)…

Computer Vision and Pattern Recognition · Computer Science 2024-10-14 Pinxue Guo , Zixu Zhao , Jianxiong Gao , Chongruo Wu , Tong He , Zheng Zhang , Tianjun Xiao , Wenqiang Zhang

Medical image segmentation has been traditionally approached by training or fine-tuning the entire model to cater to any new modality or dataset. However, this approach often requires tuning a large number of parameters during training.…

Computer Vision and Pattern Recognition · Computer Science 2024-08-14 Jay N. Paranjape , Shameema Sikder , S. Swaroop Vedula , Vishal M. Patel

The Segment Anything Model (SAM) emerges as a powerful vision foundation model to generate high-quality 2D segmentation results. This paper aims to generalize SAM to segment 3D objects. Rather than replicating the data acquisition and…

Computer Vision and Pattern Recognition · Computer Science 2024-04-17 Jiazhong Cen , Jiemin Fang , Zanwei Zhou , Chen Yang , Lingxi Xie , Xiaopeng Zhang , Wei Shen , Qi Tian

Foundation models are experiencing a surge in popularity. The Segment Anything model (SAM) asserts an ability to segment a wide spectrum of objects but required supervised training at unprecedented scale. We compared SAM's performance…

Image and Video Processing · Electrical Eng. & Systems 2025-03-11 Danielle Ferreira , Rima Arnaout

Segment Anything Model (SAM), a vision foundation model trained on large-scale annotations, has recently continued raising awareness within medical image segmentation. Despite the impressive capabilities of SAM on natural scenes, it…

Image and Video Processing · Electrical Eng. & Systems 2023-11-15 Yinuo Wang , Kai Chen , Weimin Yuan , Cai Meng , XiangZhi Bai

Automatic segmentation of medical images is crucial in modern clinical workflows. The Segment Anything Model (SAM) has emerged as a versatile tool for image segmentation without specific domain training, but it requires human prompts and…

Computer Vision and Pattern Recognition · Computer Science 2024-05-16 Yunxiang Li , Bowen Jing , Zihan Li , Jing Wang , You Zhang