English
Related papers

Related papers: FUSAR-KLIP: Towards Multimodal Foundation Models f…

200 papers

Guided image super-resolution (GISR) aims to obtain a high-resolution (HR) target image by enhancing the spatial resolution of a low-resolution (LR) target image under the guidance of a HR image. However, previous model-based methods mainly…

Image and Video Processing · Electrical Eng. & Systems 2022-03-11 Man Zhou , Keyu Yan , Jinshan Pan , Wenqi Ren , Qi Xie , Xiangyong Cao

Remote sensing imagery presents vast, inherently unstructured spatial data, necessitating sophisticated reasoning to interpret complex user intents and contextual relationships beyond simple recognition tasks. In this paper, we aim to…

Computer Vision and Pattern Recognition · Computer Science 2025-12-24 Liang Yao , Fan Liu , Hongbo Lu , Chuanyi Zhang , Rui Min , Shengxiang Xu , Shimin Di , Pai Peng

The landscape of publicly available vision foundation models (VFMs), such as CLIP and Segment Anything Model (SAM), is expanding rapidly. VFMs are endowed with distinct capabilities stemming from their pre-training objectives. For instance,…

Sub-pixel matching of multimodal optical images is a critical step in combined application of multiple sensors. However structural noise and inconsistencies arising from variations in multimodal image responses usually limit the accuracy of…

Computer Vision and Pattern Recognition · Computer Science 2026-01-06 Tao Huang , Hongbo Pan , Nanxi Zhou , Siyuan Zou , Shun Zhou

High-quality remote sensing (RS) image acquisition is fundamentally constrained by physical limitations. While Multi-Frame Super-Resolution (MFSR) and Pansharpening address this by exploiting complementary information, they are typically…

Image and Video Processing · Electrical Eng. & Systems 2026-04-08 Yufei Tong , Guanjie Cheng , Peihan Wu , Feiyi Chen , Xinkui Zhao , Shuiguang Deng

Multimodal Large Language Models (MLLMs) demonstrate exceptional semantic reasoning but struggle with 3D spatial perception when restricted to pure RGB inputs. Despite leveraging implicit geometric priors from 3D reconstruction models,…

Computer Vision and Pattern Recognition · Computer Science 2026-03-18 Jiaxin Zhang , Junjun Jiang , Haijie Li , Youyu Chen , Kui Jiang , Dave Zhenyu Chen

The rapid development of photo-realistic face generation methods has raised significant concerns in society and academia, highlighting the urgent need for robust and generalizable face forgery detection (FFD) techniques. Although existing…

Computer Vision and Pattern Recognition · Computer Science 2025-06-03 Yaning Zhang , Tianyi Wang , Zitong Yu , Zan Gao , Linlin Shen , Shengyong Chen

Is vision good enough for language? Recent advancements in multimodal models primarily stem from the powerful reasoning abilities of large language models (LLMs). However, the visual component typically depends only on the instance-level…

Computer Vision and Pattern Recognition · Computer Science 2024-04-26 Shengbang Tong , Zhuang Liu , Yuexiang Zhai , Yi Ma , Yann LeCun , Saining Xie

Establishing dense correspondence across 3D shapes is crucial for fundamental downstream tasks, including texture transfer, shape interpolation, and robotic manipulation. However, learning these mappings without manual supervision remains a…

Computer Vision and Pattern Recognition · Computer Science 2026-03-10 Qinfeng Xiao , Guofeng Mei , Qilong Liu , Chenyuan Yi , Fabio Poiesi , Jian Zhang , Bo Yang , Yick Kit-lun

Modern unmanned aerial vehicles (UAVs) are irreplaceable in search and rescue (SAR) missions to obtain a situational overview or provide closeups without endangering personnel. However, UAVs heavily rely on global navigation satellite…

Robotics · Computer Science 2024-12-04 Jan Quenzel , Linus T. Mallwitz , Benedikt T. Arnold , Sven Behnke

Spatial intelligence requires multimodal large language models (MLLMs) to move beyond single-view perception and reason consistently about objects, visibility, geometry, and interactions across multiple viewpoints. However, progress in…

Computer Vision and Pattern Recognition · Computer Science 2026-05-19 Wei Wang , Yuqian Yuan , Tianwei Lin , Wenqiao Zhang , Siliang Tang , Jun Xiao , Yueting Zhuang

Self-supervised frameworks for representation learning have recently stirred up interest among the remote sensing community, given their potential to mitigate the high labeling costs associated with curating large satellite image datasets.…

Computer Vision and Pattern Recognition · Computer Science 2025-03-25 Hugo Chan-To-Hing , Bharadwaj Veeravalli

Subsurface geomodeling plays a critical role in reservoir characterization, uncertainty quantification, and subsurface flow prediction. However, integrating heterogeneous sources of geological information, including conceptual geological…

Geophysics · Physics 2026-05-26 Jiayuan Huang , Suihong Song , Tapan Mukerji

Despite their proficiency in general tasks, Multi-modal Large Language Models (MLLMs) struggle with automatic Geometry Problem Solving (GPS), which demands understanding diagrams, interpreting symbols, and performing complex reasoning. This…

Computer Vision and Pattern Recognition · Computer Science 2025-01-13 Renqiu Xia , Mingsheng Li , Hancheng Ye , Wenjie Wu , Hongbin Zhou , Jiakang Yuan , Tianshuo Peng , Xinyu Cai , Xiangchao Yan , Bin Wang , Conghui He , Botian Shi , Tao Chen , Junchi Yan , Bo Zhang

Visual latent reasoning lets a multimodal large language model (MLLM) create intermediate visual evidence as continuous tokens, avoiding external tools or image generators. However, existing methods usually follow an output-as-input latent…

Computer Vision and Pattern Recognition · Computer Science 2026-05-26 Yanting Miao , Yutao Sun , Dexin Wang , Mengyu Zhou , Pascal Poupart , Lei Lv , Qi Zhao , Li Wang , Hao Li , Xiaoxi Jiang , Guanjun Jiang

In this work, we address the critical yet underexplored challenge of symmetric multimodal-to-multimodal (MM2MM) retrieval, where queries and contexts are interchangeable. Existing universal multimodal retrieval works struggle with this…

Computer Vision and Pattern Recognition · Computer Science 2026-05-18 Wenjie Yang , Hang Yu , Yuyu Guo , Peng Di

Registration of optical and synthetic aperture radar (SAR) remote sensing images serves as a critical foundation for image fusion and visual navigation tasks. This task is particularly challenging because of their modal discrepancy,…

Image and Video Processing · Electrical Eng. & Systems 2025-11-04 Zixuan Sun , Shuaifeng Zhi , Ruize Li , Jingyuan Xia , Yongxiang Liu , Weidong Jiang

Recent advances in remote sensing have led to an increase in the number of available foundation models; each trained on different modalities, datasets, and objectives, yet capturing only part of the vast geospatial knowledge landscape.…

Computer Vision and Pattern Recognition · Computer Science 2026-03-25 Joelle Hanna , Damian Falk , Stella X. Yu , Damian Borth

Multi-modal large language models (MLLMs) have achieved remarkable success in image- and region-level remote sensing (RS) image understanding tasks, such as image captioning, visual question answering, and visual grounding. However,…

Computer Vision and Pattern Recognition · Computer Science 2025-03-14 Ruizhe Ou , Yuan Hu , Fan Zhang , Jiaxin Chen , Yu Liu

This paper presents a unique outdoor aerial visual-inertial-LiDAR dataset captured using a multi-sensor payload to promote the global navigation satellite system (GNSS)-denied navigation research. The dataset features flight distances…