English
Related papers

Related papers: Contrastive Learning Meets Transfer Learning: A Ca…

200 papers

Recent advances in robotic learning in simulation have shown impressive results in accelerating learning complex manipulation skills. However, the sim-to-real gap, caused by discrepancies between simulation and reality, poses significant…

Robotics · Computer Science 2025-03-25 Jacinto Colan , Keisuke Sugita , Ana Davila , Yutaro Yamada , Yasuhisa Hasegawa

In many machine learning problems, large-scale datasets have become the de-facto standard to train state-of-the-art deep networks at the price of heavy computation load. In this paper, we focus on condensing large training sets into…

Machine Learning · Computer Science 2021-06-11 Bo Zhao , Hakan Bilen

Brain-computer interfaces (BCIs), is ways for electronic devices to communicate directly with the brain. For most medical-type brain-computer interface tasks, the activity of multiple units of neurons or local field potentials is sufficient…

Machine Learning · Computer Science 2022-05-25 Lang Qian , Shengjie Zheng , Chunshan Deng , Cheng Yang , Xiaojian Li

Self-training and contrastive learning have emerged as leading techniques for incorporating unlabeled data, both under distribution shift (unsupervised domain adaptation) and when it is absent (semi-supervised learning). However, despite…

Learning good representations involves capturing the diverse ways in which data samples relate. Contrastive loss - an objective matching related samples - underlies methods from self-supervised to multimodal learning. Contrastive losses,…

Computer Vision and Pattern Recognition · Computer Science 2024-09-13 Vlad Sobal , Mark Ibrahim , Randall Balestriero , Vivien Cabannes , Diane Bouchacourt , Pietro Astolfi , Kyunghyun Cho , Yann LeCun

Detecting critical transitions in complex, noisy time-series data is a fundamental challenge across science and engineering. Such transitions may be anticipated by the emergence of a low-dimensional order parameter, whose signature is often…

Machine Learning · Computer Science 2025-12-16 Wenqi Fang , Ye Li

Recent self-supervised contrastive learning methods greatly benefit from the Siamese structure that aims to minimizing distances between positive pairs. These methods usually apply random data augmentation to input images, expecting the…

Computer Vision and Pattern Recognition · Computer Science 2023-05-16 Sheng Wang , Zixu Zhuang , Xi Ouyang , Lichi Zhang , Zheren Li , Chong Ma , Tianming Liu , Dinggang Shen , Qian Wang

Contrastive Language-Image Pretraining (CLIP) models excel at understanding image-text relationships but struggle with adapting to new data without forgetting prior knowledge. To address this, models are typically fine-tuned using both new…

Machine Learning · Computer Science 2026-05-06 Ryan King , Gang Li , Bobak Mortazavi , Tianbao Yang

Vast quantities of person-generated health data (wearables) are collected but the process of annotating to feed to machine learning models is impractical. This paper discusses ways in which self-supervised approaches that use contrastive…

Machine Learning · Computer Science 2021-11-16 Kevalee Shah , Dimitris Spathis , Chi Ian Tang , Cecilia Mascolo

Graph representation learning plays a vital role in processing graph-structured data. However, prior arts on graph representation learning heavily rely on labeling information. To overcome this problem, inspired by the recent success of…

Machine Learning · Computer Science 2021-07-19 Ming Jin , Yizhen Zheng , Yuan-Fang Li , Chen Gong , Chuan Zhou , Shirui Pan

Recent methods for learning unsupervised visual representations, dubbed contrastive learning, optimize the noise-contrastive estimation (NCE) bound on mutual information between two views of an image. NCE uses randomly sampled negative…

Machine Learning · Computer Science 2020-10-06 Mike Wu , Milan Mosse , Chengxu Zhuang , Daniel Yamins , Noah Goodman

The deep learning technique has been shown to be effectively addressed several image analysis tasks in the computer-aided diagnosis scheme for mammography. The training of an efficacious deep learning model requires large data with diverse…

Computer Vision and Pattern Recognition · Computer Science 2023-09-08 Zheren Li , Zhiming Cui , Lichi Zhang , Sheng Wang , Chenjin Lei , Xi Ouyang , Dongdong Chen , Xiangyu Zhao , Yajia Gu , Zaiyi Liu , Chunling Liu , Dinggang Shen , Jie-Zhi Cheng

In-Batch contrastive learning is a state-of-the-art self-supervised method that brings semantically-similar instances close while pushing dissimilar instances apart within a mini-batch. Its key to success is the negative sharing strategy,…

Machine Learning · Computer Science 2023-06-07 Zhen Yang , Tinglin Huang , Ming Ding , Yuxiao Dong , Rex Ying , Yukuo Cen , Yangliao Geng , Jie Tang

In Magnetic Resonance Imaging (MRI), image acquisitions are often undersampled in the measurement domain to accelerate the scanning process, at the expense of image quality. However, image quality is a crucial factor that influences the…

Image and Video Processing · Electrical Eng. & Systems 2024-05-31 Mevan Ekanayake , Zhifeng Chen , Mehrtash Harandi , Gary Egan , Zhaolin Chen

Contrastive learning, which aims at minimizing the distance between positive pairs while maximizing that of negative ones, has been widely and successfully applied in unsupervised feature learning, where the design of positive and negative…

Computer Vision and Pattern Recognition · Computer Science 2021-08-09 Rui Zhu , Bingchen Zhao , Jingen Liu , Zhenglong Sun , Chang Wen Chen

Contrastive learning is commonly applied to self-supervised learning, and has been shown to outperform traditional approaches such as the triplet loss and N-pair loss. However, the requirement of large batch sizes and memory banks has made…

Computer Vision and Pattern Recognition · Computer Science 2022-08-15 Rishab Balasubramanian , Rupashree Dey , Kunal Rathore

The synergy of long-range dependencies from transformers and local representations of image content from convolutional neural networks (CNNs) has led to advanced architectures and increased performance for various medical image analysis…

Computer Vision and Pattern Recognition · Computer Science 2023-10-03 Yiqing Shen , Pengfei Guo , Jingpu Wu , Qianqi Huang , Nhat Le , Jinyuan Zhou , Shanshan Jiang , Mathias Unberath

Recent studies have demonstrated the superior performance of introducing ``scan-wise" contrast labels into contrastive learning for multi-organ segmentation on multi-phase computed tomography (CT). However, such scan-wise labels are…

Computer Vision and Pattern Recognition · Computer Science 2022-10-18 Ho Hin Lee , Yucheng Tang , Han Liu , Yubo Fan , Leon Y. Cai , Qi Yang , Xin Yu , Shunxing Bao , Yuankai Huo , Bennett A. Landman

Contrastive learning has moved the state of the art for many tasks in computer vision and information retrieval in recent years. This poster is the first work that applies supervised contrastive learning to the task of product matching in…

Machine Learning · Computer Science 2022-05-03 Ralph Peeters , Christian Bizer

Unsupervised image translation using adversarial learning has been attracting attention to improve the image quality of medical images. However, adversarial training based on the global evaluation values of discriminators does not provide…

Image and Video Processing · Electrical Eng. & Systems 2022-10-25 Takumi Hase , Megumi Nakao , Mitsuhiro Nakamura , Tetsuya Matsuda