English
Related papers

Related papers: Survey: Transformer-based Models in Data Modality …

200 papers

Transformer architectures have revolutionized machine learning across a wide range of domains, from natural language processing to scientific computing. However, their growing deployment in high-stakes applications, such as computer vision,…

Machine Learning · Computer Science 2026-02-17 Trishit Mondal , Ameya D. Jagtap

Transformers have been at the heart of the Natural Language Processing (NLP) and Computer Vision (CV) revolutions. The significant success in NLP and CV inspired exploring the use of Transformers in point cloud processing. However, how do…

Computer Vision and Pattern Recognition · Computer Science 2022-09-22 Dening Lu , Qian Xie , Mingqiang Wei , Kyle Gao , Linlin Xu , Jonathan Li

Multimodal machine translation involves drawing information from more than one modality, based on the assumption that the additional modalities will contain useful alternative views of the input data. The most prominent tasks in this area…

Computation and Language · Computer Science 2019-12-02 Umut Sulubacak , Ozan Caglayan , Stig-Arne Grönroos , Aku Rouhe , Desmond Elliott , Lucia Specia , Jörg Tiedemann

The Transformer architecture has become prominent in developing large causal language models. However, mechanisms to explain its capabilities are not well understood. Focused on the training process, here we establish a meta-learning view…

Machine Learning · Computer Science 2024-03-26 Xinbo Wu , Lav R. Varshney

The advancements in deep learning, particularly the introduction of transformers, have been pivotal in enhancing various natural language processing (NLP) tasks. These include text-to-text applications such as machine translation, text…

Artificial Intelligence · Computer Science 2024-12-24 Gospel Ozioma Nnadi , Flavio Bertini

Transformer, first applied to the field of natural language processing, is a type of deep neural network mainly based on the self-attention mechanism. Thanks to its strong representation capabilities, researchers are looking at ways to…

Computer Vision and Pattern Recognition · Computer Science 2023-07-11 Kai Han , Yunhe Wang , Hanting Chen , Xinghao Chen , Jianyuan Guo , Zhenhua Liu , Yehui Tang , An Xiao , Chunjing Xu , Yixing Xu , Zhaohui Yang , Yiman Zhang , Dacheng Tao

Transformer architecture has widespread applications, particularly in Natural Language Processing and computer vision. Recently Transformers have been employed in various aspects of time-series analysis. This tutorial provides an overview…

Machine Learning · Computer Science 2023-07-27 Sabeen Ahmed , Ian E. Nielsen , Aakash Tripathi , Shamoon Siddiqui , Ghulam Rasool , Ravi P. Ramachandran

The remarkable performance of the Transformer architecture in natural language processing has recently also triggered broad interest in Computer Vision. Among other merits, Transformers are witnessed as capable of learning long-range…

Computer Vision and Pattern Recognition · Computer Science 2023-11-07 Reza Azad , Amirhossein Kazerouni , Moein Heidari , Ehsan Khodapanah Aghdam , Amirali Molaei , Yiwei Jia , Abin Jose , Rijo Roy , Dorit Merhof

Modeling long texts has been an essential technique in the field of natural language processing (NLP). With the ever-growing number of long documents, it is important to develop effective modeling methods that can process and analyze such…

Computation and Language · Computer Science 2025-06-11 Zican Dong , Tianyi Tang , Junyi Li , Wayne Xin Zhao

This survey explores the adaptation of visual transformer models in Autonomous Driving, a transition inspired by their success in Natural Language Processing. Surpassing traditional Recurrent Neural Networks in tasks like sequential image…

Computer Vision and Pattern Recognition · Computer Science 2024-03-13 Quoc-Vinh Lai-Dang

Transformers have demonstrated remarkable performance in natural language processing and computer vision. However, existing vision Transformers struggle to learn from limited medical data and are unable to generalize on diverse medical…

Image and Video Processing · Electrical Eng. & Systems 2023-04-06 Yunhe Gao , Mu Zhou , Di Liu , Zhennan Yan , Shaoting Zhang , Dimitris N. Metaxas

Image captioning involves generating textual descriptions from input images, bridging the gap between computer vision and natural language processing. Recent advancements in transformer-based models have significantly improved caption…

Computer Vision and Pattern Recognition · Computer Science 2025-06-09 Israa A. Albadarneh , Bassam H. Hammo , Omar S. Al-Kadi

Recently, transformers have become incredibly popular in computer vision and vision-language tasks. This notable rise in their usage can be primarily attributed to the capabilities offered by attention mechanisms and the outstanding ability…

Computer Vision and Pattern Recognition · Computer Science 2023-12-08 Mayank Vatsa , Anubhooti Jain , Richa Singh

The past year has witnessed the rapid development of applying the Transformer module to vision problems. While some researchers have demonstrated that Transformer-based models enjoy a favorable ability of fitting data, there are still…

Computer Vision and Pattern Recognition · Computer Science 2021-12-21 Zhengsu Chen , Lingxi Xie , Jianwei Niu , Xuefeng Liu , Longhui Wei , Qi Tian

Dynamic attention mechanism and global modeling ability make Transformer show strong feature learning ability. In recent years, Transformer has become comparable to CNNs methods in computer vision. This review mainly investigates the…

Computer Vision and Pattern Recognition · Computer Science 2022-03-25 Yuting Yang , Licheng Jiao , Xu Liu , Fang Liu , Shuyuan Yang , Zhixi Feng , Xu Tang

Temporal information extraction (IE) aims to extract structured temporal information from unstructured text, thereby uncovering the implicit timelines within. This technique is applied across domains such as healthcare, newswire, and…

Computation and Language · Computer Science 2025-04-11 Xin Su , Phillip Howard , Steven Bethard

In recent times, Transformer-based language models are making quite an impact in the field of natural language processing. As relevant parallels can be drawn between biological sequences and natural languages, the models used in NLP can be…

Computation and Language · Computer Science 2025-12-25 Nimisha Ghosh , Daniele Santoni , Indrajit Saha , Giovanni Felici

Vision transformers are emerging as a powerful tool to solve computer vision problems. Recent techniques have also proven the efficacy of transformers beyond the image domain to solve numerous video-related tasks. Among those, human action…

Computer Vision and Pattern Recognition · Computer Science 2022-09-14 Anwaar Ulhaq , Naveed Akhtar , Ganna Pogrebna , Ajmal Mian

The fields of generative AI and transfer learning have experienced remarkable advancements in recent years especially in the domain of Natural Language Processing (NLP). Transformers have been at the heart of these advancements where the…

Computation and Language · Computer Science 2024-02-28 Majd Saleh , Stéphane Paquelet

Multimodality Representation Learning, as a technique of learning to embed information from different modalities and their correlations, has achieved remarkable success on a variety of applications, such as Visual Question Answering (VQA),…

Artificial Intelligence · Computer Science 2024-03-04 Muhammad Arslan Manzoor , Sarah Albarri , Ziting Xian , Zaiqiao Meng , Preslav Nakov , Shangsong Liang