English
Related papers

Related papers: Automated Image Captioning with CNNs and Transform…

200 papers

In this paper, we propose a novel conditional-generative-adversarial-nets-based image captioning framework as an extension of traditional reinforcement-learning (RL)-based encoder-decoder architecture. To deal with the inconsistent…

Computer Vision and Pattern Recognition · Computer Science 2020-09-10 Chen Chen , Shuai Mu , Wanpeng Xiao , Zexiong Ye , Liesi Wu , Qi Ju

Image captioning models aim at connecting Vision and Language by providing natural language descriptions of input images. In the past few years, the task has been tackled by learning parametric models and proposing visual feature extraction…

Computer Vision and Pattern Recognition · Computer Science 2022-08-23 Sara Sarto , Marcella Cornia , Lorenzo Baraldi , Rita Cucchiara

Automated audio captioning is a cross-modal translation task that aims to generate natural language descriptions for given audio clips. This task has received increasing attention with the release of freely available datasets in recent…

Audio and Speech Processing · Electrical Eng. & Systems 2022-09-28 Xinhao Mei , Xubo Liu , Mark D. Plumbley , Wenwu Wang

Automatically describing the content of an image is a fundamental problem in artificial intelligence that connects computer vision and natural language processing. In this paper, we present a generative model based on a deep recurrent…

Computer Vision and Pattern Recognition · Computer Science 2015-04-22 Oriol Vinyals , Alexander Toshev , Samy Bengio , Dumitru Erhan

Image captioning is a fast-growing research field of computer vision and natural language processing that involves creating text explanations for images. This study aims to develop a system that uses a pre-trained convolutional neural…

Computation and Language · Computer Science 2022-03-04 Rashid Khan , M Shujah Islam , Khadija Kanwal , Mansoor Iqbal , Md. Imran Hossain , Zhongfu Ye

Image captioning is a challenging task at the intersection of computer vision and natural language processing, requiring models to generate meaningful textual descriptions of images. Traditional approaches rely on recurrent neural networks…

Computer Vision and Pattern Recognition · Computer Science 2025-02-27 Hemanth Teja Yanambakkam , Rahul Chinthala

Generating natural language descriptions of images is an important capability for a robot or other visual-intelligence driven AI agent that may need to communicate with human users about what it is seeing. Such image captioning methods are…

Computer Vision and Pattern Recognition · Computer Science 2017-11-29 Li Zhang , Flood Sung , Feng Liu , Tao Xiang , Shaogang Gong , Yongxin Yang , Timothy M. Hospedales

Much recent progress in Vision-to-Language problems has been achieved through a combination of Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs). This approach does not explicitly represent high-level semantic…

Computer Vision and Pattern Recognition · Computer Science 2016-12-19 Qi Wu , Chunhua Shen , Anton van den Hengel , Peng Wang , Anthony Dick

Automatically generating a natural language description of an image is a task close to the heart of image understanding. In this paper, we present a multi-model neural network method closely related to the human visual system that…

Computer Vision and Pattern Recognition · Computer Science 2017-06-09 Zhongliang Yang , Yu-Jin Zhang , Sadaqat ur Rehman , Yongfeng Huang

In this paper, we perform a thorough sensitivity analysis on state-of-the-art image captioning approaches using two different architectures: CNN+LSTM and CNN+Transformer. Experiments were carried out using the Flickr8k dataset. The biggest…

Computer Vision and Pattern Recognition · Computer Science 2020-06-22 Amish Patel , Aravind Varier

Video captioning is an essential technology to understand scenes and describe events in natural language. To apply it to real-time monitoring, a system needs not only to describe events accurately but also to produce the captions as soon as…

Computer Vision and Pattern Recognition · Computer Science 2021-08-05 Chiori Hori , Takaaki Hori , Jonathan Le Roux

This research explores the realm of neural image captioning using deep learning models. The study investigates the performance of different neural architecture configurations, focusing on the inject architecture, and proposes a novel…

Computer Vision and Pattern Recognition · Computer Science 2023-12-04 Pooja Bhatnagar , Sai Mrunaal , Sachin Kamnure

Image captioning creates informative text from an input image by creating a relationship between the words and the actual content of an image. Recently, deep learning models that utilize transformers have been the most successful in…

Computer Vision and Pattern Recognition · Computer Science 2025-01-28 Israa Al Badarneh , Bassam Hammo , Omar Al-Kadi

Image captioning is a challenging task involving generating a textual description for an image using computer vision and natural language processing techniques. This paper proposes a deep neural framework for image caption generation using…

Computer Vision and Pattern Recognition · Computer Science 2023-10-12 Rashid Khan , Bingding Huang , Haseeb Hassan , Asim Zaman , Zhongfu Ye

Language Models based on recurrent neural networks have dominated recent image caption generation tasks. In this paper, we introduce a Language CNN model which is suitable for statistical language modeling tasks and shows competitive…

Computer Vision and Pattern Recognition · Computer Science 2017-08-03 Jiuxiang Gu , Gang Wang , Jianfei Cai , Tsuhan Chen

Language and vision are processed as two different modal in current work for image captioning. However, recent work on Super Characters method shows the effectiveness of two-dimensional word embedding, which converts text classification…

Computation and Language · Computer Science 2019-06-05 Baohua Sun , Lin Yang , Michael Lin , Charles Young , Patrick Dong , Wenhan Zhang , Jason Dong

Image captioning is a challenging task and attracting more and more attention in the field of Artificial Intelligence, and which can be applied to efficient image retrieval, intelligent blind guidance and human-computer interaction, etc. In…

Computer Vision and Pattern Recognition · Computer Science 2019-05-21 Yiyu Wang , Jungang Xu , Yingfei Sun , Ben He

Image captioning implies automatically generating textual descriptions of images based only on the visual input. Although this has been an extensively addressed research topic in recent years, not many contributions have been made in the…

Computer Vision and Pattern Recognition · Computer Science 2021-02-09 Eva Cetinic

Image captioning is shown to be able to achieve a better performance by using scene graphs to represent the relations of objects in the image. The current captioning encoders generally use a Graph Convolutional Net (GCN) to represent the…

Computer Vision and Pattern Recognition · Computer Science 2022-07-18 Xuewen Yang , Yingru Liu , Xin Wang

Despite the substantial progress in recent years, the image captioning techniques are still far from being perfect.Sentences produced by existing methods, e.g. those based on RNNs, are often overly rigid and lacking in variability. This…

Computer Vision and Pattern Recognition · Computer Science 2017-08-14 Bo Dai , Sanja Fidler , Raquel Urtasun , Dahua Lin