English
Related papers

Related papers: An Overview of Indian Language Datasets used for T…

200 papers

Measuring the semantic similarity between two sentences (or Semantic Textual Similarity - STS) is fundamental in many NLP applications. Despite the remarkable results in supervised settings with adequate labeling, little attention has been…

Computation and Language · Computer Science 2018-10-31 Xin Tang , Shanbo Cheng , Loc Do , Zhiyu Min , Feng Ji , Heng Yu , Ji Zhang , Haiqin Chen

With nearly 1.5 billion people and more than 120 major languages, India represents one of the most diverse regions in the world. As multilingual Vision-Language Models (VLMs) gain prominence, robust evaluation methodologies are essential to…

In this study, we tackle the challenge of limited labeled data for low-resource languages in ASR, focusing on Hindi. Specifically, we explore pseudo-labeling, by proposing a generic framework combining multiple ideas from existing works.…

Large language models (LLMs) excel in abstractive summarization tasks, delivering fluent and pertinent summaries. Recent advancements have extended their capabilities to handle long-input contexts, exceeding 100k tokens. However, in…

Computation and Language · Computer Science 2024-11-15 Mathieu Ravaut , Aixin Sun , Nancy F. Chen , Shafiq Joty

Scene text recognition in low-resource Indian languages is challenging because of complexities like multiple scripts, fonts, text size, and orientations. In this work, we investigate the power of transfer learning for all the layers of deep…

Computer Vision and Pattern Recognition · Computer Science 2022-01-11 Sanjana Gunna , Rohit Saluja , C. V. Jawahar

Multilingual large language models (LLMs) often demonstrate a performance gap between English and non-English languages, particularly in low-resource settings. Aligning these models to low-resource languages is essential yet challenging due…

Computation and Language · Computer Science 2025-10-16 Rakesh Paul , Anusha Kamath , Kanishk Singla , Raviraj Joshi , Utkarsh Vaidya , Sanjay Singh Chauhan , Niranjan Wartikar

The distribution of subpopulations is an important property hidden within a dataset. Uncovering and analyzing the subpopulation distribution within datasets provides a comprehensive understanding of the datasets, standing as a powerful tool…

Computer Vision and Pattern Recognition · Computer Science 2024-07-25 Yulin Luo , Ruichuan An , Bocheng Zou , Yiming Tang , Jiaming Liu , Shanghang Zhang

We present iNLTK, an open-source NLP library consisting of pre-trained language models and out-of-the-box support for Data Augmentation, Textual Similarity, Sentence Embeddings, Word Embeddings, Tokenization and Text Generation in 13 Indic…

Computation and Language · Computer Science 2021-02-15 Gaurav Arora

Language models (LMs) are capable of acquiring elements of human-like syntactic knowledge. Targeted syntactic evaluation tests have been employed to measure how well they form generalizations about syntactic phenomena in high-resource…

Computation and Language · Computer Science 2024-12-13 Daria Kryvosheieva , Roger Levy

Relation classification (sometimes called 'extraction') requires trustworthy datasets for fine-tuning large language models, as well as for evaluation. Data collection is challenging for Indian languages, because they are syntactically and…

Computation and Language · Computer Science 2021-10-20 Arijit Nag , Bidisha Samanta , Animesh Mukherjee , Niloy Ganguly , Soumen Chakrabarti

The advancements in the Large Language Model (LLM) have helped in solving several problems related to language processing. Most of the researches have focused on the English language only, because of its popularity and abundance on the…

Computation and Language · Computer Science 2024-12-31 Sanjay Chouhan , Shubha Brata Nath , Aparajita Dutta

Low-resource languages (LRLs) lack sufficient linguistic resources and are underrepresented in benchmark datasets, resulting in persistently lower translation quality than high-resource languages, especially in privacy-sensitive and…

Computation and Language · Computer Science 2025-08-25 Yewei Song , Lujun Li , Cedric Lothritz , Saad Ezzini , Lama Sleem , Niccolo Gentile , Radu State , Tegawendé F. Bissyandé , Jacques Klein

Recent methods in speech and language technology pretrain very LARGE models which are fine-tuned for specific tasks. However, the benefits of such LARGE models are often limited to a few resource rich languages of the world. In this work,…

Realignment is a promising strategy to improve cross-lingual transfer in multilingual language models. However, empirical results are mixed and often unreliable, particularly for typologically distant or low-resource languages (LRLs)…

Computation and Language · Computer Science 2025-11-11 Quang Phuoc Nguyen , David Anugraha , Felix Gaschi , Jun Bin Cheng , En-Shiun Annie Lee

In the rapidly evolving digital era, there is an increasing demand for concise information as individuals seek to distil key insights from various sources. Recent attention from researchers on Multi-document Summarisation (MDS) has resulted…

Computation and Language · Computer Science 2024-09-19 Kushan Hewapathirana , Nisansa de Silva , C. D. Athuraliya

We present the first parallel dataset for English-Tulu translation. Tulu, classified within the South Dravidian linguistic family branch, is predominantly spoken by approximately 2.5 million individuals in southwestern India. Our dataset is…

Computation and Language · Computer Science 2024-03-29 Manu Narayanan , Noëmi Aepli

Unstructured text in medical notes and dialogues contains rich information. Recent advancements in Large Language Models (LLMs) have demonstrated superior performance in question answering and summarization tasks on unstructured text data,…

Computation and Language · Computer Science 2024-05-31 Yuhao Chen , Zhimu Wang , Bo Wen , Farhana Zulkernine

Text summarization research has undergone several significant transformations with the advent of deep neural networks, pre-trained language models (PLMs), and recent large language models (LLMs). This survey thus provides a comprehensive…

Computation and Language · Computer Science 2024-06-18 Haopeng Zhang , Philip S. Yu , Jiawei Zhang

Text summarization has been a crucial problem in natural language processing (NLP) for several decades. It aims to condense lengthy documents into shorter versions while retaining the most critical information. Various methods have been…

Computation and Language · Computer Science 2023-02-17 Xianjun Yang , Yan Li , Xinlu Zhang , Haifeng Chen , Wei Cheng

Archive collections are nowadays mostly available through search engines interfaces, which allow a user to retrieve documents by issuing queries. The study of these collections may be, however, impaired by some aspects of search engines,…

Computation and Language · Computer Science 2023-02-01 Nicolas Gutehrlé , Antoine Doucet , Adam Jatowt