English
Related papers

Related papers: Navigating the Pitfalls of Active Learning Evaluat…

200 papers

The goal of multi-label learning (MLL) is to associate a given instance with its relevant labels from a set of concepts. Previous works of MLL mainly focused on the setting where the concept set is assumed to be fixed, while many real-world…

Machine Learning · Computer Science 2021-10-01 Cheng-Yu Hsieh , Wei-I Lin , Miao Xu , Gang Niu , Hsuan-Tien Lin , Masashi Sugiyama

High-quality arguments are an essential part of decision-making. Automatically predicting the quality of an argument is a complex task that recently got much attention in argument mining. However, the annotation effort for this task is…

Machine Learning · Computer Science 2021-09-24 Nataliia Kees , Michael Fromm , Evgeniy Faerman , Thomas Seidl

Consider a sequential active learning problem where, at each round, an agent selects a batch of unlabeled data points, queries their labels and updates a binary classifier. While there exists a rich body of work on active learning in this…

Machine Learning · Computer Science 2020-05-26 Abbas Kazerouni , Qi Zhao , Jing Xie , Sandeep Tata , Marc Najork

Active Learning (AL) promises to reduce annotation cost by prioritizing informative samples, yet its reliability is undermined when labels are noisy or when the data distribution shifts. In practice, annotators make mistakes, rare…

Machine Learning · Computer Science 2025-10-14 Atharv Goel , Sharat Agarwal , Saket Anand , Chetan Arora

Behavioral models are the key enablers for behavioral analysis of Software Product Lines (SPL), including testing and model checking. Active model learning comes to the rescue when family behavioral models are non-existent or outdated. A…

Software Engineering · Computer Science 2022-03-11 Shaghayegh Tavassoli , Carlos Diego Nascimento Damasceno , Mohammad Reza Mousavi , Ramtin Khosravi

Large amounts of annotated data have become more important than ever, especially since the rise of deep learning techniques. However, manual annotations are costly. We propose a tool that enables researchers to create large, high-quality,…

Digital Libraries · Computer Science 2021-12-23 Franziska Weeber , Felix Hamborg , Karsten Donnay , Bela Gipp

The remarkable advancements in large language models (LLMs) have significantly enhanced the performance in few-shot learning settings. By using only a small number of labeled examples, referred to as demonstrations, LLMs can effectively…

Computation and Language · Computer Science 2023-11-23 Katerina Margatina , Timo Schick , Nikolaos Aletras , Jane Dwivedi-Yu

Labeled data are critical to modern machine learning applications, but obtaining labels can be expensive. To mitigate this cost, machine learning methods, such as transfer learning, semi-supervised learning and active learning, aim to be…

In many classification problems unlabelled data is abundant and a subset can be chosen for labelling. This defines the context of active learning (AL), where methods systematically select that subset, to improve a classifier by retraining.…

Machine Learning · Statistics 2014-07-31 Lewis P. G. Evans , Niall M. Adams , Christoforos Anagnostopoulos

In settings where only a budgeted amount of labeled data can be afforded, active learning seeks to devise query strategies for selecting the most informative data points to be labeled, aiming to enhance learning algorithms' efficiency and…

Machine Learning · Computer Science 2024-06-26 Valentin Margraf , Marcel Wever , Sandra Gilhuber , Gabriel Marques Tavares , Thomas Seidl , Eyke Hüllermeier

The availability of labelled data is one of the main limitations in machine learning. We can alleviate this using weak supervision: a framework that uses expert-defined rules $\boldsymbol{\lambda}$ to estimate probabilistic labels…

Machine Learning · Computer Science 2021-05-03 Samantha Biegel , Rafah El-Khatib , Luiz Otavio Vilas Boas Oliveira , Max Baak , Nanne Aben

Recent network traffic classification methods benefitfrom machine learning (ML) technology. However, there aremany challenges due to use of ML, such as: lack of high-qualityannotated datasets, data-drifts and other effects causing aging…

Networking and Internet Architecture · Computer Science 2022-11-16 Jaroslav Pešek , Dominik Soukup , Tomáš Čejka

Deep learning models for natural language processing rely heavily on high-quality labeled datasets. However, existing labeling approaches often struggle to balance label quality with labeling cost. To address this challenge, we propose…

Human-Computer Interaction · Computer Science 2026-02-17 Guozheng Li , Ao Wang , Shaoxiang Wang , Yu Zhang , Pengcheng Cao , Yang Bai , Chi Harold Liu

Active learning (AL) reduces labeling cost by querying informative samples, but in tabular settings its cold-start gains are often limited because uncertainty estimates are unreliable when models are trained on very few labels. Tabular…

Machine Learning · Computer Science 2026-03-31 Wilailuck Treerath , Fabrizio Pittorino

Pretrained on web-scale open data, VLMs offer powerful capabilities for solving downstream tasks after being adapted to task-specific labeled data. Yet, data labeling can be expensive and may demand domain expertise. Active Learning (AL)…

Computer Vision and Pattern Recognition · Computer Science 2025-06-03 Tong Wang , Jiaqi Wang , Shu Kong

Active learning aims to identify the most informative data from an unlabeled data pool that enables a model to reach the desired accuracy rapidly. This benefits especially deep neural networks which generally require a huge number of…

Computer Vision and Pattern Recognition · Computer Science 2023-03-28 Jihyo Kim , Jeonghyeon Kim , Sangheum Hwang

Exfiltration of data via email is a serious cybersecurity threat for many organizations. Detecting data exfiltration (anomaly) patterns typically requires labeling, most often done by a human annotator, to reduce the high number of false…

Machine Learning · Computer Science 2023-07-19 Jaturong Kongmanee , Mark Chignell , Khilan Jerath , Abhay Raman

Active learning (AL) on attributed graphs has received increasing attention with the prevalence of graph-structured data. Although AL has been widely studied for alleviating label sparsity issues with the conventional non-related data, how…

Machine Learning · Computer Science 2020-08-07 Yayong Li , Jie Yin , Ling Chen

Active learning (AL) aims to minimize labeling efforts for data-demanding deep neural networks (DNNs) by selecting the most representative data points for annotation. However, currently used methods are ill-equipped to deal with biased…

Computer Vision and Pattern Recognition · Computer Science 2020-03-03 Denis Gudovskiy , Alec Hodgkinson , Takuya Yamaguchi , Sotaro Tsukizawa

Most of the existing learning models, particularly deep neural networks, are reliant on large datasets whose hand-labeling is expensive and time demanding. A current trend is to make the learning of these models frugal and less dependent on…

Computer Vision and Pattern Recognition · Computer Science 2022-12-12 Sebastien Deschamps , Hichem Sahbi