English
Related papers

Related papers: Comprehensive Algorithm Portfolio Evaluation using…

200 papers

This article is a brief guide to the field of algorithmic information theory (AIT), its underlying philosophy, and the most important concepts. AIT arises by mixing information theory and computation theory to obtain an objective and…

Information Theory · Computer Science 2009-09-29 Marcus Hutter

In the past few years, cross-modal image-text retrieval (ITR) has experienced increased interest in the research community due to its excellent research value and broad real-world application. It is designed for the scenarios where the…

Information Retrieval · Computer Science 2022-11-21 Min Cao , Shiping Li , Juntao Li , Liqiang Nie , Min Zhang

The pursuit of Artificial General Intelligence (AGI) is a central goal in language model development, in which consciousness-like processing could serve as a key facilitator. While current language models are not conscious, they exhibit…

Artificial Intelligence · Computer Science 2026-02-02 Hamid Reza Akbari , Mohammad Hossein Sameti , Amir M. Mansourian , Mohammad Hossein Rohban , Hossein Sameti

Thousands of diverse benchmarks have been developed to measure the quality of large language models (LLMs). Yet prior work has demonstrated that LLM performance is often sufficiently explained by a small set of latent factors, or abilities.…

Computation and Language · Computer Science 2026-04-03 Michael Krumdick , Adam Wiemerslage , Seth Ebner , Charles Lovering , Chris Tanner

The Rasch model is one of the most fundamental models in \emph{item response theory} and has wide-ranging applications from education testing to recommendation systems. In a universe with $n$ users and $m$ items, the Rasch model assumes…

Machine Learning · Computer Science 2023-10-31 Duc Nguyen , Anderson Zhang

Inverse reinforcement learning (IRL) addresses the problem of recovering a task description given a demonstration of the optimal policy used to solve such a task. The optimal policy is usually provided by an expert or teacher, making IRL…

Machine Learning · Computer Science 2012-02-09 Héctor Ratia , Luis Montesano , Ruben Martinez-Cantin

This scientific paper propose a novel portfolio optimization model using an improved deep reinforcement learning algorithm. The objective function of the optimization model is the weighted sum of the expectation and value at risk(VaR) of…

Machine Learning · Computer Science 2022-08-30 Boyi Jin

Class incremental learning (CIL) algorithms aim to continually learn new object classes from incrementally arriving data while not forgetting past learned classes. The common evaluation protocol for CIL algorithms is to measure the average…

Machine Learning · Computer Science 2024-06-26 Sungmin Cha , Jihwan Kwak , Dongsub Shim , Hyunwoo Kim , Moontae Lee , Honglak Lee , Taesup Moon

Essays are considered a valuable mechanism for evaluating learning outcomes in writing. Textual cohesion is an essential characteristic of a text, as it facilitates the establishment of meaning between its parts. Automatically scoring…

Computation and Language · Computer Science 2025-07-14 Bruno Alexandre Rosa , Hilário Oliveira , Luiz Rodrigues , Eduardo Araujo Oliveira , Rafael Ferreira Mello

We introduce a novel eigenvalue algorithm for near-diagonal matrices inspired by Rayleigh-Schr\"odinger perturbation theory and termed Iterative Perturbative Theory (IPT). Contrary to standard eigenvalue algorithms, which are either…

Numerical Analysis · Mathematics 2022-11-18 Maseim Kenmoe , Ronald Kriemann , Matteo Smerlak , Anton S. Zadorin

Random testing (RT) is a black-box software testing technique that tests programs by generating random test inputs. It is a widely used technique for software quality assurance, but there has been much debate by practitioners concerning its…

Software Engineering · Computer Science 2019-10-01 Jinfu Chen , Hilary Ackah-Arthur , Chengying Mao , Patrick Kwaku Kudjo

Selecting the most suitable algorithm for a given problem instance remains a challenging task, particularly in online or dynamic environments where problem characteristics evolve over time. Relying solely on instantaneous performance…

Multiagent Systems · Computer Science 2026-05-26 Jayprakash S. Nair , Jimson Mathew , Shivashankar B. Nair

Within the educational context, students' assessment tests are routinely validated through Item Response Theory (IRT) models which assume unidimensionality and absence of Differential Item Functioning (DIF). In this paper, we investigate if…

Applications · Statistics 2012-12-04 Michela Gnaldi , Francesco Bartolucci , Silvia Bacci

The experimental evaluation of algorithms results in a large set of data which generally do not follow a normal distribution or are not heteroscedastic. Besides, some of its entries may be missing, due to the inability of an algorithm to…

Machine Learning · Computer Science 2019-08-16 Iago A Carvalho

Robustness is of central importance in machine learning and has given rise to the fields of domain generalization and invariant learning, which are concerned with improving performance on a test distribution distinct from but related to the…

Machine Learning · Computer Science 2020-12-03 Robert Adragna , Elliot Creager , David Madras , Richard Zemel

As Large Language Models (LLMs) grow increasingly adept at managing complex tasks, the evaluation set must keep pace with these advancements to ensure it remains sufficiently discriminative. Item Discrimination (ID) theory, which is widely…

Computation and Language · Computer Science 2024-10-08 Fan Lin , Shuyi Xie , Yong Dai , Wenlin Yao , Tianjiao Lang , Zishan Xu , Zhichao Hu , Xiao Xiao , Yuhong Liu , Yu Zhang

We present and evaluate new techniques for designing algorithm portfolios. In our view, the problem has both a scheduling aspect and a machine learning aspect. Prior work has largely addressed one of the two aspects in isolation. Building…

Artificial Intelligence · Computer Science 2012-06-18 Matthew Streeter , Stephen F. Smith

Deep reinforcement learning (DRL) has been applied in financial portfolio management to improve returns in changing market conditions. However, unlike most fields where DRL is widely used, the stock market is more volatile and dynamic as it…

Machine Learning · Computer Science 2025-02-12 Fengchen Gu , Angelos Stefanidis , Ángel García-Fernández , Jionglong Su , Huakang Li

In complex reinforcement learning (RL) problems, policies with similar rewards may have substantially different behaviors. It remains a fundamental challenge to optimize rewards while also discovering as many diverse strategies as possible,…

Machine Learning · Computer Science 2023-10-24 Wei Fu , Weihua Du , Jingwei Li , Sunli Chen , Jingzhao Zhang , Yi Wu

Individualized treatment regimes (ITRs) aim to improve clinical outcomes by assigning treatment based on patient-specific characteristics. However, existing methods often struggle with high-dimensional covariates, limiting accuracy,…

Machine Learning · Statistics 2026-01-13 Sungtaek Son , Eardi Lila , Kwun Chuen Gary Chan