Related papers: Cross-benchmarking for performance evaluation: loo…

As frontier artificial intelligence (AI) models rapidly advance, benchmarks are integral to comparing different models and measuring their progress in different task-specific domains. However, there is a lack of guidance on when and how…

Cross-market recommendation aims to recommend products to users in a resource-scarce target market by leveraging user behaviors from similar rich-resource markets, which is crucial for E-commerce companies but receives less research…

Commonly, AI or machine learning (ML) models are evaluated on benchmark datasets. This practice supports innovative methodological research, but benchmark performance can be poorly correlated with performance in real-world applications -- a…

Method comparisons are essential to provide recommendations and guidance for applied researchers, who often have to choose from a plethora of available approaches. While many comparisons exist in the literature, these are often not neutral…

There is a growing need for empirical benchmarks that support researchers and practitioners in selecting the best machine learning technique for given prediction tasks. In this paper, we consider the next event prediction task in business…

Deep learning (DL) and machine learning (ML) models have shown promise in drug response prediction (DRP), yet their ability to generalize across datasets remains an open question, raising concerns about their real-world applicability. Due…

This paper proposes a new method to evaluate Decision Making Units (DMUs) under uncertainty using fuzzy Data Envelopment Analysis (DEA). In the proposed multi-objective nonlinear programming methodology both the objective functions and the…

In this article, the concepts of technical efficiency, efficiency, effectiveness and productivity are illustrated. It is discussed that when firms are not homogenous, the situation is the same as when each factor has a different unit of…

Data warehouse architectural choices and optimization techniques are critical to decision support query performance. To facilitate these choices, the performance of the designed data warehouse must be assessed, usually with benchmarks.…

In data envelopment analysis (DEA), the concept of efficiency is examined in either Farrell (DEA) or Pareto senses. In either of these senses, the efficiency status of a decision making unit (DMU) is classified as either weak or strong. It…

It is well established that multiple reference sets may occur for a decision making unit (DMU) in the non-radial DEA (data envelopment analysis) setting. As our first contribution, we differentiate between three types of reference set.…

Reinforcement learning has recently experienced increased prominence in the machine learning community. There are many approaches to solving reinforcement learning problems with new techniques developed constantly. When solving problems…

The increasing attention on deep learning has tremendously spurred the design of intelligence processing hardware. The variety of emerging intelligence processors requires standard benchmarks for fair comparison and system optimization (in…

Identification of the reference set for each decision making unit (DMU) is a main concern in the data envelopment analysis (DEA). All of the methods developed to date have been focused on finding the furthest reference DMUs. In this paper,…

We present in this paper a new benchmark for evaluating the performances of data warehouses. Benchmarking is useful either to system users for comparing the performances of different systems, or to system engineers for testing the effect of…

Evaluating the efficiency of organizations and branches within an organization is a challenging issue for managers. Evaluation criteria allow organizations to rank their internal units, identify their position concerning their competitors,…

Dynamic multi-objective optimization (DMOO) has recently attracted increasing interest from both academic researchers and engineering practitioners, as numerous real-world applications that evolve over time can be naturally formulated as…

Transferability scores aim to quantify how well a model trained on one domain generalizes to a target domain. Despite numerous methods proposed for measuring transferability, their reliability and practical usefulness remain inconclusive,…

Existing multi-criteria decision-making (MCDM) methods often face challenges when evaluating a large number of alternatives, leading to skewed results in selecting the optimal choice. Similarly, conventional efficiency analysis (EA)…

Data Envelopment Analysis (DEA) is a nonparametric, data driven technique used to perform relative performance analysis among a group of comparable decision making units (DMUs). Efficiency is assessed by comparing input and output data for…