Related papers: Pairwise Comparisons Rating Scale Paradox
In this paper, we consider large-scale ranking problems where one is given a set of (possibly non-redundant) pairwise comparisons and the underlying ranking explained by those comparisons is desired. We show that stochastic gradient descent…
A new trend in medicine is the use of algorithms to analyze big datasets, e.g. using everything your phone measures about you for diagnostics or monitoring. However, these algorithms are commonly compared against weak baselines, which may…
Learning to rank is an effective recommendation approach since its introduction around 2010. Famous algorithms such as Bayesian Personalized Ranking and Collaborative Less is More Filtering have left deep impact in both academia and…
The "paradox" arises in the Two Envelopes Paradox from the incorrect formulation of the argument. The infomation given is misused and therefore the results are incorrect for the question asked. The key is to be clear on what question we are…
We explore "omitted label contexts," in which training data is limited to a subset of the possible labels. This setting is standard among specialized human experts or specific, focused studies. By studying Simpson's paradox, we observe that…
The selection of the best classification algorithm for a given dataset is a very widespread problem, occuring each time one has to choose a classifier to solve a real-world problem. It is also a complex task with many important…
Ranking evaluation metrics are a fundamental element of design and improvement efforts in information retrieval. We observe that most popular metrics disregard information portrayed in the scores used to derive rankings, when available.…
Properties of weighted averages are studied for the general case that the individual measurements are subject to hidden correlations and have asymmetric statistical as well as systematic errors. Explicit expressions are derived for an…
Seven years ago, researchers proposed a postprocessing method to equalize the error rates of a model across different demographic groups. The work launched hundreds of papers purporting to improve over the postprocessing baseline. We…
Regularized pairwise ranking with Gaussian kernels is one of the cutting-edge learning algorithms. Despite a wide range of applications, a rigorous theoretical demonstration still lacks to support the performance of such ranking estimators.…
We examine three methods for ranking by pairwise comparison: Principal Eigenvector, HodgeRank and Tropical Eigenvector. It is shown that the choice of method can produce arbitrarily different rank order.To be precise, for any two of the…
Offline evaluations of recommender systems attempt to estimate users' satisfaction with recommendations using static data from prior user interactions. These evaluations provide researchers and developers with first approximations of the…
We study the performance of general dynamic matching models. This model is defined by a connected graph, where nodes represent the class of items and the edges the compatibilities between items. Items of different classes arrive one by one…
Meta-analysis is an important tool for combining results from multiple studies and has been widely used in evidence-based medicine for several decades. This paper reports, for the first time, an interesting and valuable paradox in…
Chess championships are often organised as a Swiss-system tournament, causing great challenges in ranking the participants due to the different strength of schedules and possible circular triads. The paper suggests that pairwise comparison…
Multiplicative logarithmic corrections to scaling are frequently encountered in the critical behavior of certain statistical-mechanical systems. Here, a Lee-Yang zero approach is used to systematically analyse the exponents of such…
We consider statistical procedures for hypothesis testing of real valued functionals of matched pairs with missing values. In order to improve the accuracy of existing methods, we propose a novel multiplication combination procedure.…
Incomplete pairwise comparison matrices offer a natural way of expressing preferences in decision making processes. Although ordinal information is crucial, there is a bias in the literature: cardinal models dominate. Ordinal models usually…
Many applications collect a large number of time series, for example, the financial data of companies quoted in a stock exchange, the health care data of all patients that visit the emergency room of a hospital, or the temperature sequences…
Many real-world classification problems are significantly class-imbalanced to detriment of the class of interest. The standard set of proper evaluation metrics is well-known but the usual assumption is that the test dataset imbalance equals…