Related papers: Pairwise Comparisons Rating Scale Paradox

Ranking by pairwise comparisons has shown improved reliability over ordinal classification. However, as the annotations of pairwise comparisons scale quadratically, this becomes less practical when the dataset is large. We propose a method…

There has been a flurry of research in recent years on notions of fairness in ranking and recommender systems, particularly on how to evaluate if a recommender allocates exposure equally across groups of relevant items (also known as…

In this study, a pairwise comparison matrix is generalized to the case when coefficients create Lie group $G$, non necessarily abelian. A necessary and sufficient criterion for pairwise comparisons matrices to be consistent is provided.…

Most decision-making models, including the pairwise comparison method, assume the decision-makers honesty. However, it is easy to imagine a situation where a decision-maker tries to manipulate the ranking results. This paper presents three…

Linear stochastic transitivity is a central assumption in paired comparison models that is rarely verified in practice. Empirical violations, however, are common and can substantially affect inference and ranking. We develop a class of…

In software testing, the large size of the input domain makes exhaustively testing the inputs a daunting and often impossible task. Pair-wise testing is a popular approach to combinatorial testing problems. This paper reviews Pair-wise…

Heuristic Rating Estimation (HRE) is a newly proposed method supporting decisions analysis based on the use of pairwise comparisons. It allows that the ranking values of some alternatives (herein referred to as concepts) are initially…

The pairwise comparisons method is a convenient tool used when the relative order of preferences among different concepts (alternatives) needs to be determined. There are several popular implementations of this method, including the…

We describe a seriation algorithm for ranking a set of items given pairwise comparisons between these items. Intuitively, the algorithm assigns similar rankings to items that compare similarly with all others. It does so by constructing a…

Learning contrastive representations from pairwise comparisons has achieved remarkable success in various fields, such as natural language processing, computer vision, and information retrieval. Collaborative filtering algorithms based on…

Jeffreys-Lindley paradox is a case where frequentist and Bayesian hypothesis testing methodologies contradict with each other. This has caused confusion among data analysts for selecting a methodology for their statistical inference tasks.…

Pairwise comparison matrices often exhibit inconsistency, therefore many indices have been suggested to measure their deviation from a consistent matrix. A set of axioms has been proposed recently that is required to be satisfied by any…

A classical proposal to derive weights from a pairwise comparison matrix is the right eigenvector. The literature has identified some potential weaknesses of this method in previous decades. This chapter discusses five of these issues.…

Listwise ranking losses have been widely studied in recommender systems. However, new paradigms of content consumption present new challenges for ranking methods. In this work we contribute an analysis of learning to rank for personalized…

It is suggested to insert into test matrix 1s for correct responses, 0s for response refusals, and negative corrective elements for incorrect responses. With the classical test theory approach test scores of examinees and items are…

Since there exist several completion methods to estimate the missing entries of pairwise comparison matrices, practitioners face a difficult task in choosing the best technique. Our paper contributes to this issue: we consider a special set…

We consider sequential or active ranking of a set of n items based on noisy pairwise comparisons. Items are ranked according to the probability that a given item beats a randomly chosen item, and ranking refers to partitioning the items…

Ordered response scales are ubiquitous in economics, but their interpretation rests on an untested assumption: that numerical labels reflect equal psychological intervals. The contribution of this paper is to provide a systematic assessment…

Quantifying the complexity of systems consisting of many interacting parts has been an important challenge in the field of complex systems in both abstract and applied contexts. One approach, the complexity profile, is a measure of the…

Score matching is an estimation procedure that has been developed for statistical models whose probability density function is known up to proportionality but whose normalizing constant is intractable, so that maximum likelihood is…