Related papers: Pairwise Comparisons Rating Scale Paradox
We describe a new approach based on tropical optimization techniques to solve the problem of rating alternatives from pairwise comparison data. The problem is formulated to approximate, in the log-Chebyshev sense, pairwise comparison…
Pairwise comparison matrices have received substantial attention in a variety of applications, especially in rank aggregation, the task of flattening items into a one-dimensional (and thus transitive) ranking. However, non-transitive…
When applied to contingency tables, dual scaling and correspondence are mathematically equivalent methods. For the analysis of rating data, however, the methods differ. To a large extent this is due to differences in preprocessing of the…
We present pairwise fairness metrics for ranking models and regression models that form analogues of statistical fairness notions such as equal opportunity, equal accuracy, and statistical parity. Our pairwise formulation supports both…
This paper examines the problem of ranking a collection of objects using pairwise comparisons (rankings of two objects). In general, the ranking of $n$ objects can be identified by standard sorting methods using $n log_2 n$ pairwise…
Comparison studies in methodological research are intended to compare methods in an evidence-based manner to help data analysts select a suitable method for their application. To provide trustworthy evidence, they must be carefully…
Pairwise comparisons are a well-known method for modelling of the subjective preferences of a decision maker. A popular implementation of the method is based on solving an eigenvalue problem for M - the matrix of pairwise comparisons. This…
We consider problems of rating alternatives based on their pairwise comparison under various assumptions, including constraints on the final scores of alternatives. The problems are formulated in the framework of tropical mathematics to…
Incomplete pairwise comparison matrices contain some missing judgements. A natural approach to estimate these values is provided by minimising a reasonable measure of inconsistency after unknown entries are replaced by variables. Two widely…
Ratings are frequently used to evaluate and compare subjects in various applications, from education to healthcare, because ratings provide succinct yet credible measures for comparing subjects. However, when multiple rating lists are…
Pairwise comparisons between alternatives are a well-known method for measuring preferences of a decision-maker. Since these often do not exhibit consistency, a number of inconsistency indices has been introduced in order to measure the…
The prevailing approach for training and evaluating paraphrase identification models is constructed as a binary classification problem: the model is given a pair of sentences, and is judged by how accurately it classifies pairs as either…
Pairwise comparisons between alternatives are a well-established tool to decompose decision problems into smaller and more easily tractable sub-problems. However, due to our limited rationality, the subjective preferences expressed by…
Comparative Judgement is an assessment method where item ratings are estimated based on rankings of subsets of the items. These rankings are typically pairwise, with ratings taken to be the estimated parameters from fitting a Bradley-Terry…
This study presents an abelian group approach to analyzing inconsistency in pairwise comparisons. However, it wrongly assumes that an inconsistency indicator can take values in any abelian linearly ordered group. The followup publication…
The pairwise comparisons method is a convenient tool used when the relative order among different concepts (alternatives) needs to be determined. One popular implementation of the method is based on solving an eigenvalue problem for the…
Propensity Score Matching (PSM) stands as a widely embraced method in comparative effectiveness research. PSM crafts matched datasets, mimicking some attributes of randomized designs, from observational data. In a valid PSM design where all…
We study the problem of ranking from crowdsourced pairwise comparisons. Answers to pairwise tasks are known to be affected by the position of items on the screen, however, previous models for aggregation of pairwise comparisons do not focus…
Incomplete pairwise comparison matrices are increasingly employed to save resources and reduce cognitive load by collecting only a subset of all possible pairwise comparisons. We present their graph representation and some completion…
A common problem in machine learning is to rank a set of n items based on pairwise comparisons. Here ranking refers to partitioning the items into sets of pre-specified sizes according to their scores, which includes identification of the…