Related papers: An ordinal measure of interrater absolute agreemen…
Multiple raters are often needed to be used interchangeably in practice for measurement or evaluation. Assessing agreement among these multiple raters via agreement indices are necessary before their participation. While the intuitively…
Specially customised Entropies are widely applied in measuring the degree of uncertainties existing in the frame of discernment. However, all of these entropies regard the frame as a whole that has already been determined which dose not…
Annotation reproducibility and accuracy rely on good consistency within annotators. We propose a novel method for measuring within annotator consistency or annotator Intraobserver Agreement (IA). The proposed approach is based on…
In this study we present a metric of consensus for Likert scales. The measure gives the level of agreement as the percentage of consensus among respondents. The proposed framework allows to design a positional indicator that gives the…
Ordinal classification problems, where labels exhibit a natural order, are prevalent in high-stakes fields such as medicine and finance. Accurate uncertainty quantification, including the decomposition into aleatoric (inherent variability)…
Agreement between measurement methods is commonly investigated by a so-called Bland-Altman plot showing if the difference is independent of the size of the measurement. However, such analysis assumes that both methods have the same…
In Ordinal Classification tasks, items have to be assigned to classes that have a relative ordering, such as positive, neutral, negative in sentiment analysis. Remarkably, the most popular evaluation metrics for ordinal classification tasks…
Measurement of the interrater agreement (IRA) is critical in various disciplines. To correct for potential confounding chance agreement in IRA, Cohen's kappa and many other methods have been proposed. However, owing to the varied strategies…
In this paper we propose a multivariate ordinal regression model which allows the joint modeling of three-dimensional panel data containing both repeated and multiple measurements for a collection of subjects. This is achieved by a…
This paper presents a new estimator of the intercept of a linear regression model in cases where the outcome varaible is observed subject to a selection rule. The intercept is often in this context of inherent interest; for example, in a…
We propose an a posteriori error estimator for a sparse optimal control problem: the control variable lies in the space of regular Borel measures. We consider a solution technique that relies on the discretization of the control variable as…
We consider the notions of agreement, diversity, and polarization in ordinal elections (that is, in elections where voters rank the candidates). While (computational) social choice offers good measures of agreement between the voters, such…
Measuring the degree of spatial spreading of a sample can be of great interest when sampling from a spatial population. The commonly used spatial balance index by Grafstr\"om et al. (2012) is particularly effective in comparing the level of…
In several observational contexts where different raters evaluate a set of items, it is common to assume that all raters draw their scores from the same underlying distribution. However, a plenty of scientific works have evidenced the…
This paper introduces a novel approach to assess model performance for predictive models characterized by an ordinal target variable in order to satisfy the lack of suitable tools in this framework. Our methodological proposal is a new…
The paper presents a construction of a quantitative measure of variability for parameter estimates in the data fitting problem under interval uncertainty. It shows the degree of variability and ambiguity of the estimate, and the need for…
To compare different forecasting methods on demand series we require an error measure. Many error measures have been proposed, but when demand is intermittent some become inapplicable, some give counter-intuitive results, and there is no…
We construct and analyze an estimator of association between random variables based on their similarity in both direction and magnitude. Under special conditions, the proposed measure becomes a robust and consistent estimator of the linear…
To obtain reliable results of expertise, which usually use individual and group expert pairwise comparisons, it is important to summarize (aggregate) expert estimates provided that they are sufficiently consistent. There are several ways to…
While sensitivity analysis improves the transparency and reliability of mathematical models, its uptake by modelers is still scarce. This is partially explained by its technical requirements, which may be hard to understand and implement by…