Related papers: Testing Differences Statistically with the Leiden …
Classification and clustering are both important topics in statistical learning. A natural question herein is whether predefined classes are really different from one another, or whether clusters are really there. Specifically, we may be…
Graded labels are ubiquitous in real-world learning-to-rank applications, especially in human rated relevance data. Traditional learning-to-rank techniques aim to optimize the ranked order of documents. They typically, however, ignore…
In an attempt to provide an answer to the increasing criticism against p-values and to bridge the gap between statistical inference and prediction modelling, we introduce the probability of improved prediction (PIP). In general, the PIP is…
This paper deals with testing the equality of $k$ ($k\ge 2$) distribution functions against possible stochastic ordering among them. Two classes of rank tests are proposed for this testing problem. The statistics of the tests under study…
Uplift, essentially being the difference between two probabilities, is a central number in marketing performance measurement. A frequent question in applications is whether the uplifts of two campaigns are significantly different. In this…
Journal Impact Factors (IFs) can be considered historically as the first attempt to normalize citation distributions by using averages over two years. However, it has been recognized that citation distributions vary among fields of science…
Developing state-of-the-art approaches for specific tasks is a major driving force in our research community. Depending on the prestige of the task, publishing it can come along with a lot of visibility. The question arises how reliable are…
Although not explicitly declared, most research rankings of countries and institutions are supposed to reveal their contribution to the advancement of knowledge. However, such advances are based on very highly cited publications with very…
We tested Rousseau's (in press) recent proposal to define percentile classes in the case of the Integrated Impact Indicator (I3) so that the largest number in a set always belongs to the highest (100th) percentile rank class. In the case a…
According to current research in bibliometrics, percentiles (or percentile rank classes) are the most suitable method for normalising the citation counts of individual publications in terms of the subject area, the document type and the…
The new excellence indicator in the World Report of the SCImago Institutions Rankings (SIR) makes it possible to test differences in the ranking in terms of statistical significance. For example, at the 17th position of these rankings, UCLA…
We investigated the role of universities in the prosperity of cities and regions. Performance characteristics of universities are derived from the Leiden Ranking 2020. The socioeconomic strength of a city is determined with the urban…
World University rankings have become well-established tools that students, university managers and policy makers read and use. Each ranking claims to have a unique methodology capable of measuring the 'quality' of universities. The purpose…
We consider an extension of the setting of label ranking, in which the learner is allowed to make predictions in the form of partial instead of total orders. Predictions of that kind are interpreted as a partial abstention: If the learner…
For better or for worse, rankings of institutions, such as universities, schools and hospitals, play an important role today in conveying information about relative performance. They inform policy decisions and budgets, and are often…
Comparing the top $k$ elements between two or more ranked results is a common task in many contexts and settings. A few measures have been proposed to compare top $k$ lists with attractive mathematical properties, but they face a number of…
The Probability Ranking Principle states that the document set with the highest values of probability of relevance optimizes information retrieval effectiveness given the probabilities are estimated as accurately as possible. The key point…
In a recent paper entitled "Inconsistencies of Recently Proposed Citation Impact Indicators and how to Avoid Them," Schreiber (2012, at arXiv:1202.3861) proposed (i) a method to assess tied ranks consistently and (ii) fractional attribution…
Statistical significance testing plays an important role when drawing conclusions from experimental results in NLP papers. Particularly, it is a valuable tool when one would like to establish the superiority of one algorithm over another.…
Every year, over one million EU students choose a secondary school track based on teacher recommendations, yet little evidence shows this yields optimal assignments. Using Dutch data, we examine whether access to standardized test scores…