Related papers: Research Power Ranking: Adapting the Elo System to…
Process supervision enhances the performance of large language models in reasoning tasks by providing feedback at each step of chain-of-thought reasoning. However, due to the lack of effective process supervision methods, even advanced…
Whereas the research program of the measurement of scientific communications emerged in a context where the delineations among academia, government, and industry were institutionalized, the systemic development of these relations during the…
We introduce Probabilistic Rank and Reward (PRR), a scalable probabilistic model for personalized slate recommendation. Our approach allows off-policy estimation of the reward in the scenario where the user interacts with at most one item…
Scientific contributions are a direct reflection of a research paper's value, illustrating its impact on existing theories or practices. Existing measurement methods assess contributions based on the authors' perceived or self-identified…
The last few years have seen the proliferation of measures that quantify the scientific output of researches. Yet, these measures focus on productivity, thus fostering the "publish or perish" paradigm. This article proposes a measure that…
The evaluation of mathematical results plays a central role in assessing researchers' contributions and shaping the direction of the field. Currently, such evaluations rely primarily on human judgment, whether through journal peer review or…
Evolution Strategy (ES) is a powerful black-box optimization technique based on the idea of natural evolution. In each of its iterations, a key step entails ranking candidate solutions based on some fitness score. For an ES method in…
I propose a new model to measure simultaneously academic and financial performances of scientific activities quantitatively. The tool is very simple and can be applied to any branches of science, while it is also adjustable to varying…
Large language models (LLMs) inevitably make mistakes when performing step-by-step mathematical reasoning. Process Reward Models (PRMs) have emerged as a promising solution by evaluating each reasoning step. However, existing PRMs typically…
We studied the research performance of 69 countries by considering two different types of new knowledge: incremental (normal) and fundamental (radical). In principle, these two types of new knowledge should be assessed at two very different…
We propose a new performance indicator to evaluate the productivity of research institutions by their disseminated scientific papers. The new quality measure includes two principle components: the normalized impact factor of the journal in…
Many environments assign several Elo ratings to the same agent: a chess player has classical, rapid, and blitz ratings; an online platform may rate by time control, mode, or format; an evaluator may rate performance across tasks or roles.…
Ranking athletes by their performance in competitions and tournaments is common in every popular sport and has significant benefits that contribute to both the organization and strategic aspects of competitions. Although rankings are…
A model is proposed for the creation and transmission of scientific knowledge, based on the network of citations among research articles. The model allows to assign to each article a nonnegative value for its creativity, i. e. its creation…
This paper presents KnoVo (Knowledge Evolution), an intelligent framework designed for quantifying and analyzing the evolution of research novelty in the scientific literature. Moving beyond traditional citation analysis, which primarily…
Percentiles are statistics pointing to the standing of a paper's citation impact relative to other papers in a given citation distribution. Percentile Ranks (PRs) often play an important role in evaluating the impact of scholars,…
We develop a method to estimate producers' productivity beliefs when output quantities and input prices are unobservable, and we use it to evaluate the market for science. Our model of researchers' labor supply shows how their willingness…
Evaluating and comparing the academic performance of a journal, a researcher or a single paper has long remained a critical, necessary but also controversial issue. Most of existing metrics invalidate comparison across different fields of…
Computational models can advance affective science by shedding light onto the interplay between cognition and emotion from an information processing point of view. We propose a computational model of emotion that integrates reinforcement…
A novel picture of the relative positions of countries in the world of science is offered through application of a two-dimensional mapping method which is based on quantity and quality indicators of the scientific production as…