相关论文: Past performance, peer review, and project selecti…
In many countries financial service providers have to elicit their customers risk preferences, when offering products and services. For instance, in the Netherlands pension funds will be legally obliged to factor in their clients risk…
Personal connections between creators and evaluators of scientific works are ubiquitous, and the possibility of bias ever-present. Although connections have been shown to bias prospective judgments of (uncertain) future performance, it is…
Interest in selection relaying is growing. The recent developments in this area have largely focused on information theoretic analyses such as outage performance. Some of these analyses are accurate only at high SNR regimes. In this paper…
Researchers have pinpointed recognition from others as one of the most important dimensions of students' science and engineering identity. Studies, however, have found gender biases in students' recognition of their peers, with inconsistent…
The National Science Foundation (NSF) will be experimenting with a new distributed approach to reviewing proposals, whereby a group of principal investigators (PIs) or proposers in a subfield act as reviewers for the proposals submitted by…
Peer assessment is a popular technique for a more fine-grained evaluation of individual students in group projects. Its effect on the evaluation is well studied. However, its effects on the learning abilities of students are often…
It is shown that under certain circumstances in particular for small datasets the recently proposed citation impact indicators I3(6PR) and R(6,k) behave inconsistently when additional papers or citations are taken into consideration. Three…
Research evaluation is usually governed by panels of peers. Procedural fairness refers to the principles that ensures decisions are made through a fair and transparent process. It requires that the composition of panels is fair. A fair…
As part of its program of 'Excellence in Research for Australia' (ERA), the Australian Research Council ranked journals into four categories (A*, A, B, C) in preparation for their performance evaluation of Australian universities. The…
In this work, we study how the relevance/quality and quantity of past data influence performance by analyzing a contextual Newsvendor problem, in which a decision-maker trades off between underage and overage costs under uncertain demand.…
Peer review is a core element of the scientific process, particularly in conference-centered fields such as ML and NLP. However, only few studies have evaluated its properties empirically. Aiming to fill this gap, we present a corpus that…
Criteria are an essential component of any procedure for assessing merit. Yet, little is known about the criteria peers use in assessing grant applications. In this systematic review we therefore identify and synthesize studies that examine…
Peer assessment has established itself as a critical pedagogical tool in academic settings, offering students timely, high-quality feedback to enhance learning outcomes. However, the efficacy of this approach depends on two factors: (1) the…
Peer review is the main mechanism by which the software engineering community assesses the quality of scientific results. However, the rapid growth of paper submissions in software engineering venues has outpaced the availability of…
Budgetary constraints force organizations to pursue only a subset of possible innovation projects. Identifying which subset is most promising is an error-prone exercise, and involving multiple decision makers may be prudent. This raises the…
Time spent performing public engagement is severely undervalued by research institutions around the globe. In this article we present one possible system that could be implemented to provide researchers with career recognition for…
Despite the increasing use of citation-based metrics for research evaluation purposes, we do not know yet which metrics best deliver on their promise to gauge the significance of a scientific paper or a patent. We assess 17 network-based…
A semi-supervised model of peer review is introduced that is intended to overcome the bias and incompleteness of traditional peer review. Traditional approaches are reliant on human biases, while consensus decision-making is constrained by…
An original cross sectional dataset referring to a medium sized Italian university is implemented in order to analyze the determinants of scientific research production at individual level. The dataset includes 942 permanent researchers of…
As one of the most pervasive applications of machine learning, recommender systems are playing an important role on assisting human decision making. The satisfaction of users and the interests of platforms are closely related to the quality…