Related papers: Past performance, peer review, and project selecti…
Modern machine learning and computer science conferences are experiencing a surge in the number of submissions that challenges the quality of peer review as the number of competent reviewers is growing at a much slower rate. To curb this…
Many different measures are used to assess academic research excellence and these are subject to ongoing discussion and debate within the scientometric, university-management and policy-making communities internationally. One topic of…
An increasing number of nations allocate public funds to research institutions on the basis of rankings obtained from national evaluation exercises. Therefore, in non-competitive higher education systems where top scientists are dispersed…
It is well known that women are underrepresented in the academic systems of many countries. Gender discrimination is one of the factors that could contribute to this phenomenon. This study considers a recent national academic recruitment…
After completing their undergraduate studies, many computer science (CS) students apply for competitive graduate programs in North America. Their long-term goal is often to be hired by one of the big five tech companies or to become a…
Sir, We write as senior scientists about a problem vital to the scientific enterprise and prosperity. Nowadays, funding is a lengthy and complex business. First, universities themselves must approve all proposals for submission. Funding…
When funding public goods, resources are often allocated via mechanisms that resemble contests, especially in the case of research grants. A common critique of these contests is that they induce ``too much'' effort from participants. This…
One of the virtues of peer review is that it provides a self-regulating selection mechanism for scientific work, papers and projects. Peer review as a selection mechanism is hard to evaluate in terms of its efficiency. Serious efforts to…
The selection of referees for evaluation of research projects under competitive financing appears particularly critical: the greater the competence of the referee concerning the core topic, the more their judgment is trustworthy and the…
Citations play an important role in researchers' careers as a key factor in evaluation of scientific impact. Many anecdotes advice authors to exploit this fact and cite prospective reviewers to try obtaining a more positive evaluation for…
In most countries, basic research is supported by research councils that select, after peer review, the individuals or teams that are to receive funding. Unfortunately, the number of grants these research councils can allocate is not…
In this article we wish to provide a common set of best practice approaches that should be considered for all effective research grant proposal reviews. The federal government performs a critical role in American competitiveness and…
With the industry trend of shifting from a traditional hierarchical approach to flatter management structure, crowdsourced performance assessment gained mainstream popularity. One fundamental challenge of crowdsourced performance assessment…
In national research assessment exercises that take the peer review approach, research organizations are evaluated on the basis of a subset of their scientific production. The dimension of the subset varies from nation to nation but is…
This study examines a basic assumption of peer review, namely, the idea that there is a consensus on evaluation criteria among peers, which is a necessary condition for the reliability of peer judgements. Empirical evidence indicating that…
The IPCC started at a time when climate policy was an aspiration for the future. The research assessed in the early IPCC reports was necessarily about potential climate policies, always stylized and often optimized. The IPCC has continued…
The conference peer review process involves three constituencies with different objectives: authors want their papers accepted at prestigious venues (and quickly), conferences want to present a program with many high-quality and few…
The advancement of various fields of science depends on the actions of individual scientists via the peer review process. The referees' work patterns and stochastic nature of decision making both relate to the particular features of…
The design of research grants has been hypothesized to be a useful tool for influencing researchers and their science. We test this by conducting two thought experiments in a nationally representative survey of academic researchers. First,…
One of the critical issues in national research assessment exercises concerns the choice of whether to evaluate the entire scientific portfolio of the institutions or a subset composed of the best products. Under the second option, the…