相关论文: Past performance, peer review, and project selecti…

We use a controlled experiment to study how information acquisition impacts candidate evaluations. We provide evaluators with group-level information on performance and the opportunity to acquire additional, individual-level performance…

There is an evident and rapid trend towards the adoption of evaluation exercises for national research systems for purposes, among others, of improving allocative efficiency in public funding of individual institutions. However the desired…

In this work we investigate the determinants of professors' career advancement in Italian universities. From the analyses, it emerges that the fundamental determinant of an academic candidate's success is not scientific merit, but rather…

In a previous article of ours, we explained the reasons why the MNCS and all similar per-publication citation indicators should not be used to measure research performance, whereas efficiency indicators (output to input) such as the FSS are…

Peer review is an integral component of contemporary science. While peer review focuses attention on promising and interesting science, it also encourages scientists to pursue some questions at the expense of others. Here, we use ideas from…

It is important to explore how scientists decide their research agenda and the corresponding consequences, as their decisions collectively shape contemporary science. There are studies focusing on the overall performance of individuals with…

There has been ample demonstration that bibliometrics is superior to peer-review for national research assessment exercises in the hard sciences. In this paper we examine the Italian case, taking the 2001-2003 university performance…

The recommendation system of the competitive grants to university researchers by using the Grants-in-Aid for Scientific Research (KAKEN) keywords has been developed. The system can determine the recommendation order of researchers to each…

Public agencies like the U.S. National Science Foundation (NSF) and the National Institutes of Health (NIH) award tens of billions of dollars in annual science funding. How can this money be distributed as efficiently as possible to best…

Competitive grant funding is associated with high costs and a potential bias to favor conservative research. This comment proposes integrating editorial preregistration, in the form of registered reports, into grant peer review processes as…

We are witnessing a rapid trend towards the adoption of exercises for evaluation of national research systems, generally based on the peer review approach. They respond to two main needs: stimulating higher efficiency in research activities…

A desirable goal of scientific management is to introduce, if it exists, a simple and reliable way to measure the scientific excellence of publicly-funded research institutions and universities to serve as a basis for their ranking and…

In this paper we revisit the 2014 NeurIPS experiment that examined inconsistency in conference peer review. We determine that 50\% of the variation in reviewer quality scores was subjective in origin. Further, with seven years passing since…

Analyzing a large data set of publications drawn from the most competitive journals in the natural and social sciences we show that research careers exhibit the broad distributions of individual achievement characteristic of systems in…

Purpose: Despite the importance of peer review for grant funding decisions, academics are often reluctant to conduct it. This can lead to long delays between submission and the final decision as well as the risk of substandard reviews from…

This case study of a typical U.S. particle physics experiment explores the issues of gender bias and how it affects the academic career advancement prospects of women in the field of physics beyond the postdoctoral level; we use public…

Most Performance-based Research Funding Systems (PRFS) draw on peer review and bibliometric indicators, two different methodologies which are sometimes combined. A common argument against the use of indicators in such research evaluation…

Background: Contemporary software development organizations lack diversity and the ratios of women in Free and open-source software (FOSS) communities are even lower than the industry average. Although the results of recent studies hint the…

While ancient scientists often had patrons to fund their work, peer review of proposals for the allocation of resources is a foundation of modern science. A very common method is that proposals are evaluated by a small panel of experts (due…

National research evaluation exercises provide a comparative measure of research performance of the nation's institutions, and as such represent a tool for stimulating research productivity, particularly if the results are used to inform…