Related papers: Past performance, peer review, and project selecti…

Innovative ideas are often situated where disciplines meet, and socio-economic problems generally require contributions from several disciplines. Ways to stimulate interdisciplinary research collaborations are therefore an increasing point…

The use of outcome control modes of research evaluation exercises is ever more frequent. They are conceived as tools to stimulate increased levels of research productivity, and to guide choices in allocating components of government…

The search for and management of external funding now occupies much valuable researcher time. Whilst funding is essential for some types of research and beneficial for others, it may also constrain academic choice and creativity. Thus, it…

The work applies the funnel plot methodology to measure and visualize uncertainty in the research performance of Italian universities in the science disciplines. The performance assessment is carried out at both discipline and overall…

Peer review by experts is central to the evaluation of grant proposals, but little is known about how gender and disciplinary differences shape the content and tone of grant peer review reports. We analyzed 39,280 review reports submitted…

Nations can be distinguished in terms of whether domestic or international research is cited. We analyzed the research output in natural sciences of three leading European research economies (Germany, the Netherlands, and the UK) and ask…

Previous work has identified that recognition from others is an important predictor of students' participation, persistence, and career intentions in physics. However, research has also found a gender bias in peer recognition in which…

In this work we ask whether and to what extent applying a predictor of publications' impact better than early citations, has an effect on the assessment of research performance of individual scientists. Specifically, we measure the total…

Recent and influential critiques of standardized testing have noted the existence of non-trivial numbers of successful scientists who received low scores on the GRE. Here we use computer simulations to show that the prevalence of such…

Context: Pre-publication peer review of scientific articles is considered a key element of the research process in software engineering, yet it is often perceived as not to work fully well. Objective: We aim at understanding the perceptions…

Research projects are primarily collaborative in nature through internal and external partnerships, but what role does funding play in their formation? Here, we examined over 43,000 funded projects in the past three decades, enabling us to…

Mainstream machine learning conferences have seen a dramatic increase in the number of participants, along with a growing range of perspectives, in recent years. Members of the machine learning community are likely to overhear allegations…

We analysed a dataset of scientific manuscripts that were submitted to various conferences in artificial intelligence. We performed a combination of semantic, lexical and psycholinguistic analyses of the full text of the manuscripts and…

The number of scientific publications is constantly rising, increasing the strain on the review process. The number of submissions is actually higher, as each manuscript is often reviewed several times before publication. To face the deluge…

Many biological, psychological and economic experiments have been designed where an organism or individual must choose between two options that have the same expected reward but differ in the variance of reward received. In this way,…

Understanding the reasons associated with successful proposals is of paramount importance to improve evaluation processes. In this context, we analyzed whether bibliometric features are able to predict the success of research grants. We…

When a new treatment is considered for use, whether a pharmaceutical drug or a search engine ranking algorithm, a typical question that arises is, will its performance exceed that of the current treatment? The conventional way to answer…

Are the sciences not advancing at an ever increasing speed? We contrast this popular perspective with the view that science funding may actually see diminishing returns, at least regarding established fields. In order to stimulate a larger…

This study provides quantitative evidence on how the use of journal rankings can disadvantage interdisciplinary research in research evaluations. Using publication and citation data, it compares the degree of interdisciplinarity and the…

The development of state-of-the-art systems in different applied areas of machine learning (ML) is driven by benchmarks, which have shaped the paradigm of evaluating generalisation capabilities from multiple perspectives. Although the…