Related papers: The first Italian research assessment exercise: a …
We compare estimates for past institutional research performances coming from two bibliometric indicators to the results of the UK's Research Assessment Exercise which last took place in 2008. We demonstrate that a version of the…
This literature review identifies indicators that associate with higher impact or higher quality research from article text (e.g., titles, abstracts, lengths, cited references and readability) or metadata (e.g., the number of authors,…
Peer reviewing is a central component in the scientific publishing process. We present the first public dataset of scientific peer reviews available for research purposes (PeerRead v1) providing an opportunity to study this important…
This book critically analyses the value of citation data, altmetrics, and artificial intelligence to support the research evaluation of articles, scholars, departments, universities, countries, and funders. It introduces and discusses…
The search engine evaluation research has quite a lot metrics available to it. Only recently, the question of the significance of individual metrics started being raised, as these metrics' correlations to real-world user experiences or…
In a university, research assessments are organized at different policy levels (faculties, research council) in different contexts (funding, council membership, personnel evaluations). Each evaluation requires its own focus and methodology.…
Accurate measurement of research productivity should take account of both the number of co-authors of every scientific work and of the different contributions of the individuals. For researchers in the life sciences, common practice is to…
Unlike competitive higher education systems, non-competitive systems show relatively uniform distributions of top professors and low performers among universities. In this study, we examine the impact of unproductive and top faculty members…
The evaluation of the quality of research at a national level has become increasingly common. The UK has been at the forefront of this trend having undertaken many assessments since 1986, the latest being the Research Excellence Framework…
For improving the performance and effectiveness of peer review, a novel review system is proposed, based on analysis of peer review process for academic journals under a parallel model built via Monte Carlo method. The model can simulate…
We present a framework consisting of three approaches that can enhance meta-analyses: 1) scoping reviews (evidence map), 2) bibliometrics, and 3) alternative impact metrics. These three "enrichment" approaches facilitate the research…
Peer-review plays a critical role in the scientific writing and publication ecosystem. To assess the efficiency and efficacy of the reviewing process, one essential element is to understand and evaluate the reviews themselves. In this work,…
When a publication is cited it generates a benefit. Through the country affiliations of the citing authors, it is possible to work upwards, tracing the countries that benefit from results produced in a national research system. In this work…
Detecting emerging research topics is essential, not only for research agencies but also for individual researchers. Previous studies have created various bibliographic indicators for the identification of emerging research topics. However,…
The disambiguation of author names is an important and challenging task in bibliometrics. We propose an approach that relies on an external source of information for selecting and validating clusters of publications identified through an…
Peer assessment has established itself as a critical pedagogical tool in academic settings, offering students timely, high-quality feedback to enhance learning outcomes. However, the efficacy of this approach depends on two factors: (1) the…
Measuring scientific development is a difficult task. Different metrics have been put forward to evaluate scientific development; in this paper we explore a metric that uses the number of peer-reviewed, and when available non-peer-reviewed…
Bibliometrics offers a particular representation of science. Through bibliometric methods a bibliometrician will always highlight particular elements of publications, and through these elements operationalize particular representations of…
Large language models (LLMs) show increasing potential in education, yet benchmarks for non-English languages in specialized domains remain scarce. We introduce MedBench-IT, the first comprehensive benchmark for evaluating LLMs on Italian…
Attempts to understand the consequence of any individual scientist's activity within the long-term trajectory of science is one of the most difficult questions within the philosophy of science. Because scientific publications play such as…