Related papers: Assessment and support of emerging research groups
By utilizing statistical methods to analyze bibliographic data, bibliometrics faces inherent limitations in identifying the most significant science and technology achievements and researchers. To overcome this challenge, we present an…
As AI systems continue to evolve, their rigorous evaluation becomes crucial for their development and deployment. Researchers have constructed various large-scale benchmarks to determine their capabilities, typically against a gold-standard…
An accurate and fair assessment of the efficiency and impact of scientific work is, despite a lot of recent research effort, still an open problem. The measurement of quality and success of individual scientists and research groups can be…
In many countries and at European level, research policy increasingly focuses on 'excellent' researchers. The concept of excellence however is complex and multidimensional. For individual scholars it involves talents for innovative…
The use of bibliometric indicators would simplify research assessments. The 2014 Research Excellence Framework (REF) is a peer review assessment of UK universities, whose results can be taken as benchmarks for bibliometric indicators. In…
Early identification of emergent topics is of eminent importance due to their potential impacts on society. There are many methods for detecting emerging terms and topics, all with advantages and drawbacks. However, there is no consensus…
Assigning qualified, unbiased and interested reviewers to paper submissions is vital for maintaining the integrity and quality of the academic publishing system and providing valuable reviews to authors. However, matching thousands of…
In this paper an analysis of the presence and possibilities of altmetrics for bibliometric and performance analysis is carried out. Using the web based tool Impact Story, we have collected metrics for 20,000 random publications from the Web…
AI agents hold the potential to revolutionize scientific productivity by automating literature reviews, replicating experiments, analyzing data, and even proposing new directions of inquiry; indeed, there are now many such agents, ranging…
The phenomenon of rankings is intimately related with the government interest in fiscalizing the research outputs of universities. New forms of managerialism have been introduced into the higher education system, leading to an increasing…
Allocation of research funding, as well as promotion and tenure decisions, are increasingly made using indicators and impact factors drawn from citations to published work. A debate among scientometricians about proper normalization of…
Although not explicitly declared, most research rankings of countries and institutions are supposed to reveal their contribution to the advancement of knowledge. However, such advances are based on very highly cited publications with very…
Social and technical trends have significantly changed methods for evaluating and disseminating computing research. Traditional venues for reviewing and publishing, such as conferences and journals, worked effectively in the past. Recently,…
Citation metrics are the best tools for research assessments. However, current metrics may be misleading in research systems that pursue simultaneously different goals, such as the advance of science and incremental innovations, because…
The peer review system has been traditionally challenged due to its many limitations especially for allocating funding. Bibliometric indicators may well present themselves as a complement. Objective: We analyze the relationship between…
The academic evaluation of the publication record of researchers is relevant for identifying talented candidates for promotion and funding. A key tool for this is the use of the indexes provided by Web of Science and SCOPUS, costly…
Bibliometrics plays an increasingly important role in research evaluation. However, no gold standard exists for a set of reliable and valid (field-normalized) impact indicators in research evaluation. This opinion paper recommends that…
Rigorous and reproducible evaluation is critical for assessing the state of the art and for guiding scientific advances in Artificial Intelligence. Evaluation is challenging in practice due to several reasons, including benchmark…
In recent years, several Scientometrics and Bibliometrics indicators were proposed to evaluate the scientific impact of individuals, institutions, colleges, universities and research teams. The h-index gives a major breakthrough in the…
Novel reinforcement learning algorithms, or improvements on existing ones, are commonly justified by evaluating their performance on benchmark environments and are compared to an ever-changing set of standard algorithms. However, despite…