Related papers: Scilit with the Integrated Impact Indicator Assess…
In this paper an analysis of the presence and possibilities of altmetrics for bibliometric and performance analysis is carried out. Using the web based tool Impact Story, we have collected metrics for 20,000 random publications from the Web…
An important issue in the field of academic measurement is how to evaluate academic influence scientifically and comprehensively, which can help government and research organizations better allocate academic resources and recruit…
In this contribution, we deal with seed-based information retrieval in networks of research publications. Using systematic reviews as a baseline, and publication data from the NIH Open Citation Collection, we compare the performance of the…
Citation analysis is one of the most frequently used methods in research evaluation. We are seeing significant growth in citation analysis through bibliometric metadata, primarily due to the availability of citation databases such as the…
Evaluating the quality of academic journal is becoming increasing important within the context of research performance evaluation. Traditionally, journals have been ranked by peer review lists such as that of the Association of Business…
This short paper introduces the u-index, a simple and objective metric to evaluate the impact and relevance of academic research output, as a possible alternative to widespread metrics such as the h-index or the i10-index. The proposed…
The rapid expansion of scholarly publications across diverse disciplines has made it increasingly difficult to systematically evaluate how research contributes to the United Nations Sustainable Development Goals (SDGs). Domain…
In this study, I employ a multifaceted comprehensive scientometric approach to explore the intellectual underpinnings of AI and ML in financial research by examining the publication patterns of articles, journals, authors, institutions, and…
The LLM-as-judge paradigm is increasingly being adopted for automated evaluation of model outputs. While LLM judges have shown promise on constrained evaluation tasks, closed source LLMs display critical shortcomings when deployed in real…
Information content (IC) based measures for finding semantic similarity is gaining preferences day by day. Semantics of concepts can be highly characterized by information theory. The conventional way for calculating IC is based on the…
With the rapid growth in the number of scientific publications, year after year, it is becoming increasingly difficult to identify quality authoritative work on a single topic. Though there is an availability of scientometric measures which…
As researchers increasingly adopt LLMs as writing assistants, generating high-quality research paper introductions remains both challenging and essential. We introduce Scientific Introduction Generation (SciIG), a task that evaluates LLMs'…
In this paper we present an exploratory research on quantifying the impact that data distribution has on the performance and evaluation of NLP models. We propose an automated framework that measures the data point distribution across 6…
Many journals post accepted articles online before they are formally published in an issue. Early citation impact evidence for these articles could be helpful for timely research evaluation and to identify potentially important articles…
This paper presents a statistical analysis of the relationship between three science indicators applied in earlier bibliometric studies, namely research leadership based on corresponding authorship, international collaboration using…
Applying graph-based approaches in deep learning receives more attention over time. This study presents statistical analysis on the use of graph-based approaches in deep learning and examines the scientific impact of the related articles.…
We propose SETI (Systematicity Evaluation of Textual Inference), a novel and comprehensive benchmark designed for evaluating pre-trained language models (PLMs) for their systematicity capabilities in the domain of textual inference.…
Systematic evaluations of publicly funded research typically employ a combination of bibliometrics and peer review, but it is not known whether the bibliometric component introduces biases. This article compares three alternative mechanisms…
In this work we ask whether and to what extent applying a predictor of publications' impact better than early citations, has an effect on the assessment of research performance of individual scientists. Specifically, we measure the total…
Although Large language Model (LLM)-powered information extraction (IE) systems have shown impressive capabilities, current fine-tuning paradigms face two major limitations: high training costs and difficulties in aligning with LLM…