Related papers: Journal Impact Factor and Peer Review Thoroughness…
We conducted a large-scale analysis of around 10,000 scientific articles, from the period 2007-2016, to study the bibliometric or formal aspects influencing citations. A transversal analysis was conducted disaggregating the articles into…
This study investigates a range of psychological, lexical, semantic, and readability features of book reviews to elucidate the factors underlying their perceived popularity. To this end, we conduct statistical analyses of various features,…
This study investigates the use of AI tools in academic writing through analysis of AI usage declarations in journals. Using a mixed-methods approach combining content analysis, statistical analysis, and text mining, this research analyzed…
Many factors could affect the number of citations to a paper. Citations have an important role in research policy and in measuring the excellence of research and researchers. This work is the first study in software engineering (SE) to…
The peer review process is often regarded as the gatekeeper of scientific integrity, yet increasing evidence suggests that it is not immune to bias. Although structural inequities in peer review have been widely debated, much less attention…
In recent years, increased stakeholder pressure to transition research to Open Access has led to many journals "flipping" from a toll access to an open access publishing model. Changing the publishing model can influence the decision of…
Artificial intelligence and social computing rely on hundreds of thousands of content reviewers to classify high volumes of harmful and forbidden content. Many workers report long-term, potentially irreversible psychological harm. This work…
In this paper we revisit the 2014 NeurIPS experiment that examined inconsistency in conference peer review. We determine that 50\% of the variation in reviewer quality scores was subjective in origin. Further, with seven years passing since…
Co-authored articles tend to be more cited in many academic fields, but is this because they tend to be higher quality or is it an audience effect: increased awareness through multiple author networks? We address this unknown with the…
Social media platforms have now emerged as an important medium for wider dissemination of research articles; with authors, readers and publishers creating different kinds of social media activity about the article. Some research studies…
Scholars present their new research at seminars and conferences, and send drafts to peers, hoping to receive comments and suggestions that will improve the quality of their work. Using a dataset of papers published in economics journals,…
A large host of scientific journals and conferences solicit peer reviews from multiple reviewers for the same submission, aiming to gather a broader range of perspectives and mitigate individual biases. In this work, we reflect on the role…
Citation metrics are becoming pervasive in the quantitative evaluation of scholars, journals and institutions. More then ever before, hiring, promotion, and funding decisions rely on a variety of impact metrics that cannot disentangle…
Presumably, peer reviewers and Large Language Models (LLMs) do very different things when asked to assess research. Still, recent evidence has shown that LLMs have a moderate ability to predict quality scores of published academic journal…
This paper focuses on the coauthor effect in different types of publications, usually not equally respected in measuring research impact. {\it A priori} unexpected relationships are found between the total coauthor core value, $m_a$, of a…
Recent studies comparing AI-generated and human-authored literary texts have produced conflicting results: some suggest AI already surpasses human quality, while others argue it still falls short. We start from the hypothesis that such…
Evaluating instruction following capabilities for multimodal, multi-turn dialogue is challenging. With potentially multiple instructions in the input model context, the task is time-consuming for human raters and we show LLM based judges…
Double-blind peer review mechanism has become the skeleton of academic research across multiple disciplines including computer science, yet several studies have questioned the quality of peer reviews and raised concerns on potential biases…
Comparing research papers is a conventional method to demonstrate progress in experimental research. We present COMPARE, a taxonomy and a dataset of comparison discussions in peer reviews of research papers in the domain of experimental…
The ability to predict the long-term impact of a scientific article soon after its publication is of great value towards accurate assessment of research performance. In this work we test the hypothesis that good predictions of long-term…