Related papers: Comparative Analysis of Existing Methods and Algor…
Collaboration among researchers is an essential component of the modern scientific enterprise, playing a particularly important role in multidisciplinary research. However, we continue to wrestle with allocating credit to the coauthors of…
Peer review is a cornerstone of scientific publishing, including at premier machine learning conferences such as ICLR. As submission volumes increase, understanding the nature and dynamics of the review process is crucial for improving its…
Large Language Models have recently been applied to text annotation tasks from social sciences, equalling or surpassing the performance of human workers at a fraction of the cost. However, no inquiry has yet been made on the impact of…
Peer review at AI conferences is stressed by rapidly rising submission volumes, leading to deteriorating review quality and increased author dissatisfaction. To address these issues, we developed Review Feedback Agent, a system leveraging…
Problems for evaluation and impact of published scientific works and their authors are discussed. The role of citations in this process is pointed out. Different bibliometric indicators are reviewed in this connection and ways for…
The rapid expansion of AI research has intensified the Reviewer Gap, threatening the peer-review sustainability and perpetuating a cycle of low-quality evaluations. This position paper critiques existing LLM approaches that automatically…
Over the past decades, the competition for academic resources has gradually intensified, and worsened with the current financial crisis. To optimize the resource allocation, individualized assessment of research results is being actively…
Mainstream machine learning conferences have seen a dramatic increase in the number of participants, along with a growing range of perspectives, in recent years. Members of the machine learning community are likely to overhear allegations…
Journals and conferences worry that peer reviews assisted by artificial intelligence (AI), in particular, large language models (LLMs), may negatively influence the validity and fairness of the peer-review system, a cornerstone of modern…
Automated Essay scoring has been explored as a research and industry problem for over 50 years. It has drawn a lot of attention from the NLP community because of its clear educational value as a research area that can engender the creation…
Peer review aims to detect flaws and deficiencies in the design and interpretation of studies, and ensure the clarity and quality of their presentation. However, it has been questioned whether peer review fulfils this function. Studies have…
Topic models are typically evaluated with respect to the global topic distributions that they generate, using metrics such as coherence, but without regard to local (token-level) topic assignments. Token-level assignments are important for…
The number of scientific publications is constantly rising, increasing the strain on the review process. The number of submissions is actually higher, as each manuscript is often reviewed several times before publication. To face the deluge…
This article presents new alternatives to the similarity function for the TextRank algorithm for automatic summarization of texts. We describe the generalities of the algorithm and the different functions we propose. Some of these variants…
Paper journals use a small number of trusted academics to select information on behalf of all their readers. This inflexibility in the selection was justified due to the expense of publishing. The advent of cheap distribution via the…
In large organisations, identifying experts on a given topic is crucial in leveraging the internal knowledge spread across teams and departments. So-called enterprise expert retrieval systems automatically discover and structure employees'…
Peer review underpins scientific progress, but it is increasingly strained by reviewer shortages and growing workloads. Large Language Models (LLMs) can automatically draft reviews now, but determining whether LLM-generated reviews are…
Many factors could affect the number of citations to a paper. Citations have an important role in research policy and in measuring the excellence of research and researchers. This work is the first study in software engineering (SE) to…
Modern machine learning and computer science conferences are experiencing a surge in the number of submissions that challenges the quality of peer review as the number of competent reviewers is growing at a much slower rate. To curb this…
This paper is a discussion of the 2025 JASA discussion paper by Su et al. (2025). We would like to congratulate the authors on conducting a comprehensive and insightful empirical investigation of the 2023 ICML ranking data. The review…