Related papers: Analysis of the ALMA Cycle 8 Distributed Peer Revi…

Peer review often involves reviewers submitting their independent reviews, followed by a discussion among reviewers of each paper. A question among policymakers is whether the reviewers of a paper should be anonymous to each other during…

With the rapid advancement of Large Language Models (LLMs), the academic community has faced unprecedented disruptions, particularly in the realm of academic communication. The primary function of peer review is improving the quality of…

Comparing with lecturer marked assessments, peer assessment is a more comprehensive learning process and many of the associated problems have occurred. In this research work, we study the peer-assessment impact on group learning activities…

Artificial intelligence (AI) has been used to advance different fields, such as education, healthcare, and finance. However, the application of AI in the field of project management (PM) has not progressed equally. This paper reports on a…

Peer review meetings (PRMs) are formal meetings during which peers systematically analyze artifacts to improve their quality and report on non-conformities. This paper presents an approach based on protocol analysis for quantifying the…

We conduct a large-scale empirical user study in a live setup to evaluate the acceptance of LLM-generated comments and their impact on the review process. This user study was performed in two organizations, Mozilla (which has its codebase…

A recent report of Littmann [Commun. ACM '21] outlines the existence and the fatal impact of collusion rings in academic peer reviewing. We introduce and analyze the problem Cycle-Free Reviewing that aims at finding a review assignment…

While large language models (LLMs) have been used for automated grading, they have not yet achieved the same level of performance as humans, especially when it comes to grading complex questions. Existing research on this topic focuses on a…

The volume of scientific submissions continues to climb, outpacing the capacity of qualified human referees and stretching editorial timelines. At the same time, modern large language models (LLMs) offer impressive capabilities in…

In recent years there has been a paradigm shift from centralised to geographically distributed resources. Individual entities are no longer able to host or afford the necessary expertise in-house, and, as a consequence, society increasingly…

Collaboration is the defining mode of modern science, yet its core mechanism -- feedback -- remains hard to observe, difficult to scale, and unequally distributed. Here we test whether large language models (LLMs) can contribute to this…

Autonomous coding agents are reshaping software development by creating pull requests (PRs) on GitHub, referred to as agentic PRs. In parallel, the review process is also becoming autonomous, thereby making reviewer bots key actors in the…

Novelty assessment is a central yet understudied aspect of peer review, particularly in high volume fields like NLP where reviewer capacity is increasingly strained. We present a structured approach for automated novelty evaluation that…

The preparation of the long-term telescope schedule follows the submission and scientific review of new proposals. At the European Southern Observatory (ESO) this process entails scheduling the scientific proposals according to their…

After an introduction to the ESA Herschel Space Observatory including a mission overview, science objectives, results and productivity we examine the process and outcomes of the announcements of observing opportunities (AOs). For Herschel,…

We examine selected aspects of peer review and suggest possible improvements. To this end, we analyse a dataset containing information about 300 papers submitted to the Biochemistry and Biotechnology section of the Journal of the Serbian…

Machine learning has rapidly become a tool of choice for the astronomical community. It is being applied across a wide range of wavelengths and problems, from the classification of transients to neural network emulators of cosmological…

We present OpenReviewer, an open-source system for generating high-quality peer reviews of machine learning and AI conference papers. At its core is Llama-OpenReviewer-8B, an 8B parameter language model specifically fine-tuned on 79,000…

Large Language Models (LLMs) are shifting how scientific research is done. It is imperative to understand how researchers interact with these models and how scientific sub-communities like astronomy might benefit from them. However, there…

This work presents a compositional approach for schedulability analysis of Distributed Integrated Modular Avionics (DIMA) systems that consist of spatially distributed ARINC-653 modules connected by a unified AFDX network. We model a DIMA…