Related papers: Combating Collusion Rings is Hard but Possible
Research funding systems are not isolated systems - they are embedded in a larger scientific system with an enormous influence on the system. This paper aims to analyze the allocation of competitive research funding from different…
Scholarly peer review is a cornerstone of scientific advancement, but the system is under strain due to increasing manuscript submissions and the labor-intensive nature of the process. Recent advancements in large language models (LLMs)…
The Massively Parallel Computation (MPC) model serves as a common abstraction of many modern large-scale data processing frameworks, and has been receiving increasingly more attention over the past few years, especially in the context of…
Properties expressed as the provability of a first-order sentence can be disproved by just finding a model of the negation of the sentence. This fact, however, is meaningful in restricted cases only, depending on the shape of the sentence…
This paper proposes an architecture and assignment management model of a conference management system that performs a precise and accurate automatic assignment of reviewers to papers. The system relies on taxonomy of keywords to describe…
The principle of peer review is central to the evaluation of research, by ensuring that only high-quality items are funded or published. But peer review has also received criticism, as the selection of reviewers may introduce biases in the…
Continual learning has received a great deal of attention recently with several approaches being proposed. However, evaluations involve a diverse set of scenarios making meaningful comparison difficult. This work provides a systematic…
This case study analyzes the expertise, potential conflicts of interest, and objectivity of editors, authors, and peer reviewers involved in a 2022 special journal issue on fertility, pregnancy, and mental health. Data were collected on…
Cycles in graphs often signify interesting processes. For example, cyclic trading patterns can indicate inefficiencies or economic dependencies in trade networks, cycles in food webs can identify fragile dependencies in ecosystems, and…
Collaboration among researchers is an essential component of the modern scientific enterprise, playing a particularly important role in multidisciplinary research. However, we continue to wrestle with allocating credit to the coauthors of…
Conference paper assignment, i.e., the task of assigning paper submissions to reviewers, presents multi-faceted issues for recommender systems research. Besides the traditional goal of predicting `who likes what?', a conference management…
Peer review is crucial for advancing and improving science through constructive criticism. However, toxic feedback can discourage authors and hinder scientific progress. This work explores an important but underexplored area: detecting…
The article focuses on the importance of the automatic assignment of reviewers to papers for increasing the assignment accuracy therefore the quality of the scientific event itself. It discusses the main aspects that influence the…
Studying corruption presents unique challenges. Recent work in the spirit of computational social science exploits newly available data and methods to give a fresh perspective on this important topic. In this chapter we highlight some of…
Massive Open Online Courses (MOOCs) have been used by students as a low-cost and low-touch educational credential in a variety of fields. Understanding the grading mechanisms behind these course assignments is important for evaluating MOOC…
Journal ranking is becoming more important in assessing the quality of academic research. Several indices have been suggested for this purpose, typically on the basis of a citation graph between the journals. We follow an axiomatic approach…
Document ranking experiments should be repeatable. However, the interaction between multi-threaded indexing and score ties during retrieval may yield non-deterministic rankings, making repeatability not as trivial as one might imagine. In…
The "crisis of reproducibility" has been a significant source of controversy, heated debate, and calls for reform to institutional science in recent years. As a long-term solution to address both the present crisis and future obstacles, I…
Peer review is a cornerstone of scientific publishing, including at premier machine learning conferences such as ICLR. As submission volumes increase, understanding the nature and dynamics of the review process is crucial for improving its…
Many different measures are used to assess academic research excellence and these are subject to ongoing discussion and debate within the scientometric, university-management and policy-making communities internationally. One topic of…