Related papers: Criteria for assessing grant applications: A syste…
This article reviews how empirical research of exploratory search is conducted. We investigated aspects of interdisciplinarity, study settings and evaluation methodologies from a systematically selected sample of 231 publications from…
In this pilot study, we investigate the use of GPT4 to assist in the peer-review process. Our key hypothesis was that GPT-generated reviews could achieve comparable helpfulness to human reviewers. By comparing reviews generated by both…
Individual performance and reputation within a company are major factors that influence wage distribution, promotion and firing. Due to the complexity and collaborative nature of contemporary business processes, the evaluation of individual…
The need for controlling and effectively managing credit risk has led financial institutions to excel in improving techniques designed for this purpose, resulting in the development of various quantitative models by financial institutions…
The selection process of proposals is a crucial component of scientific progress and innovations. Limited resources must be allocated in the most effective way to maximise advancements and the production of new knowledge, especially as it…
Predictive artificial intelligence (AI) offers an opportunity to improve clinical practice and patient outcomes, but risks perpetuating biases if fairness is inadequately addressed. However, the definition of "fairness" remains unclear. We…
At the heart of many scientific conferences is the problem of matching submitted papers to suitable reviewers. Arriving at a good assignment is a major and important challenge for any conference organizer. In this paper we propose a…
Purpose: Despite the importance of peer review for grant funding decisions, academics are often reluctant to conduct it. This can lead to long delays between submission and the final decision as well as the risk of substandard reviews from…
The importance of peer-review in the scientific process can not be overestimated. Yet, due to increasing pressures of research and exponentially growing number of publications the task faced by the referees becomes ever more difficult. We…
Peer grading systems make large courses more scalable, provide students with faster and more detailed feedback, and help students to learn by thinking critically about the work of others. A key obstacle to the broader adoption of peer…
Scholarly journals rely on peer review to identify the science most worthy of publication. Yet finding willing and qualified reviewers to evaluate manuscripts has become an increasingly challenging task, possibly even threatening the…
This work establishes a foundational framework for standardizing AI evaluation RCTs (sometimes called human uplift studies). Drawing on established experimental practices from disciplines with established RCT traditions, including software…
Research in information systems includes a wide range of approaches which make a contribution in terms of knowledge, understanding, or practical developments. The measure of any research is, ultimately, its validity: are its finding true,…
Quantifying systematic disparities in numerical quantities such as employment rates and wages between population subgroups provides compelling evidence for the existence of societal biases. However, biases in the text written for members of…
Scholarly article impact reflects the significance of academic output recognised by academic peers, and it often plays a crucial role in assessing the scientific achievements of researchers, teams, institutions and countries. It is also…
Many decision-making processes involve evaluating and then selecting items; examples include scientific peer review, job hiring, school admissions, and investment decisions. The eventual selection is performed by applying rules or…
Peer reviews, evaluations, and selections are a fundamental aspect of modern science. Funding bodies the world over employ experts to review and select the best proposals from those submitted for funding. The problem of peer selection,…
The paper describes a potential platform to facilitate academic peer review with emphasis on early-stage research. This platform aims to make peer review more accurate and timely by rewarding reviewers on the basis of peer prediction…
Context: Over the last decade, software researchers and engineers have developed a vast body of methodologies and technologies in requirements engineering for self-adaptive systems. Although existing studies have explored various aspects of…
Literature search is critical for any scientific research. Different from Web or general domain search, a large portion of queries in scientific literature search are entity-set queries, that is, multiple entities of possibly different…