Related papers: Errors and secret data in the Italian research ass…
This work examines whether the macroeconomic divide between northern and southern Italy is also present at the level of higher education. The analysis confirms that the research performance in the sciences of the professors in the south is…
Nations, universities, and regional governments commit resources to promote the dissemination of scientific and technical knowledge. One focuses on knowledge-based innovations and the economic function of the university in terms of…
In this paper we revisit the 2014 NeurIPS experiment that examined inconsistency in conference peer review. We determine that 50\% of the variation in reviewer quality scores was subjective in origin. Further, with seven years passing since…
Scientific discoveries must be communicated clearly to realize their full potential. Without effective communication, even the most groundbreaking findings risk being overlooked or misunderstood. The primary way scientists communicate their…
Reproducibility is a key requirement for scientific progress. It allows the reproduction of the works of others, and, as a consequence, to fully trust the reported claims and results. In this work, we argue that, by facilitating…
In the era of big science, many national governments are helping to build well-funded teams of scientists to serve nationalistic ambitions, providing financial incentives for certain outcomes for purposes other than advancing science. That…
In response to the call for a science of science policy, we discuss the contribution of indicators at the macro-level of nations from a scientometric perspective. In addition to global trends such as the rise of China, one can relate…
A noniterative sample size procedure is proposed for a general hypothesis test based on the t distribution by modifying and extending Guenther's (1981) approach for the one sample and two sample t tests. The generalized procedure is…
The literature on gender differences in research performance seems to suggest a gap between men and women, where the former outperform the latter. Whether one agrees with the different factors proposed to explain the phenomenon, it is…
Large-scale assessment data typically include numerous categorical variables, often affected by missing values. Motivated by the challenges arising in this framework, we extend the knockoffs method for selecting predictors to settings with…
Reproducibility, the ability to reproduce the results of published papers or studies using their computer code and data, is a cornerstone of reliable scientific methodology. Studies where results cannot be reproduced by the scientific…
Hypothesis tests based on linear models are widely accepted by organizations that regulate clinical trials. These tests are derived using strong assumptions about the data-generating process so that the resulting inference can be based on…
We report our efforts in identifying a set of previous human evaluations in NLP that would be suitable for a coordinated study examining what makes human evaluations in NLP more/less reproducible. We present our results and findings, which…
Empirical claims about autonomous Kubernetes operations agents are largely unfalsifiable. Published work reports observational results without controlled comparisons against an agent-disabled baseline, selection bias is endemic,…
The rapid advancement of General Purpose AI (GPAI) models necessitates robust evaluation frameworks, especially with emerging regulations like the EU AI Act and its associated Code of Practice (CoP). Current AI evaluation practices depend…
Deceptive reviews, refer to fabricated feedback designed to artificially manipulate the perceived quality of products. Within modern e-commerce ecosystems, these reviews remain a critical governance challenge. Despite advances in…
Higher education systems featuring intense competition have developed world-class universities, capable of attracting top professors and students and considerable public-private funding. This does not occur in non-competitive systems, where…
This study investigates the reliability and validity of five advanced Large Language Models (LLMs), Claude 3.5, DeepSeek v2, Gemini 2.5, GPT-4, and Mistral 24B, for automated essay scoring in a real world higher education context. A total…
For improving the performance and effectiveness of peer review, a novel review system is proposed, based on analysis of peer review process for academic journals under a parallel model built via Monte Carlo method. The model can simulate…
PurposeThe purpose of this paper is to present empirical evidence on the implementation, acceptance and quality-related aspects of research information systems (RIS) in academic institutions.Design/methodology/approachThe study is based on…