Related papers: Quantifying replicability and consistency in syste…
Coherent superpositions are one of the hallmarks of quantum mechanics and are vital for any quantum mechanical device to outperform the classically achievable. Generically, superpositions are verified in interference experiments, but…
The advent of modern data collection and processing techniques has seen the size, scale, and complexity of data grow exponentially. A seminal step in leveraging these rich datasets for downstream inference is understanding the…
Data consistency is very desirable because strong semantic properties make it easier to write correct programs that perform as users expect. However, there are good reasons why consistency may have to be weakened to achieve other business…
In this paper, we propose a robust method to estimate the average treatment effects in observational studies when the number of potential confounders is possibly much greater than the sample size. We first use a class of penalized…
Variance theories quantify the variance that one or more independent variables cause in a dependent variable. In software engineering (SE), variance theories are used to quantify -- among others -- the impact of tools, techniques, and other…
The lack of predictability of citation-based measures frequently used to gauge impact, from impact factors to short-term citations, raises a fundamental question: Is there long-term predictability in citation patterns? Here, we derive a…
Coherence is an essential property of well-written texts, that refers to the way textual units relate to one another. In the era of generative AI, coherence assessment is essential for many NLP tasks; summarization, generation, long-form…
Peer review is vital in academia for evaluating research quality. Top AI conferences use reviewer confidence scores to ensure review reliability, but existing studies lack fine-grained analysis of text-score consistency, potentially missing…
Background: The software engineering community has increasingly conducted systematic literature reviews (SLR) as a means to summarize evidence from different studies and bring to light the state of the art of a given research topic. While…
Currently used metrics for assessing summarization algorithms do not account for whether summaries are factually consistent with source documents. We propose a weakly-supervised, model-based approach for verifying factual consistency and…
Machine learning enables the development of new, supplemental, and empowering tools that can either expand existing technologies or invent new ones. In education, space exists for a tool that supports generic student course review formats…
Ranking individuals based on their performance in different coalitions is a problem emerging in various domains (teams sports, scientific evaluation, argumentation, etc.). Often, for practical reasons, the number of comparable coalitions is…
In this paper I introduce quantile spectral densities that summarize the cyclical behavior of time series across their whole distribution by analyzing periodicities in quantile crossings. This approach can capture systematic changes in the…
We introduce a set of resampling-based methods for quantifying uncertainty and statistical precision of evaluation metrics in multilingual and/or multitask NLP benchmarks. We show how experimental variation in performance scores arises from…
A bibliographic database containing studies on recurrence plots and related methods is analyzed from various perspectives. This allows a detailed view of the field's development, showcasing the continuous growth in the method's popularity,…
Informatics and technological advancements have triggered generation of huge volume of data with varied complexity in its management and analysis. Big Data analytics is the practice of revealing hidden aspects of such data and making…
In the criminal legal context, risk assessment algorithms are touted as data-driven, well-tested tools. Studies known as validation tests are typically cited by practitioners to show that a particular risk assessment algorithm has…
Based on existing ideas in the field of imprecise probabilities, we present a new approach for assessing the reliability of the individual predictions of a generative probabilistic classifier. We call this approach robustness…
Recurrence rate, determinism, average line length, and entropy of line lengths are measures of complexity in recurrence quantification analysis, that help to understand the structure, predictability and complexity of dynamical systems. In…
Identifying critical research within the growing body of academic work is an intrinsic aspect of conducting quality research. Systematic review processes used in evidence-based medicine formalise this as a procedure that must be followed in…