Related papers: Systematic Analysis of Experiment Precision Measur…
As quantum computing continues to emerge, ensuring the quality of quantum programs has become increasingly critical. Quantum program testing has emerged as a prominent research area within the scope of quantum software engineering. While…
Ideally, a meta-analysis will summarize data from several unbiased studies. Here we consider the less than ideal situation in which contributing studies may be compromised by measurement error. Measurement error affects every study design,…
Similarity measures are fundamental tools for quantifying the alignment between artificial and biological systems. However, the diversity of similarity measures and their varied naming and implementation conventions makes it challenging to…
Summarization evaluation remains an open research problem: current metrics such as ROUGE are known to be limited and to correlate poorly with human judgments. To alleviate this issue, recent work has proposed evaluation metrics which rely…
Predicting the collaboration likelihood and measuring cognitive trust to AI systems is more important than ever. To do that, previous research mostly focus solely on the model features (e.g., accuracy, confidence) and ignore the human…
Predictive models are often introduced to decision-making tasks under the rationale that they improve performance over an existing decision-making policy. However, it is challenging to compare predictive performance against an existing…
The Wigner's friend type of thought experiments manifest the conceptual challenge on how different observers can have consistent descriptions of a quantum measurement event. In this paper, we analyze the extended version of Wigner's friend…
Meta-evaluation of automatic evaluation metrics -- assessing evaluation metrics themselves -- is crucial for accurately benchmarking natural language processing systems and has implications for scientific inquiry, production model…
Prior research has shown that physics students often think about experimental procedures and data analysis very differently from experts. One key framework for analyzing student thinking has found that student thinking is more point-like,…
A recent research trend has emerged to identify developers' emotions, by applying sentiment analysis to the content of communication traces left in collaborative development environments. Trying to overcome the limitations posed by using…
A novel approach for comparing quality attributes of different products when there is considerable product-related variability is proposed. In such a case, the whole range of possible realizations must be considered. Looking, for example,…
Molecular simulations and biophysical experiments can be used to provide independent and complementary insights into the molecular origin of biological processes. A particularly useful strategy is to use molecular simulations as a modelling…
Systematic reviews of interventions are important tools for synthesizing evidence from multiple studies. They serve to increase power and improve precision, in the same way that larger studies can do, but also to establish the consistency…
Experimental program review in our field may benefit from a more quantitative framework within which to quantitatively discuss the scientific merit of a proposed program of research, and to assess the scientific merit of a particular…
Objective evaluation of audio processed with Time-Scale Modification (TSM) remains an open problem. Recently, a dataset of time-scaled audio with subjective quality labels was published and used to create an initial objective measure of…
As neural networks become more popular, the need for accompanying uncertainty estimates increases. There are currently two main approaches to test the quality of these estimates. Most methods output a density. They can be compared by…
Effectively measuring, understanding, and improving mobile app performance is of paramount importance for mobile app developers. Across the mobile Internet landscape, companies run online controlled experiments (A/B tests) with thousands of…
Binary classification is a fundamental task in machine learning, with applications spanning various scientific domains. Whether scientists are conducting fundamental research or refining practical applications, they typically assess and…
Context: Empirical studies in software engineering are typically centered on human subjects, ranging from novice to experienced developers. The experience of these individuals is a key context factor that should be properly characterized…
Subjectivity is the expression of internal opinions or beliefs which cannot be objectively observed or verified, and has been shown to be important for sentiment analysis and word-sense disambiguation. Furthermore, subjectivity is an…