Related papers: Partial VOROS: A Cost-aware Performance Metric for…

Estimating average human performance has been performed inconsistently in research in diagnostic medicine. This has been particularly apparent in the field of medical artificial intelligence, where humans are often compared against AI…

Many performance metrics have been introduced for the evaluation of classification performance, with different origins and niches of application: accuracy, macro-accuracy, area under the ROC curve, the ROC convex hull, the absolute error,…

Algorithmic bias continues to be a key concern of learning analytics. We study the statistical properties of the Absolute Between-ROC Area (ABROCA) metric. This fairness measure quantifies group-level differences in classifier performance…

While test-time scaling with verification has shown promise in improving the performance of large language models (LLMs), the role of the verifier and its imperfections remain underexplored. The effect of verification manifests through…

In a clustered observational study, treatment is assigned to groups and all units within the group are exposed to the treatment. Here, we use a clustered observational study (COS) design to estimate the effectiveness of Magnet Nursing…

Ratio-based biomarkers (RBBs), such as the proportion of necrotic tissue within a tumor, are widely used in clinical practice to support diagnosis, prognosis, and treatment planning. These biomarkers are typically estimated from…

While crowdsourcing has emerged as a practical solution for labeling large datasets, it presents a significant challenge in learning accurate models due to noisy labels from annotators with varying levels of expertise. Existing methods…

Nonlinear and nonaffine terms in parametric partial differential equations can potentially lead to a computational cost of a reduced order model (ROM) that is comparable to the cost of the original full order model (FOM). To address this,…

The performance of risk prediction models is often characterized in terms of discrimination and calibration. The Receiver Operating Characteristic (ROC) curve is widely used for evaluating model discrimination. When evaluating the…

Time-dependent partial differential equations are ubiquitous in physics-based modeling, but they remain computationally intensive in many-query scenarios, such as real-time forecasting, optimal control, and uncertainty quantification.…

In the last decade, research on artificial intelligence has seen rapid growth with deep learning models, especially in the field of medical image segmentation. Various studies demonstrated that these models have powerful prediction…

Given measurements from sensors and a set of standard forces, an optimization based approach to identify weakness in structures is introduced. The key novelty lies in letting the load and measurements to be random variables. Subsequently…

Conventional active learning algorithms assume a single labeler that produces noiseless label at a given, fixed cost, and aim to achieve the best generalization performance for given classifier under a budget constraint. However, in many…

This paper proposes a new robust optimization (RO) formulation namely the RO under objective functional uncertainty (ObRO). The ObRO adopts a min-max structure where the inner problem finds the worst-case objective function in a continuous…

As a variant of the Area Under the ROC Curve (AUC), the partial AUC (PAUC) focuses on a specific range of false positive rate (FPR) and/or true positive rate (TPR) in the ROC curve. It is a pivotal evaluation metric in real-world scenarios…

Compared with multi-class classification, multi-label classification that contains more than one class is more suitable in real life scenarios. Obtaining fully labeled high-quality datasets for multi-label classification problems, however,…

In this paper, we propose a computationally efficient approach -- space(Sparse PArtial Correlation Estimation)-- for selecting non-zero partial correlations under the high-dimension-low-sample-size setting. This method assumes the overall…

Probability forecasts for binary outcomes, often referred to as probabilistic classifiers or confidence scores, are ubiquitous in science and society, and methods for evaluating and comparing them are in great demand. We propose and study a…

Methods for the evaluation of the predictive accuracy of biomarkers with respect to survival outcomes subject to right censoring have been discussed extensively in the literature. In cancer and other diseases, survival outcomes are commonly…

When evaluating the performance of clinical machine learning models, one must consider the deployment population. When the population of patients with observed labels is only a subset of the deployment population (label selection), standard…