An Interval-Score ROC Curve for Assessment, Calibration and Ensembling of Probabilistic Forecasts

Probabilistic forecast evaluation is inherently multi-objective, yet existing proper scoring rules reduce predictive performance to a single scalar value, potentially obscuring the trade-off between forecast concentration and predictive accuracy. We introduce the Interval-Score Receiver Operating Characteristic (IS-ROC) Curve, a graphical framework that represents the complete family of interval forecasts generated by varying prediction tightness. We show that the IS-ROC Curve induced by the data generating process is Pareto optimal and convex, providing a geometric characterization of the optimal forecasting frontier. Building on these properties, we propose a geometry-based calibration procedure based on tangent optimization and convexification, together with an ensemble strategy that combines competing forecasters through convex hull construction. Finally, we provide a practical workflow and numerical examples illustrating forecast comparison, calibration, and ensemble construction within the proposed framework.