Related papers: A hypothesis test for the long-term calibration in…
Runtime verification focuses on analyzing the execution of a given program by a monitor to determine if it is likely to violate its specifications. There is often an impedance mismatch between the assumptions/model of the monitor and that…
Calibration of mean estimates for predictions is a crucial property in many applications, particularly in the fields of financial and actuarial decision-making. In this paper, we first review classical approaches for validating…
In a system, there are identical replaceable components working for a given task and a failed component is replaced by a functioning one in the corresponding position, which characterizes a repairable system. Assuming that a replaced…
Credit risk stress testing has become an important risk management device which is used both by banks internally and by regulators. Stress testing is complex because it essentially means projecting a bank's full balance sheet conditional on…
The inverse probability weighting approach is popular for evaluating treatment effects in observational studies, but extreme propensity scores could bias the estimator and induce excessive variance. Recently, the overlap weighting approach…
We consider the problem of asynchronous online testing, aimed at providing control of the false discovery rate (FDR) during a continual stream of data collection and testing, where each test may be a sequential test that can start and stop…
Machine-learned systems are in widespread use for making decisions about humans, and it is important that they are fair, i.e., not biased against individuals based on sensitive attributes. We present a general framework of runtime…
Time delay is ubiquitous in many experimental and real-world situations. It is often unclear whether time delay plays a significant role in observed phenomena, and if it does, how long the time lag really is. This would be invaluable…
This work presents a theoretical and empirical evaluation of Anderson-Darling test when the sample size is limited. The test can be applied in order to backtest the risk factors dynamics in the context of Counterparty Credit Risk modelling.…
We propose a general purpose confidence interval procedure (CIP) for statistical functionals constructed using data from a stationary time series. The procedures we propose are based on derived distribution-free analogues of the $\chi^2$…
We introduce a new statistical test based on the observed spacings of ordered data. The statistic is sensitive to detect non-uniformity in random samples, or short-lived features in event time series. Under some conditions, this new test…
We study a simple, solvable model that allows us to investigate effects of credit contagion on the default probability of individual firms, in both portfolios of firms and on an economy wide scale. While the effect of interactions may be…
Regulators require financial institutions to estimate counterparty default risks from liquid CDS quotes for the valuation and risk management of OTC derivatives. However, the vast majority of counterparties do not have liquid CDS quotes and…
This paper develops inference methods for ratios of deterministic trend slopes in systems of pairs of time series. Hypotheses based on linear cross-equation restrictions are considered with particular interest in tests that trend ratios are…
In typical machine learning systems, an estimate of the probability of the prediction is used to assess the system's confidence in the prediction. This confidence measure is usually uncalibrated; i.e.\ the system's confidence in the…
The instability of the financial system as experienced in recent years and in previous periods is often linked to credit defaults, i.e., to the failure of obligors to make promised payments. Given the large number of credit contracts, this…
Cross-validation is a widely-used technique to estimate prediction error, but its behavior is complex and not fully understood. Ideally, one would like to think that cross-validation estimates the prediction error for the model at hand, fit…
Machine learning models are often used to inform real world risk assessment tasks: predicting consumer default risk, predicting whether a person suffers from a serious illness, or predicting a person's risk to appear in court. Given…
We propose experimentally feasible ways to probe universal features of absorbing phase transitions from two different approaches, both based on numerical validations. On one hand, we numerically study a probability distribution of…
We introduce a simple approach for testing the reliability of homogeneous generators and the Markov property of the stochastic processes underlying empirical time series of credit ratings. We analyze open access data provided by Moody's and…