Related papers: Backtesting forecast accuracy
Performance estimation aims at estimating the loss that a predictive model will incur on unseen data. These procedures are part of the pipeline in every machine learning project and are used for assessing the overall generalisation ability…
The integrated conditional moment (ICM) test is a classical and widely used method for assessing the adequacy of regression models. Although it performs well in fixed-dimension settings, its behavior changes dramatically when the predictor…
A criterion is proposed for testing hypothesis about the nature of the error variance in the dependent variable in linear model, which separates correctly and incorrectly specified models. In the former only measurement errors determine the…
Regression analyses based on transformations of cumulative incidence functions are often adopted when modeling and testing for treatment effects in clinical trial settings involving competing and semi-competing risks. Common frameworks…
Spatial point processes are used as models in many different fields ranging from ecology and forestry to cosmology and materials science. In recent years, model validation, and in particular goodness-of-fit testing of a proposed point…
When considering many survival models, decisions become more challenging in health economic evaluation. In this paper, we present a set of methods to assist with selecting the most appropriate parametric survival models. The methods…
In statistical exercises where there are several candidate models, the traditional approach is to select one model using some data driven criterion and use that model for estimation, testing and other purposes, ignoring the variability of…
Traditionally model averaging has been viewed as an alternative to model selection with the ultimate goal to incorporate the uncertainty associated with the model selection process in standard errors and confidence intervals by using a…
Numerical predictions of quantities of interest measured within physical systems rely on the use of mathematical models that should be validated, or at best, not invalidated. Model validation usually involves the comparison of experimental…
Two-sample testing is a fundamental problem in statistics. Despite its long history, there has been renewed interest in this problem with the advent of high-dimensional and complex data. Specifically, in the machine learning literature,…
This paper proposes a new way of evaluating the accuracy and validity of probabilistic forecasts that change over time (such as an in-game win probability model, or an election forecast). Under this approach, each model to be evaluated is…
This project uses a spatial model (Geographically Weighted Regression) to relate various physical and social features to crime rates. Besides making interesting predictions from basic data statistics, the trained model can be used to…
Credit scoring models based on accepted applications may be biased and their consequences can have a statistical and economic impact. Reject inference is the process of attempting to infer the creditworthiness status of the rejected…
A key goal of empirical research in software engineering is to assess practical significance, which answers whether the observed effects of some compared treatments show a relevant difference in practice in realistic scenarios. Even though…
CONTEXT: There is growing interest in establishing software engineering as an evidence-based discipline. To that end, replication is often used to gain confidence in empirical findings, as opposed to reproduction where the goal is showing…
The biastest command in Stata is a powerful and user-friendly tool designed to compare the coefficients of different regression models, enabling researchers to assess the robustness and consistency of their empirical findings. This command…
We perform the a posteriori error analysis of residual type of a transmission problem with sign changing coefficients. According to [6] if the contrast is large enough, the continuous problem can be transformed into a coercive one. We…
Considering a regression model, we address the question of testing the nullity of the regression function. The testing procedure is available when the variance of the observations is unknown and does not depend on any prior information on…
Competing risks models for a repairable system subject to several failure modes are discussed. Under minimal repair, it is assumed that each failure mode has a power law intensity. An orthogonal reparametrization is used to obtain an…
We propose an empirical likelihood ratio test for nonparametric model selection, where the competing models may be nested, nonnested, overlapping, misspecified, or correctly specified. It compares the squared prediction errors of models…