Related papers: A model sufficiency test using permutation entropy

We describe an approach to improving model fitting and model generalization that considers the entropy of distributions of modelling residuals. We use simple simulations to demonstrate the observational signatures of overfitting on ordered…

There is a fundamental disconnect between what is tested in a model adequacy test, and what we would like to test. The usual approach is to test the null hypothesis "Model M is the true model." However, Model M is never the true model. A…

This paper proposes new parametric model adequacy tests for possibly nonlinear and nonstationary time series models with noncontinuous data distribution, which is often the case in applied work. In particular, we consider the correct…

We consider here together the inference questions and the change-point problem in Poisson autoregressions (see Tj{\o}stheim, 2012). The conditional mean (or intensity) of the process is involved as a non-linear function of it past values…

In scientific inference problems, the underlying statistical modeling assumptions have a crucial impact on the end results. There exist, however, only a few automatic means for validating these fundamental modelling assumptions. The…

We seek to find normative criteria of adequacy for nonmonotonic logic similar to the criterion of validity for deductive logic. Rather than stipulating that the conclusion of an inference be true in all models in which the premises are…

Economic models may exhibit incompleteness depending on whether or not they admit certain policy-relevant features such as strategic interaction, self-selection, or state dependence. We develop a novel test of model incompleteness and…

A rich set of frequentist model averaging methods has been developed, but their applications have largely been limited to point prediction, as measuring prediction uncertainty in general settings remains an open problem. In this paper we…

Unbiased assessment of the predictivity of models learnt by supervised machine-learning methods requires knowledge of the learned function over a reserved test set (not used by the learning algorithm). The quality of the assessment depends,…

This paper is devoted to change-point detection using only the ordinal structure of a time series. A statistic based on the conditional entropy of ordinal patterns characterizing the local up and down in a time series is introduced and…

Estimation of permutation entropy (PE) using Bayesian statistical methods is presented for systems where the ordinal pattern sampling follows an independent, multinomial distribution. It is demonstrated that the PE posterior distribution is…

When the transferable set is unknowable, transfering informative knowledge as much as possible\textemdash a principle we refer to as \emph{sufficiency}, becomes crucial for enhancing transfer learning effectiveness. However, existing…

Determining whether a provided context contains sufficient information to answer a question is a critical challenge for building reliable question-answering systems. While simple prompting strategies have shown success on factual questions,…

In the field of machine learning, regression problems are pivotal due to their ability to predict continuous outcomes. Traditional error metrics like mean squared error, mean absolute error, and coefficient of determination measure model…

Computers are deterministic dynamical systems (CHAOS 19:033124, 2009). Among other things, that implies that one should be able to use deterministic forecast rules to predict their behavior. That statement is sometimes-but not always-true.…

Monotonicity is a key qualitative prediction of a wide array of economic models derived via robust comparative statics. It is therefore important to design effective and practical econometric methods for testing this prediction in empirical…

Classical two-sample permutation tests for equality of distributions have exact size in finite samples, but they fail to control size for testing equality of parameters that summarize each distribution. This paper proposes permutation tests…

Ordinal measures provide a valuable collection of tools for analyzing correlated data series. However, using these methods to understand the information interchange in networks of dynamical systems, and uncover the interplay between…

This book is about conformal prediction and related inferential techniques that build on permutation tests and exchangeability. These techniques are useful in a diverse array of tasks, including hypothesis testing and providing uncertainty…

We introduce a novel approach for comparing out-of-sample multi-step forecasts obtained from a pair of nested models that is based on the forecast encompassing principle. Our proposed approach relies on an alternative way of testing the…