Related papers: On optimality of Mallows model averaging
Model Agnostic Meta Learning or MAML has become the standard for few-shot learning as a meta-learning problem. MAML is simple and can be applied to any model, as its name suggests. However, it often suffers from instability and…
Approximate message passing (AMP) is a low-cost iterative parameter-estimation technique for certain high-dimensional linear systems with non-Gaussian distributions. However, AMP only applies to independent identically distributed (IID)…
The normalized maximized likelihood (NML) provides the minimax regret solution in universal data compression, gambling, and prediction, and it plays an essential role in the minimum description length (MDL) method of statistical modeling…
The inherent bias pathology of the maximum likelihood (ML) estimation method is confirmed for models with unknown parameters $\theta$ and $\psi$ when MLE $\hat \psi$ is function of MLE $\hat \theta.$ To reduce $\hat \psi$'s bias the…
A tenet of reinforcement learning is that the agent always observes rewards. However, this is not true in many realistic settings, e.g., a human observer may not always be available to provide rewards, sensors may be limited or…
We study statistical properties of the optimal value of the Sample Average Approximation. The focus is on the tail function of the absolute error induced by the Sample Average Approximation, deriving upper estimates of its outcomes…
In this paper, we propose a sampling algorithm based on state-of-the-art statistical machine learning techniques to obtain conditional nonlinear optimal perturbations (CNOPs), which is different from traditional (deterministic) optimization…
The Method of Successive Approximations (MSA) is a fixed-point iterative method used to solve stochastic optimal control problems. It is an indirect method based on the conditions derived from the Stochastic Maximum Principle (SMP), an…
We consider a critically-loaded multiclass queueing control problem with model uncertainty. The model consists of $I$ types of customers and a single server. At any time instant, a decision-maker (DM) allocates the server's effort to the…
We provide an optimization-based argument for the monotonicity of the multiplicative algorithm (MA) for a class of optimal experimental design problems considered in Yu (2010). Our proof avoids introducing auxiliary variables (or problems)…
Approximate message passing (AMP) refers to a class of efficient algorithms for statistical estimation in high-dimensional problems such as compressed sensing and low-rank matrix estimation. This paper analyzes the performance of AMP in the…
We study a new model-free algorithm to compute $\varepsilon$-optimal policies for average reward Markov decision processes, in the weakly communicating case. Given a generative model, our procedure combines a recursive sampling technique…
We propose an algorithm for generating explicit solutions of multiparametric mixed-integer convex programs to within a given suboptimality tolerance. The algorithm is applicable to a very general class of optimization problems, but is most…
Rejection Sampling is a fundamental Monte-Carlo method. It is used to sample from distributions admitting a probability density function which can be evaluated exactly at any given point, albeit at a high computational cost. However,…
The aims of this study are twofold. First, we consider an optimal risk allocation problem with non-convex preferences. By establishing an infimal representation for distortion risk measures, we give some necessary and sufficient conditions…
Averaging, or smoothing, is a fundamental approach to obtain stable, de-noised estimates from noisy observations. In certain scenarios, observations made along trajectories of random dynamical systems are of particular interest. One popular…
We study in this paper the consequences of using the Mean Absolute Percentage Error (MAPE) as a measure of quality for regression models. We prove the existence of an optimal MAPE model and we show the universal consistency of Empirical…
This paper studies a Bayesian approach to non-asymptotic minimax adaptation in nonparametric estimation. Estimating an input function on the basis of output functions in a Gaussian white-noise model is discussed. The input function is…
This paper investigates the optimization problem of an infinite stage discrete time Markov decision process (MDP) with a long-run average metric considering both mean and variance of rewards together. Such performance metric is important…
Stochastic optimization has found wide applications in minimizing objective functions in machine learning, which motivates a lot of theoretical studies to understand its practical success. Most of existing studies focus on the convergence…