Related papers: Deterministic Finite-Memory Bias Estimation

Transformer attention scales quadratically with sequence length O(n^2), limiting long-context use. We propose Adaptive Retention, a probabilistic, layer-wise token selection mechanism that learns which representations to keep under a strict…

Given a stream of Bernoulli random variables, consider the problem of estimating the mean of the random variable within a specified relative error with a specified probability of failure. Until now, the Gamma Bernoulli Approximation Scheme…

A central problem in Binary Hypothesis Testing (BHT) is to determine the optimal tradeoff between the Type I error (referred to as false alarm) and Type II (referred to as miss) error. In this context, the exponential rate of convergence of…

This paper develops asymptotic theory for estimation of parameters in regression models for binomial response time series where serial dependence is present through a latent process. Use of generalized linear model (GLM) estimating…

For affine stochastic differential equation with uniformly distributed time delay the local asymptotic properties of the likelihood function are studied. Local asymptotic normality, local asymptotic mixed normality, periodic local…

Biased stochastic estimators, such as finite-differences for noisy gradient estimation, often contain parameters that need to be properly chosen to balance impacts from the bias and the variance. While the optimal order of these parameters…

We analyse the asymptotic behaviour of the probability of observing the expected number of successes at each stage of a sequence of nested Bernoulli trials. Our motivation is the attempt to give a genuinely frequentist interpretation to the…

Feature selection problems have been extensively studied for linear estimation, for instance, Lasso, but less emphasis has been placed on feature selection for non-linear functions. In this study, we propose a method for feature selection…

Opportunistic detection rules (ODRs) are variants of fixed-sample-size detection rules in which the statistician is allowed to make an early decision on the alternative hypothesis opportunistically based on the sequentially observed…

We present a purely deep neural network-based approach for estimating long memory parameters of time series models that incorporate the phenomenon of long-range dependence. Parameters, such as the Hurst exponent, are critical in…

We analyze the practices of reservoir computing in the framework of statistical learning theory. In particular, we derive finite sample upper bounds for the generalization error committed by specific families of reservoir computing systems…

We consider the estimation of parametric fractional time series models in which not only is the memory parameter unknown, but one may not know whether it lies in the stationary/invertible region or the nonstationary or noninvertible…

Assume that we observe a stochastic process $(X(t))_{t\in[-r,T]}$, which satisfies the linear stochastic delay differential equation \[ \mathrm{d} X(t) = \vartheta \int_{[-r,0]} X(t + u) \, a(\mathrm{d} u) \, \mathrm{d} t + \mathrm{d} W(t)…

Every language recognized by a non-deterministic finite automaton can be recognized by a deterministic automaton, at the cost of a potential increase of the number of states, which in the worst case can go from $n$ states to $2^n$ states.…

Consider the problem of a controller sampling sequentially from a finite number of $N \geq 2$ populations, specified by random variables $X^i_k$, $ i = 1,\ldots , N,$ and $k = 1, 2, \ldots$; where $X^i_k$ denotes the outcome from population…

Let $X$ be a random vector with distribution $P_{\theta}$ where $\theta$ is an unknown parameter. When estimating $\theta$ by some estimator $\varphi(X)$ under a loss function $L(\theta,\varphi)$, classical decision theory advocates that…

We consider upper exponential bounds for the probability of the event that an absolute deviation of sample mean from mathematical expectation p is bigger comparing with some ordered level epsilon. These bounds include 2 coefficients {alpha,…

We study the bias and the mean-squared error of the maximum likelihood estimators (MLE) of parameters associated with a two-parameter mean-reverting process for a finite time $T$. Using the likelihood ratio process, we derive the…

We give improved constants for data dependent and variance sensitive confidence bounds, called empirical Bernstein bounds, and extend these inequalities to hold uniformly over classes of functionswhose growth function is polynomial in the…

Consider the problem of predicting the next symbol given a sample path of length n, whose joint distribution belongs to a distribution class that may have long-term memory. The goal is to compete with the conditional predictor that knows…