相关论文: Inverse Exponential Decay: Stochastic Fixed Point …

In this work, we offer a theoretical analysis of two modern optimization techniques for training large and complex models: (i) adaptive optimization algorithms, such as Adam, and (ii) the model exponential moving average (EMA).…

We consider the problem of learning convex aggregation of models, that is as good as the best convex aggregation, for the binary classification problem. Working in the stream based active learning setting, where the active learner has to…

We consider solutions of the stochastic equation $X \stackrel{d}= \sum_{i=1}^N A_iX_i + B$, where $N$ is a random natural number, $B$ and $A_i$ are random positive numbers and $X_i$ are independent copies of $X$, which are independent also…

Stochasticity in language model fine-tuning, often caused by the small batch sizes typically used in this regime, can destabilize training by introducing large oscillations in generation quality. A popular approach to mitigating this…

Weight averaging of Stochastic Gradient Descent (SGD) iterates is a popular method for training deep learning models. While it is often used as part of complex training pipelines to improve generalization or serve as a `teacher' model,…

We address the problem of estimating the edge of a bounded set in R^d given a random set of points drawn from the interior. Our method is based on a transformation of estimators dedicated to uniform point processes and obtained by smoothing…

We perform a qualitative analysis of the critical equation associated with a stationary ergodic Hamiltonian through a stochastic version of the metric method, where the notion of closed random stationary set, issued from stochastic…

We provide conditions for the existence of measurable solutions to the equation $\xi(T\omega)=f(\omega,\xi(\omega))$, where $T:\Omega \rightarrow\Omega$ is an automorphism of the probability space $\Omega$ and $f(\omega,\cdot)$ is a…

When using the finite element method (FEM) in inverse problems, its discretization error can produce parameter estimates that are inaccurate and overconfident. The Bayesian finite element method (BFEM) provides a probabilistic model for the…

This work investigates the tail behavior of solutions to the affine stochastic fixed-point equation of the form $X\stackrel{d}{=}AX+B$, where $X$ and $(A,B)$ are independent. Focusing on the light-tail regime, following [Burdzy et al.…

Given any finite or countable collection of real numbers $T_j,j\in J$, we find all solutions $F$ to the stochastic fixed point equation \[W\stackrel{\mathrm {d}}{=}\inf_{j\in J}T_jW_j,\] where $W$ and the $W_j,j\in J$, are independent…

In this paper, we introduce a class of backward stochastic equations (BSEs) that extend classical BSDEs and include many interesting examples of generalized BSDEs as well as semimartingale backward equations. We show that a BSE can be…

Nonstationarity of real-life time series requires model adaptation. In classical approaches like ARMA-ARCH there is assumed some arbitrarily chosen dependence type. To avoid their bias, we will focus on novel more agnostic approach: moving…

In this paper, we consider a stochastic system described by a differential equation admitting a spatially varying random coefficient. The differential equation has been employed to model various static physics systems such as elastic…

We derive a closed-form expression for the finite predictor coefficients of multivariate ARMA (autoregressive moving-average) processes. The expression is given in terms of several explicit matrices that are of fixed sizes independent of…

The class of locally stationary processes assumes that there is a time-varying spectral representation, that is, the existence of finite second moment. We propose the $\alpha$-stable locally stationary process by modifying the innovations…

Automatic amortized resource analysis (AARA) is a type-based technique for inferring concrete (non-asymptotic) bounds on a program's resource usage. Existing work on AARA has focused on bounds that are polynomial in the sizes of the inputs.…

For adaptive mixed finite element methods (AMFEM), we first introduce the data oscillation to analyze, without the restriction that the inverse of the coefficient matrix of the partial differential equations (PDEs) is a piecewise polynomial…

Optimization over the set of matrices $X$ that satisfy $X^\top B X = I_p$, referred to as the generalized Stiefel manifold, appears in many applications involving sampled covariance matrices such as the canonical correlation analysis (CCA),…

Mixtures of $r$ independent distributions for two discrete random variables can be represented by matrices of nonnegative rank $r$. Likelihood inference for the model of such joint distributions leads to problems in real algebraic geometry…