相关论文: The Collusion of Memory and Nonlinearity in Stocha…
We consider a generic convex-concave saddle point problem with separable structure, a form that covers a wide-ranged machine learning applications. Under this problem structure, we follow the framework of primal-dual updates for saddle…
The paper concerns the $d$-dimensional stochastic approximation recursion, $$ \theta_{n+1}= \theta_n + \alpha_{n + 1} f(\theta_n, \Phi_{n+1}) $$ where $ \{ \Phi_n \}$ is a stochastic process on a general state space, satisfying a…
We derive new and improved non-asymptotic deviation inequalities for the sample average approximation (SAA) of an optimization problem. Our results give strong error probability bounds that are "sub-Gaussian"~even when the randomness of the…
In this paper, we study the almost sure boundedness and the convergence of the stochastic approximation (SA) algorithm. At present, most available convergence proofs are based on the ODE method, and the almost sure boundedness of the…
It is shown that an exact solution of the transient dynamics of an associative memory model storing an infinite number of limit cycles with l finite steps by means of the path-integral analysis. Assuming the Maxwell construction ansatz, we…
The law of the iterated logarithm (LIL) for the time-homogeneous Markov process with a unique invariant measure characterizes the almost sure maximum possible fluctuation of time averages around the ergodic limit. Whether a numerical…
In this paper, we present a novel stochastic method for solving variational inequalities (VI) in the context of Markovian noise. By leveraging Extragradient technique, we can productively solve VI optimization problems characterized by…
Polyak-Ruppert averaging is a widely used technique to achieve the optimal asymptotic variance of stochastic approximation (SA) algorithms, yet its high-probability performance guarantees remain underexplored in general settings. In this…
We propose a system identification method, Non-Markovian Optimization-based Modeling for Approximate Dynamics with Spatially-homogeneous memory (NOMADS), for identifying linear dynamical systems from a set of multi-dimensional time-series…
We discuss numerical approximation methods for Random Time Change equations which possess a deterministic drift part and jump with state-dependent rates. It is first established that solutions to such equations are versions of certain…
The problem of non-monotone $k$-submodular maximization under a knapsack constraint ($\kSMK$) over the ground set size $n$ has been raised in many applications in machine learning, such as data summarization, information propagation, etc.…
We study the rate of convergence of linear two-time-scale stochastic approximation methods. We consider two-time-scale linear iterations driven by i.i.d. noise, prove some results on their asymptotic covariance and establish asymptotic…
In present paper we suggest a new universal approach to study complex systems by microscopic, mesoscopic and macroscopic methods. We discuss new possibilities of extracting information on nonstationarity, unsteadiness and non-Markovity of…
In this paper we present the concept of description of random processes in complex systems with the discrete time. It involves the description of kinetics of discrete processes by means of the chain of finite-difference non-Markov equations…
We study asymptotic behaviour of stochastic approximation procedures with three main characteristics: truncations with random moving bounds, a matrix valued random step-size sequence, and a dynamically changing random regression function.…
In this paper we propose a new method for approximating the nonstationary moment dynamics of one dimensional Markovian birth-death processes. By expanding the transition probabilities of the Markov process in terms of Poisson-Charlier…
In this paper, inspired from our previous algorithm, which was based on the theory of Tsallis statistical mechanics, we develop a new evolving stochastic learning algorithm for neural networks. The new algorithm combines deterministic and…
It is known that adaptive optimization algorithms represent the key pillar behind the rise of the Machine Learning field. In the Optimization literature numerous studies have been devoted to accelerated gradient methods but only recently…
Motivated by the pressing challenges in the digital twin development for biomanufacturing systems, we introduce an adjoint sensitivity analysis (SA) approach to expedite the learning of mechanistic model parameters. In this paper, we…
In this article, we study the dynamics of a nonlinear system governed by an ordinary differential equation under the combined influence of fast periodic sampling with period $\delta$ and small jump noise of size $\varepsilon, 0<…