Related papers: Linear Algebraic Truncation Algorithm with A Poste…
We present a novel algorithm to solve a non-linear system of equations, whose solution can be interpreted as a tight lower bound on the vector of expected hitting times of a Markov chain whose transition probabilities are only partially…
Here, we study the ultimately bounded stability of network of mismatched systems using Lyapunov direct method. The upper bound on the error of oscillators from the center of the neighborhood is derived. Then the performance of an adaptive…
Stability is one of the most fundamental requirements for systems synthesis. In this paper, we address the stabilization problem for unknown linear systems via policy gradient (PG) methods. We leverage a key feature of PG for Linear…
This paper develops a new approach to the estimation of the degree of boundedness or stability of multidimensional nonlinear systems with time-dependent nonperiodic coefficients-an essential task in various engineering and natural science…
This paper studies the augmented truncation of discrete-time block-monotone Markov chains under geometric drift conditions. We first present a bound for the total variation distance between the stationary distributions of an original Markov…
We study the optimization of the expected long-term reward in finite partially observable Markov decision processes over the set of stationary stochastic policies. In the case of deterministic observations, also known as state aggregation,…
We study methods based on reproducing kernel Hilbert spaces for estimating the value function of an infinite-horizon discounted Markov reward process (MRP). We study a regularized form of the kernel least-squares temporal difference (LSTD)…
We study and develop the stochastic Markov reward model (sMRM), which extends the Markov chain where transition time/reward as modelled as random variables. Techniques are presented to enable computing first-passage time distributions (or…
Lyapunov redesign is a classical technique that uses a nominal control and its corresponding nominal Lyapunov function to design a discontinuous control, such that it compensates the uncertainties and disturbances. In this paper, the idea…
We develop a family of reformulations of an arbitrary consistent linear system into a stochastic problem. The reformulations are governed by two user-defined parameters: a positive definite matrix defining a norm, and an arbitrary discrete…
Lyapunov functions are fundamental to establishing the stability of Markovian models, yet their construction typically demands substantial creativity and analytical effort. In this paper, we show that deep learning can automate this process…
We study the problem of numerical differentiation of functions from weighted Wiener classes. We construct and analyze a truncation Legendre method to recover arbitrary order derivatives. The main focus is on obtaining error estimates in…
We consider a policy gradient algorithm applied to a finite-arm bandit problem with Bernoulli rewards. We allow learning rates to depend on the current state of the algorithm, rather than use a deterministic time-decreasing learning rate.…
We study the sequential general online regression, known also as the sequential probability assignments, under logarithmic loss when compared against a broad class of experts. We focus on obtaining tight, often matching, lower and upper…
Motivated by the widespread use of temporal-difference (TD-) and Q-learning algorithms in reinforcement learning, this paper studies a class of biased stochastic approximation (SA) procedures under a mild "ergodic-like" assumption on the…
This paper presents a novel stochastic gradient descent algorithm for constrained optimization. The proposed algorithm randomly samples constraints and components of the finite sum objective function and relies on a relaxed logarithmic…
Linear programming approaches have been applied to derive upper bounds on the size of classical codes and quantum codes. In this paper, we derive similar results for general quantum codes with entanglement assistance, including nonadditive…
We consider a discrete-time Markov decision process with Borel state and action spaces. The performance criterion is to maximize a total expected {utility determined by unbounded return function. It is shown the existence of optimal…
We study upper and lower bounds on the sample-complexity of learning near-optimal behaviour in finite-state discounted Markov Decision Processes (MDPs). For the upper bound we make the assumption that each action leads to at most two…
This paper considers an approximation usually used when implementing Ramaswami's recursion for the stationary distribution of the M/G/1-type Markov chain. The approximation is called the level-increment-truncation approximation because it…