Related papers: Levy Approximation of Impulsive Recurrent Process …

This paper considers optimization over multiple renewal systems coupled by time average constraints. These systems act asynchronously over variable length frames. For each system, at the beginning of each renewal frame, it chooses an action…

In this paper, we study the weak convergence of the extremes of supercritical branching L\'evy processes $\{\mathbb{X}_t, t \ge0\}$ whose spatial motions are L\'evy processes with regularly varying tails. The result is drastically different…

This paper analyzes reinforcement learning (RL) algorithms for Markov decision processes (MDPs) under the average-reward criterion. We focus on Q-learning algorithms based on relative value iteration (RVI), which are model-free stochastic…

We study stability issue of reset and impulsive switched systems. We find time constraints (dwell time and flee time) on switching signals which stabilize a given reset switched system. For a given collection of matrices, we find an…

We provide necessary and sufficient conditions for convergence of exponential integrals of Markov additive processes. Other than in the classical L\'evy case studied by Erickson and Maller we have to distinguish between almost sure…

In this paper, we provide strong $L_2$-rates of approximation of the integral-type functionals of Markov processes by integral sums. We improve the method developed in [2]. Under assumptions on the process formulated only in terms of its…

We propose a new policy, called the LP-update policy, to solve finite horizon weakly-coupled Markov decision processes. The latter can be seen as multi-constraint multi-action bandits, and generalize the classical restless bandit problems.…

The law of the iterated logarithm (LIL) for the time-homogeneous Markov process with a unique invariant measure characterizes the almost sure maximum possible fluctuation of time averages around the ergodic limit. Whether a numerical…

We relax a number of assumptions in Alexeev and Tapon (2012) in order to account for non-normally distributed, skewed, multi-regime, and leptokurtic asset return distributions. We calibrate a Markov-modulated Levy process model to equity…

We identify the linear space spanned by the real-valued excessive functions of a Markov process with the set of those functions which are quasimartingales when we compose them with the process. Applications to semi-Dirichlet forms are…

This article develops general conditions for weak convergence of adaptive Markov chain Monte Carlo processes and is shown to imply a weak law of large numbers for bounded Lipschitz continuous functions. This allows an estimation theory for…

We derive characteristic function identities for conditional distributions of an r-trimmed Levy process given its r largest jumps up to a designated time t. Assuming the underlying Levy process is in the domain of attraction of a stable…

Let $(Q_t)$ be a stationary workload process, and $r(t)$ the correlation coefficient of $Q_0$ and $Q_t$. In a series of previous papers (i) the transform of $r(\cdot)$ has been derived for the case that the driving process is…

We present for the first time an asymptotic convergence analysis of two time-scale stochastic approximation driven by "controlled" Markov noise. In particular, the faster and slower recursions have non-additive controlled Markov noise…

Model reduction of Markov processes is a basic problem in modeling state-transition systems. Motivated by the state aggregation approach rooted in control theory, we study the statistical state compression of a discrete-state Markov chain…

We study the modified log-Sobolev inequality for a class of pure jump Markov processes that describe the interactions between brain neurons. In particular, we focus on a finite and compact process with degenerate jumps inspired by the model…

This paper provides a general and abstract approach to approximate ergodic regimes of Markov and Feller processes. More precisely, we show that the recursive algorithm presented in Lamberton & Pages (2002) and based on simulation algorithms…

Recent works in Learning-Based Model Predictive Control of dynamical systems show impressive sample complexity performances using criteria from Information Theory to accelerate the learning procedure. However, the sequential exploration…

In this paper we consider the convergence of the conditional entropy to the entropy rate for Markov chains. Convergence of certain statistics of long range dependent processes, such as the sample mean, is slow. It has been shown in Carpio…

This paper presents a new algorithmic framework for computing sparse solutions to large-scale linear discrete ill-posed problems. The approach is motivated by recent perspectives on iteratively reweighted norm schemes, viewed through the…