Related papers: Persistent-Transient Policy Evaluation for Markov …
This paper addresses distributed parameter estimation in stochastic dynamic systems with quantized measurements, constrained by quantized communication and Markovian switching directed topologies. To enable accurate recovery of the original…
A non-Markovian model of quantum repeated interactions between a small quantum system and an infinite chain of quantum systems is presented. By adapting and applying usual pro jection operator techniques in this context, discrete versions…
Discounted algorithms often encounter evaluation errors due to their reliance on short-term estimations, which can impede their efficacy in addressing simple, short-term tasks and impose undesired temporal discounts (\(\gamma\)).…
This paper presents a new condition for the existence of optimal stationary policies in average-cost continuous-time Markov decision processes with unbounded cost and transition rates, arising from controlled queueing systems. This…
We consider periodic Markov chains with absorption. Applying to iterates of this periodic Markov chain criteria for the exponential convergence of conditional distributions of aperiodic absorbed Markov chains, we obtain exponential…
This paper considers solving distributed optimization problems in peer-to-peer multi-agent networks. The network is synchronous and connected. By using the proportional-integral (PI) control strategy, various algorithms with fixed stepsize…
A decade ago, Abdulla, Ben Henda and Mayr introduced the elegant concept of decisiveness for denumerable Markov chains [1]. Roughly speaking, decisiveness allows one to lift most good properties from finite Markov chains to denumerable…
As the proportion of converter-interfaced renewable energy resources in the power system is increasing, the strength of the power grid at the connection point of wind turbine generators (WTGs) is gradually weakening. Existing research has…
We consider the problem of estimation in Hidden Markov models with finite state space and nonparametric emission distributions. Efficient estimators for the transition matrix are exhibited, and a semiparametric Bernstein-von Mises result is…
This paper develops a joint spectral radius (JSR) framework for analyzing rank-one deflated Q-value iteration (Q-VI) in discounted Markov decision process control. Focusing on an all-ones residual correction, we interpret the resulting…
Systems of interacting continuous-time Markov chains are a powerful model class, but inference is typically intractable in high dimensional settings. Auxiliary information, such as noisy observations, is typically only available at discrete…
Penalized transformation models (PTMs) are a semiparametric location-scale regression family that estimate a response's conditional distribution directly from the data, and model the location and scale through structured additive…
We consider the parameter estimation of Markov chain when the unknown transition matrix belongs to an exponential family of transition matrices. Then, we show that the sample mean of the generator of the exponential family is an…
Kunchenko's method of polynomial maximization provides a semiparametric apparatus for parameter estimation under non-Gaussian errors, but its classical power basis relies on finite higher-order integer moments. This paper introduces the…
This paper surveys the analysis of parametric Markov models whose transitions are labelled with functions over a finite set of parameters. These models are symbolic representations of uncountable many concrete probabilistic models, each…
We address a numerical framework for the stability and bifurcation analysis of nonlinear partial differential equations (PDEs) in which the solution is sought in the function space spanned by physics-informed random projection neural…
Solving Markov Decision Processes (MDPs) remains a central challenge in sequential decision-making, especially when dealing with large state spaces and long-term optimization criteria. A key step in Bellman dynamic programming algorithms is…
In robust Markov decision processes (RMDPs), it is assumed that the reward and the transition dynamics lie in a given uncertainty set. By targeting maximal return under the most adversarial model from that set, RMDPs address performance…
Stability is one of the most fundamental requirements for systems synthesis. In this paper, we address the stabilization problem for unknown linear systems via policy gradient (PG) methods. We leverage a key feature of PG for Linear…
We study some regularity properties in locally stationary Markov models which are fundamental for controlling the bias of nonparametric kernel estimators. In particular, we provide an alternative to the standard notion of derivative process…