Related papers: Constructing Strong Markov Processes
A Markov assumption considers a physical system memoryless to simplify its dynamics. Whereas memory effect or the non-Markovian phenomenon is more general in nature. In the quantum regime, it is challenging to define or quantify the…
In the L\'evy construction of Brownian motion, a Haar-derived basis of functions is used to form a finite-dimensional process $W^{N}$ and to define the Wiener process as the almost sure path-wise limit of $W^{N}$ when $N$ tends to infinity.…
We present a new class of multifractal process on R, constructed using an embedded branching process. The construction makes use of known results on multitype branching random walks, and along the way constructs cascade measures on the…
Big networks express various large-scale networks in many practical areas such as computer networks, internet of things, cloud computation, manufacturing systems, transportation networks, and healthcare systems. This paper analyzes such big…
In this paper, we provide novel characterizations of the weakly unobservable and the strongly reachable subspaces corresponding to a given state-space system. These characterizations provide closed-form representations for the said…
We define a Markov process on the set of countable graphs with spins. Transitions are local substitutions in the graph. It is proved that the scaling macrodimension is an invariant of such dynamics.
Stochastic and soft optimal policies resulting from entropy-regularized Markov decision processes (ER-MDP) are desirable for exploration and imitation learning applications. Motivated by the fact that such policies are sensitive with…
Consider an N-dimensional Markov chain obtained from N one-dimensional random walks by Doob h-transform with the q-Vandermonde determinant. We prove that as N becomes large, these Markov chains converge to an infinite-dimensional Feller…
This work extends the results of the recently developed theory of a rather complete thermodynamic formalism for discrete-state, continuous-time Markov processes with and without detailed balance. We aim at investigating the question that…
In this paper, we consider a class of inhomogeneous semi-Markov processes directly based on intensity processes for marked point processes. We show that this class satisfies the semi-Markov properties defined elsewhere in the literature. We…
The general framework on the non-local Markovian symmetric forms on weighted $l^p$ $(p \in [1, \infty])$ spaces constructed by [A,Kagawa,Yahagi,Y 2020], by restricting the situation where $p =2$, is applied to such measure spaces as the…
The thermodynamic formalism, which was first developed for dynamical systems and then applied to discrete Markov processes, turns out to be well suited for continuous time Markov processes as well, provided the definitions are interpreted…
The output of a discrete Markov source is to be encoded instantaneously by a variable-rate encoder and decoded by a finite-state decoder. Our performance measure is a linear combination of the distortion and the instantaneous rate.…
Let $X$ be a Markov process taking values in $\mathbf{E}$ with continuous paths and transition function $(P_{s,t})$. Given a measure $\mu$ on $(\mathbf{E}, \mathscr{E})$, a Markov bridge starting at $(s,\varepsilon_x)$ and ending at…
We consider a discrete time semi-Markov process where the characteristics defining the process depend on a small perturbation parameter. It is assumed that the state space consists of one finite communicating class of states and, in…
We study a class of dynamical systems generated by random substitutions, which contains both intrinsically ergodic systems and instances with several measures of maximal entropy. In this class, we show that the measures of maximal entropy…
Consider a Markov process $\{\Phi(t) : t\geq 0\}$ evolving on a Polish space ${\sf X}$. A version of the $f$-Norm Ergodic Theorem is obtained: Suppose that the process is $\psi$-irreducible and aperiodic. For a given function $f\colon{\sf…
We consider a robust approach to address uncertainty in model parameters in Markov Decision Processes (MDPs), which are widely used to model dynamic optimization in many applications. Most prior works consider the case where the uncertainty…
General Markov chains with a countably additive transition probability in arbitrary phase space are considered. Markov operators extend from the space of countably additive measures to the space of finitely additive measures. In the…
In this paper, we consider Markov chain and linear quadratic models for deep structured teams with discounted and time-average cost functions under two non-classical information structures, namely, deep state sharing and no sharing. In deep…