Related papers: Further remarks on absorbing Markov decision proce…
We consider an absorbing Markov decision process with Borel state and action spaces. We study conditions under which the MDP is uniformly absorbing and the set of occupation measures of the MDP is compact in the usual weak topology. These…
In this paper, we prove the following assertion for an absorbing Markov decision process (MDP) with the given initial distribution, which is also assumed to be semi-continuous: the continuity of the projection mapping from the space of…
In this paper, we study discrete-time absorbing Markov Decision Processes (MDP) with measurable state space and Borel action space with a given initial distribution. For such models, solutions to the characteristic equation that are not…
In this paper we investigate several geometric properties of the set of occupancy measures. In particular, we analyse the structure of the faces generated by a given occupancy measure, together with their relative algebraic interior. We…
In this paper, we consider a Markov decision process (MDP) with a Borel state space $\textbf{X}\cup\{\Delta\}$, where $\Delta$ is an absorbing state (cemetery), and a Borel action space $\textbf{A}$. We consider the space of finite…
Structural results impose sufficient conditions on the model parameters of a Markov decision process (MDP) so that the optimal policy is an increasing function of the underlying state. The classical assumptions for MDP structural results…
Here we propose the Donsker-Varadhan-type compactness conditions and prove the joint large deviation principle for the empirical measure and empirical flow of Markov renewal processes (semi-Markov processes) with a countable state space,…
We prove that if $X$ is a topological space that admits Debreu's classical utility theorem (eg.\ $X$ is separable and connected, second countable, etc.), then order relations on $X$ satisfying milder completeness conditions can be…
This paper, based on the compactness-continuity and finite value conditions, establishes the sufficiency of the class of stationary policies out of the general class of history-dependent ones for a constrained continuous-time Markov…
Topological measures and deficient topological measures generalize Borel measures and correspond to certain non-linear functionals. We study integration with respect to deficient topological measures on locally compact spaces. Such an…
In Markov Decision Processes (MDPs) with intermittent state information, decision-making becomes challenging due to periods of missing observations. Linear programming (LP) methods can play a crucial role in solving MDPs, in particular,…
We propose and study a model where, for the first time, two aspects are present: parity conservation and infinitely many absorbing states. Whereas steady-state simulations show that the static critical behaviour is not affected by the…
In this article we investigate the restrictions imposed by the dominant energy condition (DEC) on the topology and conformal type of \textsl{possibly non-compact} marginally outer-trapped surfaces (thus extending Hawking's classical theorem…
A Peskun ordering between two samplers, implying a dominance of one over the other, is known among the Markov chain Monte Carlo community for being a remarkably strong result. It is however also known for being a result that is notably…
In this paper we examine two basic topological properties of partial metric spaces, namely compactness and completeness. Our main result claims that in these spaces compactness is equivalent to sequential compactness. We also show that…
The Feinberg-Piunovskiy Theorem established in [14, Theorem 3.8] asserts that for a discrete-time uniformly absorbing and atomless Markov Decision Process (MDP) with Borel state space and multiple criteria, the family of deterministic…
For a $C^{r}$ $(r>1)$ diffeomorphism on a compact manifold that admits a dominated splitting, this paper establishes the upper semi-continuity of the entropy map. More precisely, this paper establishes the upper semi-continuity of the…
We consider exchangeable Markov multi-state survival processes -- temporal processes taking values over a state-space$\mathcal{S}$ with at least one absorbing failure state $\flat \in \mathcal{S}$ that satisfy natural invariance properties…
This paper studies Markov Decision Processes (MDPs) with atomless initial state distributions and atomless transition probabilities. Such MDPs are called atomless. The initial state distribution is considered to be fixed. We show that for…
We study homeomorphisms of compact metric spaces whose restriction to the nonwandering set has the pseudo-orbit tracing property. We prove that if there are positively expansive measures, then the topological entropy is positive. Some short…