Related papers: Dense periodic optimization for countable Markov s…
This paper is devoted to studying constrained continuous-time Markov decision processes (MDPs) in the class of randomized policies depending on state histories. The transition rates may be unbounded, the reward and costs are admitted to be…
By a result of Blokh from 1984, every transitive map of a tree has the relative specification property, and so it has finite decomposition ideal, positive entropy and dense periodic points. In this paper we construct a transitive dendrite…
A method for the construction of approximate analytical expressions for the stationary marginal densities of general stochastic search processes is proposed. By the marginal densities, regions of the search space that with high probability…
In this article we show how ideas, methods and results from optimal transportation can be used to study various aspects of the stationary measuresof Iterated Function Systems equipped with a probability distribution. We recover a classical…
We study discrete-time discounted constrained Markov decision processes (CMDPs) on Borel spaces with unbounded reward functions. In our approach the transition probability functions are weakly or set-wise continuous. The reward functions…
This note outlines a mean-field approach to dynamic optimal transport problems based on the recently proposed McKean-Pontryagin maximum principle. Key aspects of the proposed methodology include i) avoidance of sampling over stochastic…
We introduce a new class of objectives for optimal transport computations of datasets in high-dimensional Euclidean spaces. The new objectives are parametrized by $\rho \geq 1$, and provide a metric space $\mathcal{R}_{\rho}(\cdot, \cdot)$…
We define a Markov process on the set of countable graphs with spins. Transitions are local substitutions in the graph. It is proved that the scaling macrodimension is an invariant of such dynamics.
We establish multi-scale convergence theory for a class of Hamilton-Jacobi PDEs in space of probability measures. They arise from context of hydrodynamic limit of N-particle deterministic action minimizing (global) Lagrangian dynamics. From…
In this paper, we consider the gradual-impulse control problem of continuous-time Markov decision processes, where the system performance is measured by the expectation of the exponential utility of the total cost. We prove, under very…
We present a class of Gauss-Markov processes which can be represented as space-time scaled stationary Ornstein-Uhlenbeck processes defined on the real line. We give several explicit examples of the representation for certain Gauss bridge…
We consider move-making algorithms for energy minimization of multi-label Markov Random Fields (MRFs). Since this is not a tractable problem in general, a commonly used heuristic is to minimize over subsets of labels and variables in an…
We study quantitative recurrence to rare events in Countable Markov Shifts with recurrent potentials, focusing on return-time statistics to natural target sets for every point. In the positive recurrent case, return-time processes…
We propose a method for approximating solutions to optimization problems involving the global stability properties of parameter-dependent continuous-time autonomous dynamical systems. The method relies on an approximation of the…
This paper concerns computation of optimal policies in which the one-step reward function contains a cost term that models Kullback-Leibler divergence with respect to nominal dynamics. This technique was introduced by Todorov in 2007, where…
We consider the problem of approximating the reachability probabilities in Markov decision processes (MDP) with uncountable (continuous) state and action spaces. While there are algorithms that, for special classes of such MDP, provide a…
We investigate the systematic mechanism for designing fast mixing Markov chain Monte Carlo algorithms to sample from discrete point processes under the Dobrushin uniqueness condition for Gibbs measures. Discrete point processes are defined…
We consider a general honest homogeneous continuous-time Markov process with restarts. The process is forced to restart from a given distribution at time moments generated by an independent Poisson process. The motivation to study such…
The convergence of simultaneous and marginal predictive classifiers under partition exchangeability in supervised classification is obtained. The result shows the asymptotic convergence of these classifiers under infinite amount of training…
Due to their conjugate posteriors, Gaussian process priors are attractive for estimating the drift of stochastic differential equations with continuous time observations. However, their performance strongly depends on the choice of the…