相关论文: Markovian explorations of random planar maps are r…

The key assumption underlying linear Markov Decision Processes (MDPs) is that the learner has access to a known feature map $\phi(x, a)$ that maps state-action pairs to $d$-dimensional vectors, and that the rewards and transitions are…

Machine learning techniques not only offer efficient tools for modelling dynamical systems from data, but can also be employed as frontline investigative instruments for the underlying physics. Nontrivial information about the original…

We study Bernoulli percolations on random lattices of the half-plane obtained as local limit of uniform planar triangulations or quadrangulations. Using the characteristic spatial Markov property or peeling process of these random lattices…

This article provides an introductory tutorial on structural results in partially observed Markov decision processes (POMDPs). Typically, computing the optimal policy of a POMDP is computationally intractable. We use lattice program- ming…

While one-dimensional Markov processes are well understood, going to higher dimensions there are only a few analytically solved Ising-like models, in practice requiring to use relatively costly, uncontrollable and inaccurate Monte-Carlo…

In this paper, we study Markovian random iterations of maps on standard measurable spaces. We establish a one-to-one correspondence between stationary measures and a certain class of invariant measures of a Markovian random iteration,…

Granular convergence is a property of a granular pack as it is repeatedly sheared in a cyclic, quasistatic fashion, as the packing configuration changes via discrete events. Under suitable conditions the set of microscopic configurations…

We establish a new spatial Markov property of the Brownian half-plane. According to this property, if one removes a hull centered at a boundary point, the remaining space equipped with an intrinsic metric is still a Brownian half-plane,…

We propose a safe exploration algorithm for deterministic Markov Decision Processes with unknown transition models. Our algorithm guarantees safety by leveraging Lipschitz-continuity to ensure that no unsafe states are visited during…

Many environments, such as unvisited planetary surfaces and oceanic regions, remain unexplored due to a lack of prior knowledge. Autonomous vehicles must sample upon arrival, process data, and either transmit findings to a teleoperator or…

We consider a Markov evolution of lozenge tilings of a quarter-plane and study its asymptotics at large times. One of the boundary rays serves as a reflecting wall. We observe frozen and liquid regions, prove convergence of the local…

We consider a Markovian evolution on point processes, the $\Psi$--process, on the unit interval in which points are added according to a rule that depends only on the spacings of the existing point configuration. Having chosen a spacing, a…

We study the loop clusters induced by Poissonian ensembles of Markov loops on a finite or countable graph (Markov loops can be viewed as excursions of Markov chains with a random starting point, up to re-rooting). Poissonian ensembles are…

Random planar maps are considered in the physics literature as the discrete counterpart of random surfaces. It is conjectured that properly rescaled random planar maps, when conditioned to have a large number of faces, should converge to a…

Most exact algorithms for general partially observable Markov decision processes (POMDPs) use a form of dynamic programming in which a piecewise-linear and convex representation of one value function is transformed into another. We examine…

We consider the time dependent probability distribution of a coarse grained observable Y whose evolution is governed by a discrete time map. If the map is mixing, the time dependent one-step transition probabilities converge in the long…

For a spatial characteristic, there exist commonly fat-tail frequency distributions of fragment-size and -mass of glass, areas enclosed by city roads, and pore size/volume in random packings. In order to give a new analytical approach for…

In this work, we study dynamic programming (DP) algorithms for partially observable Markov decision processes with jointly continuous and discrete state-spaces. We consider a class of stochastic systems which have coupled discrete and…

This paper introduces a new technique for learning probabilistic models of mass and friction distributions of unknown objects, and performing robust sliding actions by using the learned models. The proposed method is executed in two…

We present an explicit construction of a Markovian random growth process on integer partitions such that given it visits some level $n$, it passes through any partition $\lambda$ of $n$ with equal probabilities. The construction has…