相关论文: Markovianity in space and time
We consider the synthesis of control policies for probabilistic systems, modeled by Markov decision processes, operating in partially known environments with temporal logic specifications. The environment is modeled by a set of Markov…
Our main purpose here is to study some qualitative aspects of space and time. These include the notion of space and time regarded as the containers of respectively bodies and events, the divisibility of space, and the unrepeatability of…
We consider processes which are functions of finite-state Markov chains. It is well known that such processes are rarely Markov. However, such processes are often regular in the following sense: the distant past values of the process have…
Let $P$ be the transition matrix of a finite, irreducible and reversible Markov chain. We say the continuous time Markov chain $X$ has transition matrix $P$ and speed $\lambda$ if it jumps at rate $\lambda$ according to the matrix $P$. Fix…
Markov automata combine non-determinism, probabilistic branching, and exponentially distributed delays. This compositional variant of continuous-time Markov decision processes is used in reliability engineering, performance evaluation and…
Motivated by applications in telecommunications, computer scienceand physics, we consider a discrete-time Markov process withrestart. At each step the process eitherwith a positive probability restarts from a given distribution, orwith the…
We consider Markov chains on partially ordered sets that generalize the success-runs and remaining life chains in reliability theory. We find conditions for recurrence and transience and give simple expressions for the invariant…
The interaction between an open quantum system and its environment induces generally memory effects generated by the fact that the response of the system to the environment is not instantaneous. Different physical reasons can be at the…
Motivated by the dynamic modeling of relative abundance data in ecology, we introduce a general approach to model stationary Markovian or non Markovian time series on (relatively) compact spaces such as a hypercube, the simplex or a sphere…
We explore two notions of stationary processes. The first is called a random-step Markov process in which the stationary process of states, $(X_i)_{i \in \mathbb{Z}}$ has a stationary coupling with an independent process on the positive…
Circular data arise in many areas of application. Recently, there has been interest in looking at circular data collected separately over time and over space. Here, we extend some of this work to the spatio-temporal setting, introducing…
We formulate a probabilistic Markov property in discrete time under a dynamic risk framework with minimal assumptions. This is useful for recursive solutions to risk-sensitive versions of dynamic optimisation problems such as optimal…
The core of generalization theory was developed for independent observations. Some PAC and PAC-Bayes bounds are available for data that exhibit a temporal dependence. However, there are constants in these bounds that depend on properties of…
Many complex systems exhibit interactions that depend not only on pairwise connections, but also group structures and memory effects. To capture such effects, we develop a unified tensor framework for modeling higher-order Markov chains…
Although the notion of diagnostic problem has been extensively investigated in the context of static systems, in most practical applications the behavior of the modeled system is significantly variable during time. The goal of the paper is…
The method of 'coupling from the past' permits exact sampling from the invariant distribution of a Markov chain on a finite state space. The coupling is successful whenever the stochastic dynamics are such that there is coalescence of all…
Philosophers now seem to agree that frequentism is an untenable strategy to explain the meaning of probabilities. Nevertheless, I want to revive frequentism, and I will do so by grounding probabilities on typicality in the same way as the…
A time-dependent finite-state Markov chain that uses doubly stochastic transition matrices, is considered. Entropic quantities that describe the randomness of the probability vectors, and also the randomness of the discrete paths, are…
Potential theory has important applications in various fields such as physics, finance, and biology. In this paper, we investigate the potentials of two classic types of discrete-time skip-free Markov chains: upward skip-free and downward…
We introduce a statistical mechanics formalism for the study of constrained graph evolution as a Markovian stochastic process, in analogy with that available for spin systems, deriving its basic properties and highlighting the role of the…