Related papers: The compositional construction of Markov processes
We develop a full-fledged analysis of an algorithmic decision process that, in a multialternative choice problem, produces computable choice probabilities and expected decision times.
By combining well-known techniques from both noncommutative algebra and computational commutative algebra, we observe that an algorithmic approach can be applied to the study of irreducible representations of finitely presented algebras. In…
In previous work, summarized in this paper, we proposed an operation of parallel composition for rewriting-logic theories, allowing compositional specification of systems and reusability of components. The present paper focuses on…
The transition structure of an automaton can be used to create a natural topology to the set of states of an automaton, generating, this way, a topological space. Probabilistic automata can also be modeled in terms of measure theory. A…
In this paper, we develop a Topological Approximate Dynamic Programming (TADP) method for planningin stochastic systems modeled as Markov Decision Processesto maximize the probability of satisfying high-level systemspecifications expressed…
Model checking probabilistic CTL properties of Markov decision processes with convex uncertainties has been recently investigated by Puggelli et al. Such model checking algorithms typically suffer from the state space explosion. In this…
We construct a hierarchy of regular languages such that the current language in the hierarchy can be accepted by 1-way quantum finite automata with a probability smaller than the corresponding probability for the preceding language in the…
We consider killed Markov decision processes for countable models on a finite time-interval. Existence of a uniform $\varepsilon$-optimal policy is proven. We show the correctness of the fundamental equation. The optimal control problem is…
The need for rigorous process composition is encountered in many situations pertaining to the development and analysis of complex systems. We discuss the use of Classical Linear Logic (CLL) for correct-by-construction resource-based process…
A selection of the relevant theorems of Probability Theory that comes directly from Kolmogorov's axioms, Set Theory basic results, definitions and rules of inference are listed and proven in a systematic approach, aiming the student who…
We study K-processes, which are Markov processes in a denumerable state space, all of whose elements are stable, with the exception of a single state, starting from which the process enters finite sets of stable states with uniform…
Markov decision processes model systems subject to nondeterministic and probabilistic uncertainty. A plethora of verification techniques addresses variations of reachability properties, such as: Is there a scheduler resolving the…
We review and extend the formalism introduced by Peliti, that maps a Markov process to a path-integral representation. After developing the mapping, we apply it to some illustrative examples: the simple decay process, the birth-and-death…
The asymptotic study of the conjugacy classes of a random element of the finite affine group leads one to define a probability measure on the set of all partitions of all positive integers. Four different probabilistic understandings of…
The binomial, the negative binomial, the Poisson, the compound Poisson and the Erlang distribution do all admit integral representations with respect to its (continuous) parameter. We use the Margulis-Russo type formulas for Bernoulli and…
This paper presents efficient algorithms for testing the finite, polynomial, and exponential ambiguity of finite automata with $\epsilon$-transitions. It gives an algorithm for testing the exponential ambiguity of an automaton $A$ in time…
This survey article gives an elementary introduction to the algebraic approach to Markov process duality, as opposed to the pathwise approach. In the algebraic approach, a Markov generator is written as the sum of products of simpler…
In a previous paper, a process algebra based on ACP (Algebra of Communicating Processes) was proposed in which processes involving data can be handled by means of features originating from imperative programming. In this paper, an extension…
In the paper we consider some piecewise deterministic Markov process whose continuous component evolves according to semiflows, which are switched at the jump times of a Poisson process. The associated Markov chain describes the states of…
We propose a numerical method to approximate the value function for the optimal stopping problem of a piecewise deterministic Markov process (PDMP). Our approach is based on quantization of the post jump location---inter-arrival time Markov…