Related papers: Approximating Acceptance Probabilities of CTMC-Pat…

In the present paper, we investigate the relationship between hitting times and hitting probabilities in discrete-time imprecise Markov chains (IMCs). We define lower and upper hitting times and probabilities for IMCs whose set of…

A new approach to the steady state detection in the uniformization method of solving continuous time Markov chains is introduced. The method is particularly useful in solving inhomogenous CTMC's in multiple steps, where the desired error…

We consider the problem of characterising expected hitting times and hitting probabilities for imprecise Markov chains. To this end, we consider three distinct ways in which imprecise Markov chains have been defined in the literature: as…

In parameter estimation problems one computes a posterior distribution over uncertain parameters defined jointly by a prior distribution, a model, and noisy data. Markov Chain Monte Carlo (MCMC) is often used for the numerical solution of…

The paper addresses two variants of the stochastic shortest path problem ('optimize the accumulated weight until reaching a goal state') in Markov decision processes (MDPs) with integer weights. The first variant optimizes partial expected…

We propose a method based on continuous time Markov chain approximation to compute the distribution of Parisian stopping times and price Parisian options under general one-dimensional Markov processes. We prove the convergence of the method…

The Markov chain Monte Carlo (MCMC) method is used to evaluate the imaginary-time path integral of a quantum oscillator with a potential that includes both a quadratic term and a quartic term whose coupling is varied by several orders of…

We revisit the classical problem of approximating a stochastic differential equation by a discrete-time and discrete-space Markov chain. Our construction iterates Caratheodory's theorem over time to match the moments of the increments…

Deterministic timed automata are strictly less expressive than their non-deterministic counterparts, which are again less expressive than those with silent transitions. As a consequence, timed automata are in general non-determinizable.…

In this paper, we consider a model of generalized timed automata (GTA) with two kinds of clocks, history and future, that can express many timed features succinctly, including timed automata, event-clock automata with and without diagonal…

This paper develops general approaches for pricing various types of American-style Parisian options (down-in/-out, perpetual/finite-maturity) with general payoff functions based on continuous-time Markov chain (CTMC) approximation under…

We come up with a class of distributed quantized averaging algorithms on asynchronous communication networks with fixed, switching and random topologies. The implementation of these algorithms is subject to the realistic constraint that the…

Autonomous driving systems, critical for safety, require real-time guarantees and can be modeled as DAGs. Their acceleration features, such as caches and pipelining, often result in execution times below the worst-case. Thus, a…

Computing optimal conditional reachability probabilities in Markov decision processes (MDPs) is tractable by a reduction to reachability probabilities. Yet, this reduction yields cyclic, challenging MDPs that are often notoriously hard to…

This paper considers a class of probabilistic cellular automata undergoing a phase transition with an absorbing state. Denoting by ${\mathcal{U}}(x)$ the neighbourhood of site $x$, the transition probability is $T(\eta_x = 1 |…

Given a Probabilistic Finite Automata (PFA), a set of states S, and an error threshold e > 0, our algorithm approximates the infimum probability (quantifying over all infinite words) that the automata reaches S. Our result contrasts with…

Dynamic Programming suffers from the curse of dimensionality due to large state and action spaces, a challenge further compounded by uncertainties in the environment. To mitigate these issue, we explore an off-policy based Temporal…

Continuous-time Markov chains with alarms (ACTMCs) allow for alarm events that can be non-exponentially distributed. Within parametric ACTMCs, the parameters of alarm-event distributions are not given explicitly and can be subject of…

This paper proposes a distributed model predicted control (DMPC) approach for consensus control of multi-agent systems (MASs) with linear agent dynamics and bounded control input constraints. Within the proposed DMPC framework, each agent…

We study the reachability problem for communicating timed processes, both in discrete and dense time. Our model comprises automata with local timing constraints communicating over unbounded FIFO channels. Each automaton can only access its…