Related papers: Limit Behaviour of Upper and Lower Expected Time A…
We study the limit behaviour of upper and lower bounds on expected time averages in imprecise Markov chains; a generalised type of Markov chain where the local dynamics, traditionally characterised by transition probabilities, are now…
We justify and discuss expressions for joint lower and upper expectations in imprecise probability trees, in terms of the sub- and supermartingales that can be associated with such trees. These imprecise probability trees can be seen as…
This paper is concerned with the development of rigorous approximations to various expectations associated with Markov chains and processes having non-stationary transition probabilities. Such non-stationary models arise naturally in…
When the initial and transition probabilities of a finite Markov chain in discrete time are not well known, we should perform a sensitivity analysis. This can be done by considering as basic uncertainty models the so-called credal sets that…
We study the limit behaviour of a generally non-linear ordinary differential equation whose solution is a superadditive generalisation of a stochastic matrix, and provide necessary and sufficient conditions for this solution to be ergodic,…
Time estimation is a fundamental task that underpins precision measurement, global navigation systems, financial markets, and the organisation of everyday life. Many biological processes also depend on time estimation by nanoscale clocks,…
We consider the problem of performing inference with imprecise continuous-time hidden Markov chains, that is, imprecise continuous-time Markov chains that are augmented with random output variables whose distribution depends on the hidden…
We present a novel algorithm to solve a non-linear system of equations, whose solution can be interpreted as a tight lower bound on the vector of expected hitting times of a Markov chain whose transition probabilities are only partially…
Continuous-time Markov chains are mathematical models that are used to describe the state-evolution of dynamical systems under stochastic uncertainty, and have found widespread applications in various fields. In order to make these models…
We provide a criterion for establishing lower bounds on the rate of convergence in $f$-variation of a continuous-time ergodic Markov process to its invariant measure. The criterion consists of novel super- and submartingale conditions for…
We study upper and lower bounds on the sample-complexity of learning near-optimal behaviour in finite-state discounted Markov Decision Processes (MDPs). For the upper bound we make the assumption that each action leads to at most two…
We study Markov processes conditioned so that their local time must grow slower than a prescribed function. Building upon recent work on Brownian motion with constrained local time in [5] and [33], we study transience and recurrence for a…
We study one-sided and $\alpha$-correct sequential hypothesis testing for data generated by an ergodic Markov chain. The null hypothesis is that the unknown transition matrix belongs to a prescribed set $P$ of stochastic matrices, and the…
We analyse the problem of meeting times for interdependent stochastic agents: random walkers whose behaviour is stochastic but controlled by their selections from some set of allowed actions, and the inference problem of when these agents…
We study the problem of characterizing the expected hitting times for a robust generalization of continuous-time Markov chains. This generalization is based on the theory of imprecise probabilities, and the models with which we work…
We study a limit behavior of a sequence of Markov processes (or Markov chains) such that their distributions outside of any neighborhood of a "singular" point attract to some probability law. In any neighborhood of this point the behavior…
Lifted Markov chains are Markov chains on graphs with added local "memory" and can be used to mix towards a target distribution faster than their memoryless counterparts. Upper and lower bounds on the achievable performance have been…
We provide a general framework for computing upper bounds on mixing times of finite Markov chains when its minimal ideal is left zero. Our analysis is based on combining results by Brown and Diaconis with our previous work on stationary…
In this paper, the question of expected time to convergence is addressed for unbiased quantized consensus on undirected connected graphs, and some strong results are obtained. The paper first provides a tight expression for the expected…
The paper is devoted to studies of perturbed Markov chains commonly used for description of information networks. In such models, the matrix of transition probabilities for the corresponding Markov chain is usually regularised by adding a…