Related papers: On structural parameter estimation of the Markov Q…
We develop a martingale approximation approach to studying the limiting behavior of quadratic forms of Markov chains. We use the technique to examine the asymptotic behavior of lag-window estimators in time series and we apply the results…
This article introduces a non parametric warping model for functional data. When the outcome of an experiment is a sample of curves, data can be seen as realizations of a stochastic process, which takes into account the small variations…
Kramers-Moyal coefficients provide a simple and easily visualized method with which to analyze stochastic time series, particularly nonlinear ones. One mechanism that can affect the estimation of the coefficients is geometric projection…
In this paper, we present a methodology to estimate the parameters of stochastically contaminated models under two contamination regimes. In both regimes, we assume that the original process is a variable length Markov chain that is…
Markov Chains with variable length are useful stochastic models for data compression that avoid the curse of dimensionality faced by that full Markov Chains. In this paper we introduce a Variable Length Markov Chain whose transition…
In the paper we consider some piecewise deterministic Markov process whose continuous component evolves according to semiflows, which are switched at the jump times of a Poisson process. The associated Markov chain describes the states of…
Weconsider Markov decision processes arising from a Markov model of an underlying natural phenomenon. Such phenomena are usually periodic (e.g. annual) in time, and so the Markov processes modelling them must be time-inhomogeneous, with…
In this paper we introduce a novel online time series forecasting model we refer to as the pM-GP filter. We show that our model is equivalent to Gaussian process regression, with the advantage that both online forecasting and online…
We consider the problem of approximating the stationary distribution of an ergodic Markov chain given a set of sampled transitions. Classical simulation-based approaches assume access to the underlying process so that trajectories of…
We consider a stationary Markov process that models certain queues with a bulk service of a fixed number $m$ of admitted customers. We find an integral expression of its transition probability function in terms of certain multi-orthogonal…
In this work, we present a general method to establish properties of multi-dimensional continuous-time Markov chains representing stochastic reaction networks. This method consists of grouping states together (via a partition of the state…
We study the problem of characterizing the expected hitting times for a robust generalization of continuous-time Markov chains. This generalization is based on the theory of imprecise probabilities, and the models with which we work…
This article provides an introductory tutorial on structural results in partially observed Markov decision processes (POMDPs). Typically, computing the optimal policy of a POMDP is computationally intractable. We use lattice program- ming…
In this paper, we study one dimensional Markov processes with spatial delay. Since the seminal work of Feller, we know that virtually any one dimensional, strong, homogeneous, continuous Markov process can be uniquely characterized via its…
We consider a hidden Markov model, where the signal process, given by a diffusion, is only indirectly observed through some noisy measurements. The article develops a variational method for approximating the hidden states of the signal…
We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of random variables. The probability distributions for these random parameters are…
Many random processes can be simulated as the output of a deterministic model accepting random inputs. Such a model usually describes a complex mathematical or physical stochastic system and the randomness is introduced in the input…
The generic identification problem is to decide whether a stochastic process $(X_t)$ is a hidden Markov process and if yes to infer its parameters for all but a subset of parametrizations that form a lower-dimensional subvariety in…
We propose deterministic timed automata (DTA) as a model-independent language for specifying performance and dependability measures over continuous-time stochastic processes. Technically, these measures are defined as limit frequencies of…
We present a systematic analysis of stochastic processes conditioned on an empirical measure $Q_T$ defined in a time interval $[0,T]$ for large $T$. We build our analysis starting from a discrete time Markov chain. Results for a continuous…