Related papers: Some thoughts on multiparameter stochastic process…
Additive processes are obtained from L\'{e}vy ones by relaxing the condition of stationary increments, hence they are spatially (but not temporally) homogeneous. By analogy with the case of time-homogeneous Markov processes, one can define…
A special class of doubly stochastic (Markov) operators is constructed. These operators come from measure preserving transformations and inherit some of their properties, namely ergodicity and positivity of entropy, yet they may have no…
A labelled Markov decision process is a labelled Markov chain with nondeterminism, i.e., together with a strategy a labelled MDP induces a labelled Markov chain. The model is related to interval Markov chains. Motivated by applications of…
Frequently, clinical trials and observational studies involve complex event history data with multiple events. When the observations are independent, the analysis of such studies can be based on standard methods for multi-state models.…
Interval Markov decision processes (IMDPs) generalise classical MDPs by having interval-valued transition probabilities. They provide a powerful modelling tool for probabilistic systems with an additional variation or uncertainty that…
This note summarizes the optimization formulations used in the study of Markov decision processes. We consider both the discounted and undiscounted processes under the standard and the entropy-regularized settings. For each setting, we…
We propose a generalization of the classical M/M/1 queue process. The resulting model is derived by applying fractional derivative operators to a system of difference-differential equations. This generalization includes both non-Markovian…
The purpose of this note is to review some recent results concerning the pseudospectra and the eigenvalues asymptotics of non-selfadjoint semiclassical pseudo-differential operators subject to small random perturbations.
This primer explains how continuous-time stochastic processes (precisely, Brownian motion and other Ito diffusions) can be defined and studied on manifolds. No knowledge is assumed of either differential geometry or continuous-time…
Given a semi-Markov law, using an additional parameter, we consider a family of stochastic flows corresponding to that law. Then we suitably select a particular flow, for which we obtain expressions of the meeting and merging probabilities…
The concepts of probability, statistics and stochastic theory are being successfully used in structural engineering. Markov Chain modelling is a simple stochastic process model that has found its application in both describing stochastic…
Scientific inference involves obtaining the unknown properties or behavior of a system in the light of what is known, typically, without changing the system. Here we propose an alternative to this approach: a system can be modified in a…
Motivated by reduction of computational complexity, this work develops sign-error adaptive filtering algorithms for estimating time-varying system parameters. Different from the previous work on sign-error algorithms, the parameters are…
Hidden Markov models have successfully been applied as models of discrete time series in many fields. Often, when applied in practice, the parameters of these models have to be estimated. The currently predominating identification methods,…
We introduce and test an algorithm that adaptively estimates large deviation functions characterizing the fluctuations of additive functionals of Markov processes in the long-time limit. These functions play an important role for predicting…
We extend the theory of labeled Markov processes with internal nondeterminism, a fundamental concept for the further development of a process theory with abstraction on nondeterministic continuous probabilistic systems. We define…
Consider a one-sided Markov additive process with an upper and a lower barrier, where each can be either reflecting or terminating. For both defective and non-defective processes and all possible scenarios we identify the corresponding…
We present a novel deep learning method for estimating time-dependent parameters in Markov processes through discrete sampling. Departing from conventional machine learning, our approach reframes parameter approximation as an optimization…
This paper presents a general approach to linear stochastic processes driven by various random noises. Mathematically, such processes are described by linear stochastic differential equations of arbitrary order (the simplest non-trivial…
A parameter estimation problem is considered for a stochastic parabolic equation with multiplicative noise under the assumption that the equation can be reduced to an infinite system of uncoupled diffusion processes. From the point of view…