Related papers: P-adic Markov process and the problem of the first…
The representation theorem is obtained for functionals of non-Markov processes and their first exit times from bounded domains. These functionals are represented via solutions of backward parabolic Ito equations. As an example of…
We consider optimal stopping problems for a Brownian motion and a geometric Brownian motion with a "disorder", assuming that the moment of a disorder is uniformly distributed on a finite interval. Optimal stopping rules are found as the…
We derive a functional limit theorem for the partial maxima process based on a long memory stationary $\alpha$-stable process. The length of memory in the stable process is parameterized by a certain ergodic-theoretical parameter in an…
We study a model of active particles that perform a simple random walk and on top of that have a preferred direction determined by an internal state which is modelled by a stationary Markov process. First we calculate the limiting diffusion…
We investigate the classical active pure exploration problem in Markov Decision Processes, where the agent sequentially selects actions and, from the resulting system trajectory, aims at identifying the best policy as fast as possible. We…
We study observation-based strategies for partially-observable Markov decision processes (POMDPs) with omega-regular objectives. An observation-based strategy relies on partial information about the history of a play, namely, on the past…
We introduce a family of stochastic processes on the integers, depending on a parameter $p \in [0,1]$ and interpolating between the deterministic rotor walk (p=0) and the simple random walk (p=1/2). This p-rotor walk is not a Markov chain…
The reduction of a continuous Markov process with multiple metastable states to a discrete rate process is investigated in the presence of slow time dependent parameters such as periodic external forces or slowly fluctuating barrier…
The paper addresses the problem of computing maximal expected time to termination of probabilistic timed automata (PTA) models, under the condition that the system will, eventually, terminate. This problem can exhibit high computational…
Clock-dependent probabilistic timed automata extend classical timed automata with discrete probabilistic choice, where the probabilities are allowed to depend on the exact values of the clocks. Previous work has shown that the quantitative…
Let g : $\Omega$ = [0, 1] d $\rightarrow$ R denote a Lipschitz function that can be evaluated at each point, but at the price of a heavy computational time. Let X stand for a random variable with values in $\Omega$ such that one is able to…
In this paper, we establish a version of the central limit theorem for Markov-Feller continuous time processes (with a Polish state space) that are exponentially ergodic in the bounded-Lipschitz distance and enjoy a continuous form of the…
Be $X_t$ a random process starting at $x \in [0,1]$ with absorbing boundary conditions at both ends of the interval. Denote $P_1(x)$ the probability to first exit at the upper boundary. For Brownian motion, $P_1(x)=x$, equivalent to…
Based on the analysis of probability flow, where the First Passage (FP) is realised as the sink of probability, we summarise the protocol to find the distribution of the First Passage Time (FTP). We also describe the corresponding formula…
In this paper we analyze the cycle-to-cycle variations of maximum pressure $p_{max}$ and peak pressure angle $\alpha_{pmax}$ in a four-cylinder spark ignition engine. We examine the experimental time series of $p_{max}$ and $\alpha_{pmax}$…
In this paper we consider a (reflected) Brownian motion with broken drift hitting a random boundary. Some dedicated calculations allow us to obtain the formula on the joint Laplace transform of the hitting time and hitting position. These…
We study Markov processes where the "time" parameter is replaced by paths in a directed graph from an initial vertex to a terminal one. Along each directed path the process is Markov and has the same distribution as the one along any other…
We consider a Brownian particle diffusing in a one dimensional interval with absorbing end points. We study the ramifications when such motion is interrupted and restarted from the same initial configuration. We provide a comprehensive…
We consider a two-agent MDP framework where agents repeatedly solve a task in a collaborative setting. We study the problem of designing a learning algorithm for the first agent (A1) that facilitates a successful collaboration even in cases…
Recent work on approximate linear programming (ALP) techniques for first-order Markov Decision Processes (FOMDPs) represents the value function linearly w.r.t. a set of first-order basis functions and uses linear programming techniques to…