Related papers: A temporal factorization at the maximum for spectr…
We establish integral tests and laws of the iterated logarithm for the upper envelope of the future infimum of positive self-similar Markov processes and for increasing self-similar Markov processes at 0 and infinity. Our proofs are based…
We study the problem of synthesizing a policy that maximizes the entropy of a Markov decision process (MDP) subject to a temporal logic constraint. Such a policy minimizes the predictability of the paths it generates, or dually, maximizes…
We study infinite-horizon Markov decision processes (MDPs) where the decision maker evaluates each of her strategies by aggregating the infinite stream of expected stage-rewards. The crucial feature of our approach is that the aggregation…
In this paper, we study discrete-time absorbing Markov Decision Processes (MDP) with measurable state space and Borel action space with a given initial distribution. For such models, solutions to the characteristic equation that are not…
This paper deals with the optimal stopping problem under partial observation for piecewise-deterministic Markov processes. We first obtain a recursive formulation of the optimal filter process and derive the dynamic programming equation of…
Path decomposition is performed to analyze the pre-supremum, post-supremum, post-infimum and the intermediate processes of a spectrally negative Levy process taken up to an independent exponential time T as motivated by the aim of finding…
We consider the evolution of Green's function of the one-dimensional Schr\"odinger equation in the presence of the complex potential $-ik\delta(x)$. Our result is the construction of an explicit time-dependent solution which we use to…
Consider the continuous-time Markov Branching Process. In critical case we consider a situation when the generating function of intensity of transformation of particles has the infinite second moment, but its tail regularly varies in sense…
We consider a strong Markov process with killing and prove an approximation method for the distribution of the process conditioned not to be killed when it is observed. The method is based on a Fleming-Viot type particle system with…
Canonical models of Markov decision processes (MDPs) usually consider geometric discounting based on a constant discount factor. While this standard modeling approach has led to many elegant results, some recent studies indicate the…
Gabor analysis is one of the most common instances of time-frequency signal analysis. Choosing a suitable window for the Gabor transform of a signal is often a challenge for practical applications, in particular in audio signal processing.…
In this article we get simple explicit formulas for $\Exp\sup_{s\leq t}X(s)$ where $X$ is a spectrally positive or negative L\'evy process with infinite variation. As a consequence we derive a generalization of the well-known formula for…
The theory of ``Markov-up'' processes is being developed. This is a new class of stochastic processes with ``partial'' markovian features; it could also be called ``one-sided Markov''. Such a behavior may be found in the real world and in…
We study the long time behaviour of a Markov process evolving in $\mathbb{N}$ and conditioned not to hit 0. Assuming that the process comes back quickly from infinity, we prove that the process admits a unique quasi-stationary distribution…
We study a Markov decision problem in which the state space is the set of finite marked point configurations in the plane, the actions represent thinnings, the reward is proportional to the mark sum which is discounted over time, and the…
Consider a continuous time Markov chain with rates Q in the state space \Lambda\cup\{0\} with 0 as an absorbing state. In the associated Fleming-Viot process N particles evolve independently in \Lambda with rates Q until one of them…
We consider periodic Markov chains with absorption. Applying to iterates of this periodic Markov chain criteria for the exponential convergence of conditional distributions of aperiodic absorbed Markov chains, we obtain exponential…
This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…
Using the Wiener-Hopf factorization, it is shown that it is possible to bound the path of an arbitrary Levy process above and below by the paths of two random walks. These walks have the same step distribution, but different random starting…
We study the mean first passage time (MFPT) to an absorbing target of a one-dimensional Brownian particle subject to an external potential $v(x)$ in a finite domain. We focus on the cases in which the external potential is confining, of the…