相关论文: Fluctuation identities for omega-killed Markov add…

The master equation and, more generally, Markov processes are routinely used as models for stochastic processes. They are often justified on the basis of randomization and coarse-graining assumptions. Here instead, we derive n-th order…

Stochastic domains often involve risk-averse decision makers. While recent work has focused on how to model risk in Markov decision processes using risk measures, it has not addressed the problem of solving large risk-averse formulations.…

An important problem in applied dynamical systems is to compute the external forcing that provokes the largest response of a desired observable quantity. For this, we investigate the perturbation theory of Markov matrices in connection with…

We introduce a general framework for Markov decision problems under model uncertainty in a discrete-time infinite horizon setting. By providing a dynamic programming principle we obtain a local-to-global paradigm, namely solving a local,…

This paper discusses algorithms for solving Markov decision processes (MDPs) that have monotone optimal policies. We propose a two-stage alternating convex optimization scheme that can accelerate the search for an optimal policy by…

Learning Markov decision processes (MDP) in an adversarial environment has been a challenging problem. The problem becomes even more challenging with function approximation, since the underlying structure of the loss function and transition…

Previous authors have considered optimal stopping problems driven by the running maximum of a spectrally negative L\'evy process $X$, as well as of a one-dimensional diffusion. Many of the aforementioned results are either implicitly or…

It is known that the distribution of nonreversible Markov processes breaking the detailed balance condition converges faster to the stationary distribution compared to reversible processes having the same stationary distribution. This is…

This paper presents with justifications a technique that is useful for the study of piecewise deterministic Markov decision processes (PDMDPs) with general policies and unbounded transition intensities. This technique produces an auxiliary…

We investigate partially observed Markov decision processes (POMDPs) with cost functions regularized by entropy terms describing state, observation, and control uncertainty. Standard POMDP techniques are shown to offer bounded-error…

We consider active maximum a posteriori (MAP) inference problem for Hidden Markov Models (HMM), where, given an initial MAP estimate of the hidden sequence, we select to label certain states in the sequence to improve the estimation…

In this paper, we address the numerical solution to the multimarginal optimal transport (MMOT) with pairwise costs. MMOT, as a natural extension from the classical two-marginal optimal transport, has many important applications including…

In this paper, we have studied option pricing methods that are based on a Bayesian Markov-Switching Vector Autoregressive (MS-BVAR) process using a risk-neutral valuation approach. A BVAR process, which is a special case of the Bayesian…

A systematic exposition of scale functions is given for positive self-similar Markov processes (pssMp) with one-sided jumps. The scale functions express as convolution series of the usual scale functions associated with spectrally one-sided…

Birth and death Markov processes can model stochastic physical systems from percolation to disease spread and, in particular, wildfires. We introduce and analyze a birth-death-suppression Markov process as a model of controlled culling of…

In the present paper, we investigate the optimal capital injection behaviour of an insurance company if the interest rate is allowed to become negative. The surplus process of the considered insurance entity is assumed to follow a Brownian…

The extinction probability of the Markovian Binary Tree (MBT) is the minimal nonnegative solution of a Quadratic Vector Equation (QVE). In this paper, we present a perturbation analysis for the extinction probability of a supercritical MBT.…

In this paper we study a reflected Markov-modulated Brownian motion with a two sided reflection in which the drift, diffusion coefficient and the two boundaries are (jointly) modulated by a finite state space irreducible continuous time…

We use a classical combinatorial inequality to establish a Markov inequality for multivariate binary Markov processes on trees. We then apply this result, alongside with the FKG inequality, to compare the expected loss of biodiversity under…

Bifurcating Markov chains (BMC) are Markov chains indexed by a full binary tree representing the evolution of a trait along a population where each individual has two children. We provide a central limit theorem for additive functionals of…