Related papers: Numerical method for impulse control of Piecewise …
We consider stochastic impulse control problems where the process is driven by a general one-dimensional diffusion. We shall show a new mathematical characterization of the value function as a linear function in a certain transformed space.…
We study a single risky financial asset model subject to price impact and transaction cost over an finite time horizon. An investor needs to execute a long position in the asset affecting the price of the asset and possibly incurring in…
Partially observable Markov decision processes (POMDPs) have recently become popular among many AI researchers because they serve as a natural model for planning under uncertainty. Value iteration is a well-known algorithm for finding…
This paper considers a distributed decision-making approach for manufacturing task assignment and condition-based machine health maintenance. Our approach considers information sharing between the task assignment and health management…
Among various rare events, the effective computation of transition paths connecting metastable states in a stochastic model is an important problem. This paper proposes a stochastic optimal control formulation for transition path problems…
This article deals with stochastic processes endowed with the Markov (memoryless) property and evolving over general (uncountable) state spaces. The models further depend on a non-deterministic quantity in the form of a control input, which…
The paper is concerned with a zero-sum continuous-time stochastic differential game with a dynamics controlled by a Markov process and a terminal payoff. The value function of the original game is estimated using the value function of a…
We investigate piecewise deterministic Markov processes (PDMP), where the deterministic dynamics follows a scalar conservation law and random jumps in the system are characterized by changes in the flux function. We show under which…
We investigate the Poisson regression method for Markov and semi-Markov jump processes from a nonparametric angle, allowing the lengths of the time and duration intervals in the partition to vary with the number of observations. Imposing no…
We consider an infinite horizon optimal control problem for a pure jump Markov process $X$, taking values in a complete and separable metric space $I$, with noise-free partial observation. The observation process is defined as $Y_t =…
In this paper we provide faster algorithms for approximately solving discounted Markov Decision Processes in multiple parameter regimes. Given a discounted Markov Decision Process (DMDP) with $|S|$ states, $|A|$ actions, discount factor…
In this paper, we study the distributionally robust joint chance constrained Markov decision process. {Utilizing the logarithmic transformation technique,} we derive its deterministic reformulation with bi-convex terms under the…
In this paper we consider the numerical solutions for a class of jump diffusions with Markovian switching. After briefly reviewing necessary notions, a new jump-adapted efficient algorithm based on the Euler scheme is constructed for…
The paper proposes a class of financial market models which are based on inhomogeneous telegraph processes and jump diffusions with alternating volatilities. It is assumed that the jumps occur when the tendencies and volatilities are…
We consider the Monte-Carlo first visit algorithm, of which the goal is to find the optimal control in a Markov decision process with finite state space and finite number of possible actions. We show its convergence when the discount factor…
We consider an investor faced with the utility maximization problem in which the risky asset price process has pure-jump dynamics affected by an unobservable continuous-time finite-state Markov chain, the intensity of which can also be…
A stochastic incremental subgradient algorithm for the minimization of a sum of convex functions is introduced. The method sequentially uses partial subgradient information and the sequence of partial subgradients is determined by a general…
We present a new, tractable method for solving and analyzing risk-aware control problems over finite and infinite, discounted time-horizons where the dynamics of the controlled process are described as a martingale problem. Supposing…
This paper considers the optimal control of time varying continuous time Markov chains whose transition rates are themselves Markov processes. In one set of problems the solution of an ordinary differential equation is shown to determine…
In computational system biology, the mesoscopic model of reaction-diffusion kinetics is described by a continuous time, discrete space Markov process. To simulate diffusion stochastically, the jump coefficients are obtained by a…