Related papers: Reward Algorithms for Semi-Markov Processes

In this paper, we consider a class of inhomogeneous semi-Markov processes directly based on intensity processes for marked point processes. We show that this class satisfies the semi-Markov properties defined elsewhere in the literature. We…

Recently there have been exciting developments in Monte Carlo methods, with the development of new MCMC and sequential Monte Carlo (SMC) algorithms which are based on continuous-time, rather than discrete-time, Markov processes. This has…

We introduce the use of reinforcement learning for indirect mechanisms, working with the existing class of sequential price mechanisms, which generalizes both serial dictatorship and posted price mechanisms and essentially characterizes all…

Given a batch of human computation tasks, a commonly ignored aspect is how the price (i.e., the reward paid to human workers) of these tasks must be set or varied in order to meet latency or cost constraints. Often, the price is set…

We introduce a new approach for deterministic sensitivity analysis of Markov reward processes, commonly used in cost-effectiveness analyses, via reformulation into a polynomial system. Our approach leverages cylindrical algebraic…

We propose a multiplicative semiparametric model for the intensity function of replicated point processes. Two examples of applications are given: a temporal one, about the dynamics of Internet auctions, and a spatial one, about the spatial…

In this paper we propose a semi-Markov modulated model of interest rates. We assume that the switching process is a semi-Markov process with finite state space E and the modulated process is a diffusive process. We derive recursive…

Semi-Markov processes represent a well known and widely used class of random processes in classical probability theory. Here, we develop an extension of this type of non-Markovian dynamics to the quantum regime. This extension is…

We consider finite horizon Markov decision processes under performance measures that involve both the mean and the variance of the cumulative reward. We show that either randomized or history-based policies can improve performance. We prove…

A new Monte Carlo algorithm for phase-space sampling, named (MC)**3, is presented. It is based on Markov Chain Monte Carlo techniques but at the same time incorporates prior knowledge about the target distribution in the form of suitable…

We present a numerical method to compute expectations of functionals of a piecewise-deterministic Markov process. We discuss time dependent functionals as well as deterministic time horizon problems. Our approach is based on the…

In the last years, many authors studied a class of continuous time semi-Markov processes obtained by time-changing Markov processes by hitting times of independent subordinators. Such processes are governed by integro-differential…

We present a novel method for computing reachability probabilities of parametric discrete-time Markov chains whose transition probabilities are fractions of polynomials over a set of parameters. Our algorithm is based on two key…

We construct a large class of non-Markovian master equations that describe the dynamics of open quantum systems featuring strong memory effects, which relies on a quantum generalization of the concept of classical semi-Markov processes.…

Lecture notes (in French) of a master 2 level course in applied mathematics. Contents: Part I. Markov chains on a countable space. 1. Examples 2. Summary of basic properties. 3. Spectral theory and speed of convergence. 4. Lyapunov…

A novel class of non-reversible Markov chain Monte Carlo schemes relying on continuous-time piecewise-deterministic Markov Processes has recently emerged. In these algorithms, the state of the Markov process evolves according to a…

Suppose there are $n$ Markov chains and we need to pay a per-step \emph{price} to advance them. The "destination" states of the Markov chains contain rewards; however, we can only get rewards for a subset of them that satisfy a…

We present a computational algorithm for computing short range forces between particles. The algorithm has two distinguishing features. First, it is optimized for multi-processor computers, and will use as many processors as are available.…

We consider the problem of computing optimal policies in average-reward Markov decision processes. This classical problem can be formulated as a linear program directly amenable to saddle-point optimization methods, albeit with a number of…

The estimation of absorption time distributions of Markov jump processes is an important task in various branches of statistics and applied probability. While the time-homogeneous case is classic, the time-inhomogeneous case has recently…