Related papers: Modeling Maximum drawdown Records with Piecewise D…

We study a Markov decision problem in which the state space is the set of finite marked point configurations in the plane, the actions represent thinnings, the reward is proportional to the mark sum which is discounted over time, and the…

In order to give quantitative estimates for approximating the ergodic limit, we investigate probabilistic limit behaviors of time-averaging estimators of numerical discretizations for a class of time-homogeneous Markov processes, by…

Machine learning techniques not only offer efficient tools for modelling dynamical systems from data, but can also be employed as frontline investigative instruments for the underlying physics. Nontrivial information about the original…

In this paper, we propose an approximate dynamic programming (ADP) algorithm to solve a Markov decision process (MDP) formulation for the admission control of elective patients. To manage the elective patients from multiple specialties…

We propose a computational method for large deviation statistics of time-averaged quantities in general Markov processes. In our proposed method, we repeat a response measurement against external forces, where the forces are determined by…

Control of drawdown, that is, the control of the drops in wealth over time from peaks to subsequent lows, is of great concern from a risk management perspective. With this motivation in mind, the focal point of this paper is to address the…

We consider Markov decision processes (MDPs) with multiple limit-average (or mean-payoff) objectives. There exist two different views: (i) the expectation semantics, where the goal is to optimize the expected mean-payoff objective, and (ii)…

In the optimization of dynamic systems, the variables typically have constraints. Such problems can be modeled as a Constrained Markov Decision Process (CMDP). This paper considers the peak Constrained Markov Decision Process (PCMDP), where…

We consider synthesis of control policies that maximize the probability of satisfying given temporal logic specifications in unknown, stochastic environments. We model the interaction between the system and its environment as a Markov…

Optimal decision-making under partial observability requires agents to balance reducing uncertainty (exploration) against pursuing immediate objectives (exploitation). In this paper, we introduce a novel policy optimization framework for…

Standard Markov decision process (MDP) and reinforcement learning algorithms optimize the policy with respect to the expected gain. We propose an algorithm which enables to optimize an alternative objective: the probability that the gain is…

We present an alternative view for the study of optimal control of partially observed Markov Decision Processes (POMDPs). We first revisit the traditional (and by now standard) separated-design method of reducing the problem to fully…

A piecewise-deterministic Markov process, specified by random jumps and switching semi-flows, as well as the associated Markov chain given by its post-jump locations, are investigated in this paper. The existence of an exponentially…

We present new algorithms for computing and approximating bisimulation metrics in Markov Decision Processes (MDPs). Bisimulation metrics are an elegant formalism that capture behavioral equivalence between states and provide strong…

Markov decision processes (MDPs) with multi-dimensional weights are useful to analyze systems with multiple objectives that may be conflicting and require the analysis of trade-offs. We study the complexity of percentile queries in such…

We derive sufficient and necessary optimality conditions in terms of a stochastic maximum principle (SMP) for controls associated with cost functionals of mean-field type, under dynamics driven by a class of Markov chains of mean-field type…

The continuous time Markov process considered in this paper belongs to a class of population models with linear growth and catastrophes. There, the catastrophes happen at the arrival times of a Poisson process, and at each catastrophe time,…

We show fundamental properties of the Markov semigroup of recently proposed MCMC algorithms based on Piecewise-deterministic Markov processes (PDMPs) such as the Bouncy Particle Sampler, the Zig-Zag process or the Randomized Hamiltonian…

We extend Monte Carlo samplers based on piecewise deterministic Markov processes (PDMP samplers) by formally defining different boundary conditions such as sticky floors, soft and hard walls and teleportation portals. This allows PDMP…

We consider stochastic processes arising from dynamical systems simply by evaluating an observable function along the orbits of the system and study marked point processes associated to extremal observations of such time series…