Related papers: A Production Model with History Based Random Machi…
Time series imputation is one of the most challenge problems and has broad applications in various fields like health care and the Internet of Things. Existing methods mainly aim to model the temporally latent dependencies and the…
In this paper, we propose a novel class of Piecewise Deterministic Markov Processes (PDMPs) that are designed to sample from probability distributions $\pi$ supported on a convex set $\mathcal{M}$. This class of PDMPs adapts the concept of…
This paper considers a Markovian model of a limit order book where time-dependent rates are allowed. With the objective of understanding the mechanisms through which a microscopic model of an orderbook can converge to more general diffusion…
The Markov assumption (MA) is fundamental to the empirical validity of reinforcement learning. In this paper, we propose a novel Forward-Backward Learning procedure to test MA in sequential decision making. The proposed test does not assume…
We introduce a Markov-functional approach to construct local volatility models that are calibrated to a discrete set of marginal distributions. The method is inspired by and extends the volatility interpolation of Bass (1983) and Conze and…
We consider killed Markov decision processes for countable models on a finite time-interval. Existence of a uniform $\varepsilon$-optimal policy is proven. We show the correctness of the fundamental equation. The optimal control problem is…
In this paper is described the general aspect of a numerical method for piecewise determin-istic Markov processes with boundary. Under very natural hypotheses, a crucial result about uniqueness of solution of a generalized Kolmogorov…
Continuous Time Markov Chains (CTMC) have been used extensively to model reliability of storage systems. While the exponentially distributed sojourn time of Markov models is widely known to be unrealistic (and it is necessary to consider…
We study the time reversal of a general PDMP. The time reversed process is defined as $X_{(T-t)-}$, where $T$ is some given time and $X_t$ is a stationary PDMP. We obtain the parameters of the reversed process, like the jump intensity and…
Many real-world objects can be modeled as a stream of events on the nodes of a graph. In this paper, we propose a class of graphical event models named temporal point process graphical models for representing the temporal dependencies among…
Predictive maintenance (PdM) is a concept, which is implemented to effectively manage maintenance plans of the assets by predicting their failures with data driven techniques. In these scenarios, data is collected over a certain period of…
Software reliability models are an important tool in quality management and release planning. There is a large number of different models that often exhibit strengths in different areas. This paper proposes a model that is based on a…
Incorporating pattern-learning for prediction (PLP) in many discrete-time or discrete-event systems allows for computation-efficient controller design by memorizing patterns to schedule control policies based on their future occurrences. In…
We consider a general honest homogeneous continuous-time Markov process with restarts. The process is forced to restart from a given distribution at time moments generated by an independent Poisson process. The motivation to study such…
Existing models for the analysis of concurrent processes tend to focus on fail-stop failures, where processes are either working or permanently stopped, and their state (working/stopped) is known. In fact, systems are often affected by grey…
We here consider the subset simulation method which approaches a failure event using a decreasing sequence of nested intermediate failure events. The method resembles importance sampling, which actively explores a probability space by…
In this article, we consider a continuous review (s, S) inventory system with failures of demand fulfillment (service) modeled as a Markov-modulated retrial queueing system. The inventory system features a single product that experiences…
Statistical Mechanics deals with ensembles of microstates that are compatible with fixed constraints and that on average define a thermodynamic macrostate. The evolution of a small system is normally subjected to changing constraints and…
Optimal policies for partially observed Markov decision processes (POMDPs) are history-dependent: Decisions are made based on the entire history of observation. Memoryless policies, which take decisions based on the last observation only,…
We introduce a class of models for multidimensional control problems which we call skip-free Markov decision processes on trees. We describe and analyse an algorithm applicable to Markov decision processes of this type that are skip-free in…