Related papers: Quantile Fourier regressions for decision making u…
A method is proposed to reconstruct a cyclic time-inhomogeneous Markov pro- cess from measured data. First, a time-inhomogeneous Markov model is fit to the data, taken here from measurements on a wind turbine. From the time-dependent…
Wind energy is becoming a top contributor to the renewable energy mix, which raises potential reliability issues for the grid due to the fluctuating nature of its source. To achieve adequate reserve commitment and to promote market…
Methods to find counterfactual explanations have predominantly focused on one step decision making processes. In this work, we initiate the development of methods to find counterfactual explanations for decision making processes in which…
Reinforcement Learning Algorithms are predominantly developed for stationary environments, and the limited literature that considers nonstationary environments often involves specific assumptions about changes that can occur in transition…
The use of factor stochastic volatility models requires choosing the number of latent factors used to describe the dynamics of the financial returns process; however, empirical evidence suggests that the number and makeup of pertinent…
This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…
We introduce a novel framework for causal explanations of stochastic, sequential decision-making systems built on the well-studied structural causal model paradigm for causal reasoning. This single framework can identify multiple,…
Complex numerical weather prediction models incorporate a variety of physical processes, each described by multiple alternative physical schemes with specific parameters. The selection of the physical schemes and the choice of the…
Complex dynamical systems are prevalent in many scientific disciplines. In the analysis of such systems two aspects are of particular interest: 1) the temporal patterns along which they evolve and 2) the underlying causal mechanisms.…
This article introduces a novel dynamic framework to Bayesian model averaging for time-varying parameter quantile regressions. By employing sequential Markov chain Monte Carlo, we combine empirical estimates derived from dynamically chosen…
We introduce a general framework for Markov decision problems under model uncertainty in a discrete-time infinite horizon setting. By providing a dynamic programming principle we obtain a local-to-global paradigm, namely solving a local,…
In the autoregressive process of first order AR(1), a homogeneous correlated time series $u_t$ is recursively constructed as $u_t = q\; u_{t-1} + \sigma \;\epsilon_t$, using random Gaussian deviates $\epsilon_t$ and fixed values for the…
The reduction of a continuous Markov process with multiple metastable states to a discrete rate process is investigated in the presence of slow time dependent parameters such as periodic external forces or slowly fluctuating barrier…
We outline a mathematical model for pricing hydropower generation. The model involves a Markov decision process that reflects the seasonal variation in historical time series of water inflows. The procedure is computationally efficient and…
We consider a sequential decision making process, such as renewable energy trading or electrical production scheduling, whose outcome depends on the future realization of a random factor, such as a meteorological variable. We assume that…
We present a short introduction into the framework of piecewise deterministic Markov processes. We illustrate the abstract mathematical setting with a series of examples related to dispersal of biological systems, cell cycle models, gene…
Markov decision processes are typically used for sequential decision making under uncertainty. For many aspects however, ranging from constrained or safe specifications to various kinds of temporal (non-Markovian) dependencies in task and…
Many psychological experiments have subjects repeat a task to gain the statistical precision required to test quantitative theories of psychological performance. In such experiments, time-on-task can have sizable effects on performance,…
A general theory is developed to study individual based models which are discrete in time. We begin by constructing a Markov chain model that converges to a one-dimensional map in the infinite population limit. Stochastic fluctuations are…
In piecewise-deterministic Markov processes (PDMPs) the state of a finite-dimensional system evolves continuously, but the evolutive equation may change randomly as a result of discrete switches. A running cost is integrated along the…