Related papers: Forecasting Densities of Fatalities from State-bas…

Semi-Markov models are widely used for survival analysis and reliability analysis. In general, there are two competing parameterizations and each entails its own interpretation and inference properties. On the one hand, a semi-Markov…

This paper introduces a new parsimonious structure for mixture of autoregressive models. the weighting coefficients are determined through latent random variables, following a hidden Markov model. We propose a dynamic programming algorithm…

Birth and death Markov processes can model stochastic physical systems from percolation to disease spread and, in particular, wildfires. We introduce and analyze a birth-death-suppression Markov process as a model of controlled culling of…

Maintenance optimization of naval ship equipment is crucial in terms of national defense. However, the mixed effect of the maintenance and the pure deterioration processes in the observed data hinders an exact comparison between candidate…

Wildfires pose an increasing threat to the safety and reliability of power systems, particularly in distribution networks located in fire-prone regions. To mitigate ignition risk from electrical infrastructure, utilities often employ safety…

Measuring states in reinforcement learning (RL) can be costly in real-world settings and may negatively influence future outcomes. We introduce the Actively Observable Markov Decision Process (AOMDP), where an agent not only selects control…

Reinforcement learning algorithms typically rely on the assumption that the environment dynamics and value function can be expressed in terms of a Markovian state representation. However, when state information is only partially observable,…

The paper studies information-theoretic opacity, an information-flow privacy property, in a setting involving two agents: A planning agent who controls a stochastic system and an observer who partially observes the system states. The goal…

Stochastic reaction network models arise in intracellular chemical reactions, epidemiological models and other population process models, and are a class of continuous time Markov chains which have the nonnegative integer lattice as state…

This work studies networked agents cooperating to track a dynamical state of nature under partial information. The proposed algorithm is a distributed Bayesian filtering algorithm for finite-state hidden Markov models (HMMs). It can be used…

This work presents a scalable control framework based on nonlinear Model Predictive Control for high-dimensional dynamical systems. The proposed approach addresses the key challenges of model scalability and partial observability by…

We study offline reinforcement learning problems with a long-run average reward objective. The state-action pairs generated by any fixed behavioral policy thus follow a Markov chain, and the {\em empirical} state-action-next-state…

We consider a Markov process in continuous time with a finite number of discrete states. The time-dependent probabilities of being in any state of the Markov chain are governed by a set of ordinary differential equations, whose dimension…

Multi-state survival analysis considers several potential events of interest along a disease pathway. Such analyses are crucial to model complex patient trajectories and are increasingly being used in epidemiological and health economic…

In this paper, we investigate the arising communication patterns on social media, and in particular the series of events happening for a single user. While the distribution of inter-event times is often assimilated to power-law density…

We study off-policy evaluation and learning from sequential data in a structured class of Markov decision processes that arise from repeated interactions with an exogenous sequence of arrivals with contexts, which generate unknown…

This paper is concerned with a characterization of the observability for a continuous-time hidden Markov model where the state evolves as a general continuous-time Markov process and the observation process is modeled as nonlinear function…

In this research, two-state Markov switching models are proposed to study accident frequencies and severities. These models assume that there are two unobserved states of roadway safety, and that roadway entities (e.g., roadway segments)…

Weather forecast information will very likely find increasing application in the control of future energy systems. In this paper, we introduce an augmented state space model formulation with linear dynamics, within which one can incorporate…

This paper proposes a joint model for longitudinal binary and count outcomes. We apply the model to a unique longitudinal study of teen driving where risky driving behavior and the occurrence of crashes or near crashes are measured…