相关论文: Discrete-time, discrete-state multistate Markov mo…

In this paper, we consider statistical estimation of time-inhomogeneous aggregate Markov models. Unaggregated models, which corresponds to Markov chains, are commonly used in multi-state life insurance to model the biometric states of an…

This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…

Motivated by wide-ranging applications such as video delivery over networks using Multiple Description Codes, congestion control, and inventory management, we study the state-tracking of a Markovian random process with a known transition…

We study discrete time Markov processes with periodic or open boundary conditions and with inhomogeneous rates in the bulk. The Markov matrices are given by the inhomogeneous transfer matrices introduced previously to prove the…

A classical approach to formal policy synthesis in stochastic dynamical systems is to construct a finite-state abstraction, often represented as a Markov decision process (MDP). The correctness of these approaches hinges on a behavioural…

We study nonlinear time-inhomogeneous Markov processes in the sense of McKean's seminal work [32]. These are given as families of laws $\mathbb{P}_{s,\zeta}$, $s\geq 0$, on path space, where $\zeta$ runs through a set of admissible initial…

In order to give quantitative estimates for approximating the ergodic limit, we investigate probabilistic limit behaviors of time-averaging estimators of numerical discretizations for a class of time-homogeneous Markov processes, by…

This paper is about the state estimation of timed probabilistic discrete event systems. The main contribution is to propose general procedures for developing state estimation approaches based on artificial neural networks. It is assumed…

We introduce multiple hidden Markov models (MHMMs) where an observed multivariate categorical time series depends on an unobservable multivariate Mar- kov chain. MHMMs provide an elegant framework for specifying various independence…

The distributionally robust Markov Decision Process (MDP) approach asks for a distributionally robust policy that achieves the maximal expected total reward under the most adversarial distribution of uncertain parameters. In this paper, we…

Time-homogeneous Markov chains are often used as disease progression models in studies of cost-effectiveness and optimal decision-making. Maximum likelihood estimation of these models can be challenging when data are collected at a time…

Inhomogeneous phase-type (IPH) distributions extend classical phase-type models by allowing transition intensities to vary over time, offering greater flexibility for modeling heavy-tailed or time-dependent absorption phenomena. We focus on…

We introduce the concept of a Markov influence system (MIS) and analyze its dynamics. An MIS models a random walk in a graph whose edges and transition probabilities change endogenously as a function of the current distribution. This…

Understanding and predicting how complex systems respond to external perturbations is a central challenge in nonequilibrium statistical physics. Here we consider continuous-time Markov networks, which we subject to perturbations along a…

This paper proposes nonparametric two-sample tests for the direct comparison of the probabilities of a particular transition between states of a continuous time nonhomogeneous Markov process with a finite state space. The proposed tests are…

Markov cohort state-transition models have been the standard approach for simulating the prognosis of patients or, more generally, the life trajectories of individuals over a time period. Current approaches for estimating the variance of a…

Nonparametric identification and maximum likelihood estimation for finite-state hidden Markov models are investigated. We obtain identification of the parameters as well as the order of the Markov chain if the transition probability…

Several Artificial Intelligence schemes for reasoning under uncertainty explore either explicitly or implicitly asymmetries among probabilities of various states of their uncertain domain models. Even though the correct working of these…

Sequential data modeling and analysis have become indispensable tools for analyzing sequential data, such as time-series data, because larger amounts of sensed event data have become available. These methods capture the sequential structure…

Focusing on stochastic systems arising in mean-field models, the systems under consideration belong to the class of switching diffusions, in which continuous dynamics and discrete events coexist and interact. The discrete events are modeled…