Related papers: Stable and practical semi-Markov modelling of inte…
We provide a new non-asymptotic analysis of distributed TD(0) with linear function approximation. Our approach relies on "one-shot averaging," where $N$ agents run local copies of TD(0) and average the outcomes only once at the very end. We…
A widely-used model for determining the long-term health impacts of public health interventions, often called a "multistate lifetable", requires estimates of incidence, case fatality, and sometimes also remission rates, for multiple…
State-space models are a popular statistical framework for analysing sequential data. Within this framework, particle filters are often used to perform inference on non-linear state-space models. We introduce a new method, StateMixNN, that…
Transferring information from observations of a dynamical system to estimate the fixed parameters and unobserved states of a system model can be formulated as the evaluation of a discrete time path integral in model state space. The…
This paper investigates MDPs with intermittent state information. We consider a scenario where the controller perceives the state information of the process via an unreliable communication channel. The transmissions of state information…
State-space models (SSM) are central to describe time-varying complex systems in countless signal processing applications such as remote sensing, networks, biomedicine, and finance to name a few. Inference and prediction in SSMs are…
Complex networks have played an important role in describing real complex systems since the end of the last century. Recently, research on real-world data sets reports intermittent interaction among social individuals. In this paper, we pay…
We introduce a multivariate hidden Markov model to jointly cluster time-series observations with different support, i.e. circular and linear. Relying on the general projected normal distribution, our approach allows for bimodal and/or…
This chapter presents an introduction to Markovian modeling for the analysis of sequence data. Contrary to the deterministic approach seen in the previous sequence analysis chapters, Markovian models are probabilistic models, focusing on…
In medical research, understanding changes in outcome measurements is crucial for inferring shifts in health conditions. However, traditional methods often struggle with large, irregularly longitudinal data and fail to account for the…
A semi-Markov process is one that changes states in accordance with a Markov chain but takes a random amount of time between changes. We consider the generalisation to semi-Markov processes of the classical Lamperti law for the occupation…
In this paper, a hybrid measurement and model-based method is proposed which can estimate the dynamic state Jacobian matrix in near real-time. The proposed method is computationally efficient and robust to the variation of network topology.…
Clustered multistate process data are commonly encountered in multicenter observational studies and clinical trials. A clinically important estimand with such data is the marginal probability of being in a particular transient state as a…
This paper introduces a linear state-space model with time-varying dynamics. The time dependency is obtained by forming the state dynamics matrix as a time-varying linear combination of a set of matrices. The time dependency of the weights…
We present a simulation methodology for Bayesian estimation of rate parameters in Markov jump processes arising for example in stochastic kinetic models. To handle the problem of missing components and measurement errors in observed data,…
In this paper, we outline a model of graph (or network) dynamics based on two ingredients. The first ingredient is a Markov chain on the space of possible graphs. The second ingredient is a semi-Markov counting process of renewal type. The…
We present a flexible Bayesian semiparametric mixed model for longitudinal data analysis in the presence of potentially high-dimensional categorical covariates. Building on a novel hidden Markov tensor decomposition technique, our proposed…
Monte Carlo (MC) simulations of transport in random porous networks indicate that for high variances of the log-normal permeability distribution, the transport of a passive tracer is non-Fickian. Here we model this non-Fickian dispersion in…
We propose a scalable semiparametric Bayesian model to capture dependencies among multiple neurons by detecting their co-firing (possibly with some lag time) patterns over time. After discretizing time so there is at most one spike at each…
Markov jump processes (or continuous-time Markov chains) are a simple and important class of continuous-time dynamical systems. In this paper, we tackle the problem of simulating from the posterior distribution over paths in these models,…