Related papers: Caliber Corrected Markov Modeling (C2M2): Correcti…

State-space models have been used in many applications, including econometrics, engineering, medical research, etc. The maximum likelihood estimation (MLE) of the static parameter of general state-space models is not straightforward because…

We consider an optimization problem related to semi-active damping of vibrating systems. The main problem is to determine the best damping matrix able to minimize influence of the input on the output of the system. We use a minimization…

We develop exact Markov chain Monte Carlo methods for discretely-sampled, directly and indirectly observed diffusions. The qualification "exact" refers to the fact that the invariant and limiting distribution of the Markov chains is the…

We introduce a new micro-macro Markov chain Monte Carlo method (mM-MCMC) to sample invariant distributions of molecular dynamics systems that exhibit a time-scale separation between the microscopic (fast) dynamics, and the macroscopic…

Learning a Markov Decision Process (MDP) from a fixed batch of trajectories is a non-trivial task whose outcome's quality depends on both the amount and the diversity of the sampled regions of the state-action space. Yet, many MDPs are…

Diffusive approximations of Markov jump processes often fail to accurately capture large fluctuations. This is confounding, as the rare events triggered by these large fluctuations, such as the failure of electronic memories, are often the…

Software-intensive systems, such as software product lines and robotics, utilise Markov decision processes (MDPs) to capture uncertainty and analyse sequential decision-making problems. Despite the usefulness of conventional policy…

We show how Markov mixed membership models (MMMM) can be used to predict the degradation of assets. We model the degradation path of individual assets, to predict overall failure rates. Instead of a separate distribution for each hidden…

The finite state semi-Markov process is a generalization over the Markov chain in which the sojourn time distribution is any general distribution. In this article we provide a sufficient stochastic maximum principle for the optimal control…

The Markov entropy decomposition (MED) is a recently-proposed, cluster-based simulation method for finite temperature quantum systems with arbitrary geometry. In this paper, we detail numerical algorithms for performing the required steps…

We introduce an extension of finite mixture models by incorporating skew-normal distributions within a Hidden Markov Model framework. By assuming a constant transition probability matrix and allowing emission distributions to vary according…

Switching state-space models (SSSM) are a very popular class of time series models that have found many applications in statistics, econometrics and advanced signal processing. Bayesian inference for these models typically relies on Markov…

This paper discusses algorithms for solving Markov decision processes (MDPs) that have monotone optimal policies. We propose a two-stage alternating convex optimization scheme that can accelerate the search for an optimal policy by…

In this paper, we provide strong $L_2$-rates of approximation of the integral-type functionals of Markov processes by integral sums. We improve the method developed in [2]. Under assumptions on the process formulated only in terms of its…

We consider Markov models of stochastic processes where the next-step conditional distribution is defined by a kernel density estimator (KDE), similar to Markov forecast densities and certain time-series bootstrap schemes. The KDE Markov…

Developing accurate and efficient coarse-grained representations of proteins is crucial for understanding their folding, function, and interactions over extended timescales. Our methodology involves simulating proteins with molecular…

Time-reversal symmetry of microscopic laws dictates that the equilibrium distribution of a stochastic process must obey the detailed balance. On the other hand, cyclic Markov processes that do not admit equilibrium distributions with…

Matching promises transparent causal inferences for observational data, making it an intuitive approach for many applications. In practice, however, standard matching methods often perform poorly compared to modern approaches such as…

The aim of this paper is to approximate a finite-state Markov process by another process with fewer states, called herein the approximating process. The approximation problem is formulated using two different methods. The first method,…

A significant challenge in molecular dynamics (MD) simulations is ensuring that sampled configurations converge to the equilibrium or nonequilibrium stationary distribution of interest. Lack of convergence constrains the estimation of free…