Related papers: A modern approach to transition analysis and proce…
We present a novel methodology utilizing Recurrent Neural Networks (RNNs) to classify Markovian and non-Markovian quantum processes, leveraging time series data derived from Choi states. The model exhibits exceptional accuracy, surpassing…
Markov branching systems form a fundamental class of stochastic models that are extensively applied in biology, physics, finance, and other domains. These systems are distinguished by their continuous-time evolution and inherent branching…
Hidden Markov models (HMMs) are widely applied in studies where a discrete-valued process of interest is observed indirectly. They have for example been used to model behaviour from human and animal tracking data, disease status from…
Phase-type distribution has been an important probabilistic tool in the analysis of complex stochastic system evolution. It was introduced by Neuts \cite{Neuts1975} in 1975. The model describes the lifetime distribution of a finite-state…
Traditional hidden Markov models have been a useful tool to understand and model stochastic dynamic data; in the case of non-Gaussian data, models such as mixture of Gaussian hidden Markov models can be used. However, these suffer from the…
Modeling unknown systems from data is a precursor of system optimization and sequential decision making. In this paper, we focus on learning a Markov model from a single trajectory of states. Suppose that the transition model has a small…
This paper presents a technique for reduced-order Markov modeling for compact representation of time-series data. In this work, symbolic dynamics-based tools have been used to infer an approximate generative Markov model. The time-series…
Data of sequential nature arise in many application domains in forms of, e.g. textual data, DNA sequences, and software execution traces. Different research disciplines have developed methods to learn sequence models from such datasets: (i)…
Random walks are a fundamental model in applied mathematics and are a common example of a Markov chain. The limiting stationary distribution of the Markov chain represents the fraction of the time spent in each state during the stochastic…
In this paper, we present a numerical framework for constructing bounds on stationary performance measures of random walks in the positive orthant using the Markov reward approach. These bounds are established in terms of stationary…
A novel approach to malware classification is introduced based on analysis of instruction traces that are collected dynamically from the program in question. The method has been implemented online in a sandbox environment (i.e., a security…
Non-Markovian models have great expressive power, at the cost of complex analysis of the stochastic process. The method of Stochastic State Classes (SSCs) derives closed-form analytical expressions for the joint Probability Density…
The goal of this paper is to analyze distributional Markov Decision Processes as a class of control problems in which the objective is to learn policies that steer the distribution of a cumulative reward toward a prescribed target law,…
Two Cox-based multistate modeling approaches are compared for analyzing a complex multicohort event history process. The first approach incorporates cohort information as a fixed covariate, thereby providing a direct estimation of the…
We propose a new Markov chain Monte Carlo method in which trial configurations are generated by evolving a state, sampled from a prior distribution, using a Markov transition matrix. We present two prototypical algorithms and derive their…
Normalizing flows, diffusion normalizing flows and variational autoencoders are powerful generative models. This chapter provides a unified framework to handle these approaches via Markov chains. We consider stochastic normalizing flows as…
This project is going to work with one example of stochastic matrix to understand how Markov chains evolve and how to use them to make faster and better decisions only looking to the present state of the system.
In risk management it is desirable to grasp the essential statistical features of a time series representing a risk factor. This tutorial aims to introduce a number of different stochastic processes that can help in grasping the essential…
Markov state models (MSMs) have been successful in computing metastable states, slow relaxation timescales and associated structural changes, and stationary or kinetic experimental observables of complex molecules from large amounts of…
In this work, we consider the problem of mode clustering in Markov jump models. This model class consists of multiple dynamical modes with a switching sequence that determines how the system switches between them over time. Under different…