Related papers: Generic identification of binary-valued hidden Mar…

In the paper we consider a stochastic model which called Markov Q-processes that forms a continuous-time Markov population system. Markov Q-processes are defined as stochastic Markov branching processes with trajectories continuing in the…

This note summarizes the optimization formulations used in the study of Markov decision processes. We consider both the discounted and undiscounted processes under the standard and the entropy-regularized settings. For each setting, we…

We provide a sufficient criterion for the unique parameter identification of combinatorially symmetric Hidden Markov Models based on the structure of their transition matrix. If the observed states of the chain form a zero forcing set of…

Markov processes are popular mathematical models, studied by theoreticians for their intriguing properties, and applied by practitioners for their flexible structure. With this book we teach how to model and analyze Markov processes. We…

We introduce a machine-learning approach for identifying hidden structural features of open quantum dynamics under restricted experimental access. Unlike most existing data-driven methods which focus on detection or prediction of dynamical…

A penalized maximum likelihood estimation approach is proposed for discrete-time hidden Markov models where covariates affect the observed responses and serial dependence is considered. The proposed penalized maximum likelihood method…

This paper formed part of a preliminary research report for a risk consultancy and academic research. Stochastic Programming models provide a powerful paradigm for decision making under uncertainty. In these models the uncertainties are…

We use the abstract method of (local) martingale problems in order to give criteria for convergence of stochastic processes. Extending previous notions, the formulation we use is neither restricted to Markov processes (or semimartingales),…

Many binary classification problems minimize misclassification above (or below) a threshold. We show that instances of ranking problems, accuracy at the top or hypothesis testing may be written in this form. We propose a general framework…

This paper introduces a mathematical framework of a stochastic process model as a generalization of diffusion stochastic processes to model latent variables in categorical responses given unobserved random effects and maximum likelihood…

Existing methods for differentiable structure learning in discrete data typically assume that the data are generated from specific structural equation models. However, these assumptions may not align with the true data-generating process,…

We present an approach for testing for the existence of continuous generators of discrete stochastic transition matrices. Typically, the known approaches to ascertain the existence of continuous Markov processes are based in the assumption…

The standard Bayesian Information Criterion (BIC) is derived under regularity conditions which are not always satisfied by the graphical models with hidden variables. In this paper we derive the BIC score for Bayesian networks in the case…

Motivated by multiple applications in social networks, nervous systems, and financial risk analysis, we consider the problem of learning the underlying (directed) influence graph or causal graph of a high-dimensional multivariate…

The solution to a multivariate linear Stochastic Differential Equation (SDE) with constant initial state is well known to be a Gaussian Markov process, but its covariance kernel involves the solution to an integral equation in the general…

This paper considers hidden Markov models where the observations are given as the sum of a latent state which lies in a general state space and some independent noise with unknown distribution. It is shown that these fully nonparametric…

We consider the problem of flexible modeling of higher order hidden Markov models when the number of latent states and the nature of the serial dependence, including the true order, are unknown. We propose Bayesian nonparametric methodology…

We introduce partial Markov categories. In the same way that Markov categories encode stochastic processes, partial Markov categories encode stochastic processes with constraints, observations and updates. In particular, we prove a…

Hidden Markov Chains (HMCs) are commonly used mathematical models of probabilistic systems. They are employed in various fields such as speech recognition, signal processing, and biological sequence analysis. We consider the problem of…

We propose a vector linear programming formulation for a non-stationary, finite-horizon Markov decision process with vector-valued rewards. Pareto efficient policies are shown to correspond to efficient solutions of the linear program, and…