Related papers: The embedding problem for Markov matrices

We aim at enforcing hard constraints to impose a global structure on sequences generated from Markov models. In this report, we study the complexity of sampling Markov sequences under two classes of constraints: Binary Equalities and…

Tensor structured Markov chains are part of stochastic models of many practical applications, e.g., in the description of complex production or telephone networks. The most interesting question in Markov chain models is the determination of…

The problem of constrained Markov decision process is considered. An agent aims to maximize the expected accumulated discounted reward subject to multiple constraints on its costs (the number of constraints is relatively small). A new dual…

Many biological and medical questions can be modeled using time-to-event data in finite-state Markov chains, with the phase-type distribution describing intervals between events. We solve the inverse problem: given a phase-type…

We consider the fluctuations of a time-integrated particle current around an atypical value in a generic stochastic Markov process involving classical particles with two-site interaction and hardcore repulsion on a finite one-dimensional…

Recently, Sidford, Wang, Wu and Ye (2018) developed an algorithm combining variance reduction techniques with value iteration to solve discounted Markov decision processes. This algorithm has a sublinear complexity when the discount factor…

We expose in full detail a constructive procedure to invert the so--called "finite Markov moment problem". The proofs rely on the general theory of Toeplitz matrices together with the classical Newton's relations.

In this paper, we seek to understand the behavior of dynamical systems that are perturbed by a parameter that changes discretely in time. If we impose certain conditions, we can study certain embedded systems within a hybrid system as…

We study a class of Markov processes that combine local dynamics, arising from a fixed Markov process, with regenerations arising at a state-dependent rate. We give conditions under which such processes possess a given target distribution…

Markov-chain Monte Carlo algorithms rely on trial moves that are either rejected or accepted based on certain criteria. Here, we provide an efficient algorithm to generate random rotation matrices in four dimensions (4D) covering an…

Verification of infinite-state Markov chains is still a challenge despite several fruitful numerical or statistical approaches. For decisive Markov chains, there is a simple numerical algorithm that frames the reachability probability as…

Filtering---estimating the state of a partially observable Markov process from a sequence of observations---is one of the most widely studied problems in control theory, AI, and computational statistics. Exact computation of the posterior…

The spectra of signed matrices have played a fundamental role in social sciences, graph theory, and control theory. In this work, we investigate the computational problems of identifying symmetric signings of matrices with natural spectral…

The linearly constrained matrix rank minimization problem is widely applicable in many fields such as control, signal processing and system identification. The tightest convex relaxation of this problem is the linearly constrained nuclear…

Predictability of behavior has emerged an an important characteristic in many fields including biology, medicine, and marketing. Behavior can be recorded as a sequence of actions performed by an individual over a given time period. This…

We discuss the limiting spectral density of real symmetric random matrices. Other than in standard random matrix theory the upper diagonal entries are not assumed to be independent, but we will fill them with the entries of a stochastic…

Embedders play a central role in machine learning, projecting any object into numerical representations that can, in turn, be leveraged to perform various downstream tasks. The evaluation of embedding models typically depends on…

We show that the complexity of the Markov bases of multidimensional tables stabilizes eventually if a single table dimension is allowed to vary. In particular, if this table dimension is beyond a computable bound, the Markov bases consist…

We derive sufficient conditions for subgeometric f-ergodicity of strongly Markovian processes. We first propose a criterion based on modulated moment of some delayed return-time to a petite set. We then formulate a criterion for polynomial…

Markov chain analysis is a key technique in formal verification. A practical obstacle is that all probabilities in Markov models need to be known. However, system quantities such as failure rates or packet loss ratios, etc. are often not --…