Related papers: Dynamic Markov Bases
Recently we pointed out the so-called Local Time Scheme as a novel approach to quantum foundations that solves the preferred pointer-basis problem. In this paper we introduce and analyze in depth a rather non-standard dynamical map that is…
The parameters of a discrete stationary Markov model are transition probabilities between states. Traditionally, data consist in sequences of observed states for a given number of individuals over the whole observation period. In such a…
In this paper, we present an overview of different types of random walk strategies with local and non-local transitions on undirected connected networks. We present a general approach to analyzing these strategies by defining the dynamics…
These lecture notes introduce the statistical analysis of continuous-time generative models built from Markov dynamics. We begin with the stochastic-calculus foundations of score-based diffusion models, including time reversal, score…
We propose a new method for optimistic planning in infinite-horizon discounted Markov decision processes based on the idea of adding regularization to the updates of an otherwise standard approximate value iteration procedure. This…
Tensor structured Markov chains are part of stochastic models of many practical applications, e.g., in the description of complex production or telephone networks. The most interesting question in Markov chain models is the determination of…
Graphical Markov models combine conditional independence constraints with graphical representations of stepwise data generating processes.The models started to be formulated about 40 years ago and vigorous development is ongoing.…
We present a generative model of human mobility in which trajectories arise as realizations of a prescribed, time-dependent Markov dynamics defined on a spatial interaction network. The model constructs a hierarchical routing structure with…
This paper develops a mathematical framework to study signal networks, in which nodes can be active or inactive, and their activation or deactivation is driven by external signals and the states of the nodes to which they are connected via…
Many optimization problems can be naturally represented as (hyper) graphs, where vertices correspond to variables and edges to tasks, whose cost depends on the values of the adjacent variables. Capitalizing on the structure of the graph,…
Understanding which parts of a dynamical system cause each other is extremely relevant in fundamental and applied sciences. However, inferring causal links from observational data, namely without direct manipulations of the system, is still…
Many tasks in human environments require performing a sequence of navigation and manipulation steps involving objects. In unstructured human environments, the location and configuration of the objects involved often change in unpredictable…
Many-body Hamiltonians obtained from first principles generally include all possible non-local interactions. But in dynamical mean field theory the non-local interactions are ignored, and only the effects of the local interactions are taken…
Random walks are a fundamental model in applied mathematics and are a common example of a Markov chain. The limiting stationary distribution of the Markov chain represents the fraction of the time spent in each state during the stochastic…
We propose a dynamic model of dependence structure between financial institutions within a financial system and we construct measures for dependence and financial instability. Employing Markov structures of joint credit migrations, our…
Variable-order Markov models generate sequences over a finite alphabet by conditioning each symbol on the longest available suffix of the generated history. Regular constraints, by contrast, describe finite-horizon control requirements by…
In the study of dynamical processes on networks, there has been intense focus on network structure -- i.e., the arrangement of edges and their associated weights -- but the effects of the temporal patterns of edges remains poorly…
It has become increasingly easy nowadays to collect approximate posterior samples via fast algorithms such as variational Bayes, but concerns exist about the estimation accuracy. It is tempting to build solutions that exploit approximate…
This paper is concerned with a data-driven technique for constructing finite Markov decision processes (MDPs) as finite abstractions of discrete-time stochastic control systems with unknown dynamics while providing formal closeness…
This paper proposes a new formulation for the dynamic resource allocation problem, which converts the traditional MDP model with known parameters and no capacity constraints to a new model with uncertain parameters and a resource capacity…