相关论文: Computing the minimal rebinding effect for non-rev…

The steady state of a Langevin equation with short ranged memory and coloured noise is analyzed. When the fluctuation-dissipation theorem of second kind is not satisfied, the dynamics is irreversible, i.e. detailed balance is violated. We…

We study the thermodynamics of a Brownian particle under the influence of a time multiplexed harmonic potential of finite width. The memory storage mechanism and the erasure protocol realized by time multiplexed potentials are utilized to…

Even simply-defined, finite-state generators produce stochastic processes that require tracking an uncountable infinity of probabilistic features for optimal prediction. For processes generated by hidden Markov chains the consequences are…

We develop an approach for estimating models described via conditional moment restrictions, with a prototypical application being non-parametric instrumental variable regression. We introduce a min-max criterion function, under which the…

We study the emergence of correlations between $N$ components of the position of a diffusive walker in $N$ dimensions that starts at the origin and resets to previously visited sites with certain probabilities. This is equivalent to $N$…

Certain biochemical reactions can only be triggered after binding of a sufficient number of particles to a specific target region such as an enzyme or a protein sensor. We investigate the distribution of the reaction time, i.e., the first…

Markov decision problems are most commonly solved via dynamic programming. Another approach is Bellman residual minimization, which directly minimizes the squared Bellman residual objective function. However, compared to dynamic…

It is common, when dealing with quantum processes involving a subsystem of a much larger composite closed system, to treat them as effectively memory-less (Markovian). While open systems theory tells us that non-Markovian processes should…

In the continuity of a recent paper ([6]), dealing with finite Markov chains, this paper proposes and analyzes a recursive algorithm for the approximation of the quasi-stationary distribution of a general Markov chain living on a compact…

In this paper, we present a new, optimization-based method to exhibit cyclic behavior in non-reversible stochastic processes. While our method is general, it is strongly motivated by discrete simulations of ordinary differential equations…

We present a closed-form, computable expression for the expected number of times any transition event occurs during the transient phase of a reducible Markov chain. Examples of events include time to absorption, number of visits to a state,…

We consider the problem of estimating the underlying graph associated with a Markov random field, with the added twist that the decoding algorithm can iteratively choose which subsets of nodes to sample based on the previous samples,…

In the analysis of Markov chains and processes, it is sometimes convenient to replace an unbounded state space with a "truncated" bounded state space. When such a replacement is made, one often wants to know whether the equilibrium behavior…

Reinforcement learning aims to learn optimal policies from interaction with environments whose dynamics are unknown. Many methods rely on the approximation of a value function to derive near-optimal policies. In partially observable…

This paper develops a decision algorithm for weak bisimulation on Markov Automata (MA). For that purpose, different notions of vanishing state (a concept known from the area of Generalised Stochastic Petri Nets) are defined. Vanishing…

Markov decision processes (MDPs) are the defacto frame-work for sequential decision making in the presence ofstochastic uncertainty. A classical optimization criterion forMDPs is to maximize the expected discounted-sum pay-off, which…

We consider continuous-time Markov chain on a finite state space X. We assume X can be clustered into several subsets such that the intra-transition rates within these subsets are of order $\mathcal{O}(\frac{1}{\epsilon})$ comparing to the…

This article provides the first procedure for computing a fully data-dependent interval that traps the mixing time $t_{\text{mix}}$ of a finite reversible ergodic Markov chain at a prescribed confidence level. The interval is computed from…

Recently regular decision processes have been proposed as a well-behaved form of non-Markov decision process. Regular decision processes are characterised by a transition function and a reward function that depend on the whole history,…

The analysis of high-dimensional time series data has become increasingly important across a wide range of fields. Recently, a method for constructing the minimum information Markov kernel on finite state spaces was established. In this…