相关论文: Markovian Testing Equivalence and Exponentially Ti…
We study the finite-time convergence of TD learning with linear function approximation under Markovian sampling. Existing proofs for this setting either assume a projection step in the algorithm to simplify the analysis, or require a fairly…
We apply general moment identities for Poisson stochastic integrals with random integrands to the computation of the moments of Markovian growth-collapse processes. This extends existing formulas for mean and variance available in the…
Mechanistic modelling of animal movement is often formulated in discrete time despite problems with scale invariance, such as handling irregularly timed observations. A natural solution is to formulate in continuous time, yet uptake of this…
We prove a boundary Harnack inequality for jump-type Markov processes on metric measure state spaces, under comparability estimates of the jump kernel and Urysohn-type property of the domain of the generator of the process. The result holds…
We consider Markov decision processes with synchronizing objectives, which require that a probability mass of $1-\epsilon$ accumulates in a designated set of target states, either once, always, infinitely often, or always from some point…
In the variational approach to statistical mechanics, equilibrium states are the rigorous analogues of thermodynamic phases; the question of which invariant measures can arise as equilibrium states is therefore the question of which phases…
Quantum mechanics, in principle, allows for processes with indefinite causal order. However, most of these causal anomalies have not yet been detected experimentally. We show that every such process can be simulated experimentally by means…
A randomized trial and an analysis of observational data designed to emulate the trial sample observations separately, but have the same eligibility criteria, collect information on some shared baseline covariates, and compare the effects…
Game-theoretic characterizations of process equivalences traditionally form a central topic in concurrency; for example, most equivalences on the classical linear-time / branching-time spectrum come with such characterizations. Recent work…
The Markov assumption in Markov Decision Processes (MDPs) is fundamental in reinforcement learning, influencing both theoretical research and practical applications. Existing methods that rely on the Bellman equation benefit tremendously…
This article considers the average optimality for a continuous-time Markov decision process with Borel state and action spaces and an arbitrarily unbounded nonnegative cost rate. The existence of a deterministic stationary optimal policy is…
We consider the issue of non-Markovianity of a quantum dynamics starting from a comparison with the classical definition of Markovian process. We point to the fact that two sufficient but not necessary signatures of non-Markovianity of a…
This paper is devoted to studying the average optimality in continuous-time Markov decision processes with fairly general state and action spaces. The criterion to be maximized is expected average rewards. The transition rates of underlying…
We present a new algorithm for the statistical model checking of Markov chains with respect to unbounded temporal properties, such as reachability and full linear temporal logic. The main idea is that we monitor each simulation run on the…
We provide a framework for speeding up algorithms for time-bounded reachability analysis of continuous-time Markov decision processes. The principle is to find a small, but almost equivalent subsystem of the original system and only analyse…
The exploration of far-from-equilibrium systems has been at the forefront of nonequilibrium thermodynamics, with a particular focus on understanding the fluctuations and response of thermodynamic systems to external perturbations. In this…
We show that the nonequilibrium response of Markovian dynamics can be made fully symmetric, both near and far from equilibrium. This is achieved by varying the affinities along equivalence classes in the space of stochastic dynamics.
The nonequilibrium response and fluctuations of Markovian systems, both near and far from equilibrium, are best understood by varying their parameters along dynamical equivalence classes. In this note, I illustrate this approach for an…
This work develops a rigorous framework for analysing ergodicity and mixing in time-inhomogeneous quantum dynamics. It considers quantum evolutions generated by sequences of quantum channels and examines in detail the relationship between…
We consider the problem of conditioning a Markov process on a rare event and of representing this conditioned process by a conditioning-free process, called the effective or driven process. The basic assumption is that the rare event used…