Related papers: Decision Making in the Arrow of Time

I consider multipartite processes in which there are constraints on each subsystem's rate matrix, restricting which other subsystems can directly affect its dynamics. I derive a strictly nonzero lower bound on the minimal achievable entropy…

We investigate the decomposition of the total entropy production in continuous stochastic dynamics when there are odd-parity variables that change their signs under time reversal. The first component of the entropy production, which…

We consider a class of stochastic dynamical systems, called piecewise deterministic Markov processes, with states $(x, \s)\in \O\times \G$, $\O$ being a region in $\bbR^d$ or the $d$--dimensional torus, $\G$ being a finite set. The…

We consider a particle moving in continuous time as a Markov jump process; its discrete chain is given by an ordinary random walk on ${\mathbb Z}^d$ , and its jump rate at $({\mathbf x},t)$ is given by a fixed function $\varphi$ of the…

We derive universal bounds for the finite-time survival probability of the stochastic work extracted in steady-state heat engines and the stochastic heat dissipated to the environment. We also find estimates for the time-dependent…

We consider sequences-indexed by time (discrete stages)-of families of multistage stochastic optimization problems. At each time, the optimization problems in a family are parameterized by some quantities (initial states, constraint…

For a Markovian dynamics on discrete states, the logarithmic ratio of waiting-time distributions between two successive, instantaneous transitions in forward and backward direction is a measure of time-irreversibility. It thus serves as an…

First passage time plays a fundamental role in dynamical characterization of stochastic processes. Crucially, our current understanding on the problem is almost entirely relies on the theoretical formulations, which assume the processes…

Living systems often function with regulatory interactions, but the question of how activity, stochasticity and regulations work together for achieving different goals still remains puzzling. We propose a stochastic model of an active…

This paper is devoted to studying the average optimality in continuous-time Markov decision processes with fairly general state and action spaces. The criterion to be maximized is expected average rewards. The transition rates of underlying…

First-passage properties are central to the kinetics of target-search processes. Theoretical approaches so far primarily focused on predicting first-passage statistics for a given process or model. In practice, however, one faces the…

Production systems deteriorate stochastically due to usage and may eventually break down, resulting in high maintenance costs at scheduled maintenance moments. This deterioration behavior is affected by the system's production rate. While…

The amount of information generated by a discrete time stochastic processes in a single step can be quantified by the entropy rate. We investigate the differences between two discrete time walk models, the discrete time quantum walk and the…

A comparison theorem for state-dependent regime-switching diffusion processes is established, which enables us to control pathwisely the evolution of the state-dependent switching component simply by Markov chains. Moreover, a sharp…

Entropy production is often interpreted as a proxy for microscopic disorder or environmental roughness in stochastic systems. We test this interpretation using controlled simulations of overdamped stochastic dynamics on curved surfaces in…

We consider stochastic energy balance and entropy production (EP) in a generalized Langevin dynamics of macrospins, allowing for both amplitude and direction fluctuations, under external magnetic field. EP is calculated using Fokker-Planck…

We address the problem of policy evaluation in discounted Markov decision processes, and provide instance-dependent guarantees on the $\ell_\infty$-error under a generative model. We establish both asymptotic and non-asymptotic versions of…

Entropy estimation, due in part to its connection with mutual information, has seen considerable use in the study of time series data including causality detection and information flow. In many cases, the entropy is estimated using…

We describe the solution of an optimal stopping problem for a stable L\'evy process killed at state-dependent rate, which can be seen as a model for bankruptcy. The killing rate is chosen in such a way that the killed process remains…

Systems with interacting degrees of freedom play a prominent role in stochastic thermodynamics. Our aim is to use the concept of detached path probabilities and detached entropy production for bipartite Markov processes and elaborate on a…