Related papers: Cycle time of stochastic max-plus linear systems
Random walks find applications in many areas of science and are the heart of essential network analytic tools. When defined on temporal networks, even basic random walk models may exhibit a rich spectrum of behaviours, due to the…
Max-algebra models of tandem single-server queueing systems with both finite and infinite buffers are developed. The dynamics of each system is described by a linear vector state equation similar to those in the conventional linear systems…
We consider deterministic Markov decision processes (MDPs) and apply max-plus algebra tools to approximate the value iteration algorithm by a smaller-dimensional iteration based on a representation on dictionaries of value functions. The…
Natural and social multivariate systems are commonly studied through sets of simultaneous and time-spaced measurements of the observables that drive their dynamics, i.e., through sets of time series. Typically, this is done via hypothesis…
In this article, we study a branching random walk in an environment which depends on the time. This time-inhomogeneous environment consists of a sequence of macroscopic time intervals, in each of which the law of reproduction remains…
A piecewise-deterministic Markov process, specified by random jumps and switching semi-flows, as well as the associated Markov chain given by its post-jump locations, are investigated in this paper. The existence of an exponentially…
We determine the large exceedance probabilities and large exceedance paths for the matrix recursive sequence $V_n = M_n V_{n-1} + Q_n, \: n=1,2,\ldots,$ where $\{M_n\}$ is an i.i.d. sequence of $d \times d$ random matrices and $\{ Q_n\}$ is…
Under some mild condition, a random walk in the plane is recurrent. In particular each trajectory is dense, and a natural question is how much time one needs to approach a given small neighborhood of the origin. We address this question in…
We extend the scope of the dynamical theory of extreme values to cover phenomena that do not happen instantaneously, but evolve over a finite, albeit unknown at the onset, time interval. We consider complex dynamical systems, composed of…
This paper presents a mathematical model for the train dynamics in a mass-transit metro line system with one symmetrically operated junction. We distinguish three parts: a central part and two branches. The tracks are spatially discretized…
We develop a theory of operator renewal sequences in the context of infinite ergodic theory. For large classes of dynamical systems preserving an infinite measure, we determine the asymptotic behaviour of iterates $L^n$ of the transfer…
Recent measurements of durations of non-equilibrium processes provide valuable information on microscopic mechanisms and energetics. Comprehensive theory for corresponding experiments so far is well developed for single-particle systems…
Iterative trajectory optimization techniques for non-linear dynamical systems are among the most powerful and sample-efficient methods of model-based reinforcement learning and approximate optimal control. By leveraging time-variant local…
The problem of making sequential decisions in unknown probabilistic environments is studied. In cycle $t$ action $y_t$ results in perception $x_t$ and reward $r_t$, where all quantities in general may depend on the complete history. The…
Reversible Probabilistic Cellular Automata are a special class of automata whose stationary behavior is described by Gibbs-like measures. For those models the dynamics can be trapped for a very long time in states which are very different…
Linear systems with many degrees of freedom containing multiplicative and additive noise are considered. The steady state probability distribution for equations of this kind is examined. With multiplicative white noise it is shown that…
This paper introduces the novel class of modulated cyclostationary processes, a class of non-stationary processes exhibiting frequency coupling, and proposes a method of their estimation from repeated trials. Cyclostationary processes also…
An overdamped system with a linear restoring force and two multiplicative colored noises is considered. Noise amplitudes depend on the system state $x$ as $x$ and $|x|^{\alpha}$. An exactly soluble model of a system is constructed due to…
In this paper, we establish a general stochastic maximum principle for optimal control for systems described by a continuous-time Markov regime-switching stochastic recursive utilities model. The control domain is postulated not to be…
We study a class of dynamical systems generated by random substitutions, which contains both intrinsically ergodic systems and instances with several measures of maximal entropy. In this class, we show that the measures of maximal entropy…