Related papers: Markov-chain approach to a process with long-time …
Discrete dynamics arise naturally in systems with broken temporal translation symmetry and are typically described by first-order recurrence relations representing classical or quantum Markov chains. When memory effects induced by hidden…
We consider the problem of optimally designing a system for repeated use under uncertainty. We develop a modeling framework that integrates design and operational phases, which are represented by a mixed-integer program and discounted-cost…
In recent years, there has been considerable interest in understanding the motion in Hamiltonian systems when phase space is divided into stochastic and integrable regions. This paper studies one aspect of this problem, namely, the motion…
We introduce a new class of stochastic processes which are stationary, Markovian and characterized by an infinite range of time-scales. By transforming the Fokker-Planck equation of the process into a Schrodinger equation with an…
In this work, we focus on the stationary analysis of a specific class of continuous time Markov-modulated reflected random walks in the quarter plane with applications in the modelling of two-node Markov-modulated queueing networks with…
The stochastic trajectories of molecules in living cells, as well as the dynamics in many other complex systems, often exhibit memory in their path over long periods of time. In addition, these systems can show dynamic heterogeneities due…
We observe a length-$n$ sample generated by an unknown,stationary ergodic Markov process (\emph{model}) over a finite alphabet $\mathcal{A}$. Given any string $\bf{w}$ of symbols from $\mathcal{A}$ we want estimates of the conditional…
In a quantum (inhomogeneous) Markov process $\rho_1:=\Gamma_1(\rho)$, $\rho_2:=\Gamma_1(\rho_1)$, ..., where $\Gamma_i$ are CPTP maps and $\rho$ is the initial state, the the state of the system is either oscillatory or convergent to a…
Changing time of simple continuous-time Markov counting processes by independent unit-rate Poisson processes results in Markov counting processes for which we provide closed-form transition rates via composition of trajectories and with…
We consider a stationary Markov process that models certain queues with a bulk service of a fixed number $m$ of admitted customers. We find an integral expression of its transition probability function in terms of certain multi-orthogonal…
We establish general theorems quantifying the notion of recurrence --- through an estimation of the moments of passage times --- for irreducible continuous-time Markov chains on countably infinite state spaces. Sharp conditions of…
A semi-Markov process is one that changes states in accordance with a Markov chain but takes a random amount of time between changes. We consider the generalisation to semi-Markov processes of the classical Lamperti law for the occupation…
in the last decade, studies of chaotic system are more often used for classical choatic system than for quantum chaotic system, there are many ways of observing the chaotic system such us analyzing the frequency with Fourier transform or…
This paper considers the queueing performance of a system that transmits coded data over a time-varying erasure channel. In our model, the queue length and channel state together form a Markov chain that depends on the system parameters.…
Markov decision processes continue to gain in popularity for modeling a wide range of applications ranging from analysis of supply chains and queuing networks to cognitive science and control of autonomous vehicles. Nonetheless, they tend…
This is an expository review paper illustrating the ``martingale method'' for proving many-server heavy-traffic stochastic-process limits for queueing models, supporting diffusion-process approximations. Careful treatment is given to an…
We introduce discrete time Markov chains that preserve uniform measures on boxed plane partitions. Elementary Markov steps change the size of the box from (a x b x c) to ((a-1) x (b+1) x c) or ((a+1) x (b-1) x c). Algorithmic realization of…
It is quite clear from a wide range of experiments that gating phenomena of ion channels is inherently stochastic. It has been discussed using BD simulations in a recent paper that memory effects in ion transport is negligible, unless the…
We investigate a processor sharing queue with renewal arrivals and generally distributed service times. Impatient jobs may abandon the queue, or renege, before completing service. The corresponding stochastic processes are represented by…
The paper presents a multidimensional model for nonlinear Markovian random walks that generalizes one we developed previously (Phys. Rev. E v.79, 011110, 2009) in order to describe the Levy type stochastic processes in terms of continuous…