Related papers: Lecture Notes on Stochastic Processes

This paper presents a general approach to linear stochastic processes driven by various random noises. Mathematically, such processes are described by linear stochastic differential equations of arbitrary order (the simplest non-trivial…

A stochastic process that arises by composing a function with a Markov process is called an aggregated Markov process (AMP). The purpose of composing a Markov process with a function can be a reduction of dimensions, e.g., a projection onto…

Markov matrices have an important role in the filed of stochastic processes. In this paper, we will show and prove a series of conclusions on Markov matrices and transformations rather than pay attention to stochastic processes although…

The discrete class algorithm presented in this paper is an efficient simulation tool for stochastic processes governed by a reasonably small set of transition rates. The algorithm is presented, its performance compared to prevailing methods…

In this review-type paper written at the occasion of the Oberwolfach workshop {\em One-sided vs. Two-sided stochastic processes} (february 22-29, 2020), we discuss and compare Markov properties and generalisations thereof in more…

These are lecture notes written at the University of Zurich during spring 2014 and spring 2015. The first part of the notes gives an introduction to probability theory. It explains the notion of random events and random variables,…

In this paper, we abstract a kind of stochastic processes from evolving processes of growing networks, this process is called growing network Markov chains. Thus the existence and the formulas of degree distribution are transformed to the…

Within systems biology there is an increasing interest in the stochastic behavior of genetic and biochemical reaction networks. An appropriate stochastic description is provided by the chemical master equation, which represents a continuous…

These notes were used in a short graduate course on branching processes the author gave in Beijing Normal University. The following main topics are covered: scaling limits of Galton--Watson processes, continuous-state branching processes,…

This article presents a short and concise description of stochastic approximation algorithms in reinforcement learning of Markov decision processes. The algorithms can also be used as a suboptimal method for partially observed Markov…

This manuscript is an edited and refined version of the lecture script for a one-semester graduate course given originally at the PhD school in the Institute of Physics of Polish Academy of Sciences in the Spring/Summer semester of 2022.…

Dynamical processes can be transformed into graphs through a family of mappings called visibility algorithms, enabling the possibility of (i) making empirical data analysis and signal processing and (ii) characterising classes of dynamical…

These lecture notes are derived from a graduate-level course in dynamic optimization, offering an introduction to techniques and models extensively used in management science, economics, operations research, engineering, and computer…

We present the observation that the process of stochastic model predictive control can be formulated in the framework of iterated function systems. The latter has a rich ergodic theory that can be applied to study the system's long-run…

Reinforced processes are known to provide a stochastic representation for the quasi-stationary distribution of a given killed Markov process - describing the killed Markov process at fixed time instants. In this paper we shall adapt the…

We consider stochastic thermodynamics as a theory of statistical inference for experimentally observed fluctuating time-series. To that end, we introduce a general framework for quantifying the knowledge about the dynamical state of the…

These notes provide a review of basic stochastic population models including branching processes and models of population genetics. Measure-valued population models including superprocesses and Fleming-Viot processes are also introduced…

Recently, a class of stochastic processes known as piecewise deterministic Markov processes has been used to define continuous-time Markov chain Monte Carlo algorithms with a number of attractive properties, including compatibility with…

In this paper, we study a class of stochastic processes, called evolving network Markov chains, in evolving networks. Our approach is to transform the degree distribution problem of an evolving network to a corresponding problem of evolving…

The applicability of machine learning for predicting chaotic dynamics relies heavily upon the data used in the training stage. Chaotic time series obtained by numerically solving ordinary differential equations embed a complicated noise of…