Related papers: Lecture Notes on Stochastic Processes

In this paper, we extend the results of Elliott and Yang \cite{elliott3} and discuss the control of a stochastic process for which the driving noise is provided by a martingale associated with a semi-Markov Chain. An existence and a…

Stochastic processes offer a flexible mathematical formalism to model and reason about systems. Most analysis tools, however, start from the premises that models are fully specified, so that any parameters controlling the system's dynamics…

We study a class of multi-stage stochastic programs, which incorporate modeling features from Markov decision processes (MDPs). This class includes structured MDPs with continuous action and state spaces. We extend policy graphs to include…

The growing attention on cryptocurrencies has led to increasing research on digital stock markets. Approaches and tools usually applied to characterize standard stocks have been applied to the digital ones. Among these tools is the…

This is a collection of lecture notes from three lectures given by Alexei Kitaev at the 2008 Les Houches summer school "Exact methods in low-dimensional physics and quantum computing." They provide a pedagogical introduction to topological…

We use nowdays classical theory of generalized moment problems by Krein-Nudelman [1977] to define a special class of stochastic Gaussian processes. The class contains, of course, stationary Gaussian processes. We obtain a spectral…

This PhD thesis deals with the Markov picture of developed turbulence from the theoretical point of view. The thesis consists of two parts. The first part introduces stochastic thermodynamics, the second part aims at transferring the…

The goal of this paper is to analyze distributional Markov Decision Processes as a class of control problems in which the objective is to learn policies that steer the distribution of a cumulative reward toward a prescribed target law,…

These lecture notes are an informal introduction to the theory of computational complexity and its links to quantum computing and statistical mechanics.

Using the white noise space setting, we define and study stochastic integrals with respect to a class of stationary increment Gaussian processes. We focus mainly on continuous functions with values in the Kondratiev space of stochastic…

In this paper, a general stochastic model with controls applied at the moments when the random process hits the boundary of a given subset of the state set is proposed and studied. The general concept of the model is formulated and its…

We introduce and summarise results from the recent paper `Scaling limits of stochastic processes associated with resistance forms', and also applications from `Time-changes of stochastic processes associated with resistance forms', which…

These lecture notes were prepared as a basic introduction to the theory of constrained systems which is how the fundamental forces of nature appear in their Hamiltonian formulation. Only a working knowledge of Lagrangian and Hamiltonian…

These lecture notes are based on lectures given in 2019 Saint-Flour Probability School.

Stochastic thermodynamics provides a framework for describing small systems like colloids or biomolecules driven out of equilibrium but still in contact with a heat bath. Both, a first-law like energy balance involving exchanged heat and…

This chapter presents an introduction to Markovian modeling for the analysis of sequence data. Contrary to the deterministic approach seen in the previous sequence analysis chapters, Markovian models are probabilistic models, focusing on…

In this paper we revisit the notion of the "minus logarithm of stationary probability" as a generalized potential in nonequilibrium systems and attempt to illustrate its central role in an axiomatic approach to stochastic nonequilibrium…

This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…

Building upon the rule-algebraic stochastic mechanics framework, we present new results on the relationship of stochastic rewriting systems described in terms of continuous-time Markov chains, their embedded discrete-time Markov chains and…

We consider a class of piecewise-deterministic Markov processes where the state evolves according to a linear dynamical system. This continuous time evolution is interspersed by discrete events that occur at random times and change (reset)…