相关论文: Computing the minimal rebinding effect for non-rev…

Resetting or restart, when applied to a stochastic process, usually brings its dynamics to a time-independent stationary state. In turn, the optimal resetting rate makes the mean time to reach a target to be the shortest one. These and…

In a circular convolution model, we aim to infer on the density of a circular random variable using observations contaminated by an additive measurement error. We highlight the interplay of the two problems: optimal testing and quadratic…

We study the relaxation of a Brownian particle with long range memory under confinement in one dimension. The particle diffuses in an arbitrary confining potential and resets at random times to previously visited positions, chosen with a…

This paper studies value iteration for infinite horizon contracting Markov decision processes under convexity assumptions and when the state space is uncountable. The original value iteration is replaced with a more tractable form and the…

An abundance of recent impossibility results establish that regret minimization in Markov games with adversarial opponents is both statistically and computationally intractable. Nevertheless, none of these results preclude the possibility…

In this paper we consider a control problem for a Partially Observable Piecewise Deterministic Markov Process of the following type: After the jump of the process the controller receives a noisy signal about the state and the aim is to…

The perceived advantage of machine learning (ML) models is that they are flexible and can incorporate a large number of features. However, many of these are typically correlated or dependent, and incorporating all of them can hinder model…

We study the emergence of quantum memory effects in a spin-boson system at finite temperature driven by an external time-periodic force. Quantifying memory effects by the trace-distance based measure for non-Markovianity and performing…

For the continuous-time and the discrete-time three-state hidden Markov model, the flux of the likelihood function up to 3-dimension of the observed process is shown explicitly. As an application, the sufficient and necessary condition of…

The effects of a stochastic reset, to its initial configuration, is studied in the exactly solvable one-dimensional coagulation-diffusion process. A finite resetting rate leads to a modified non-equilibrium stationary state. If in addition…

We consider the class of Hopfield models of associative memory with activation function $F$ and state space $\{-1,1\}^N$, where each vertex of the cube describes a configuration of $N$ binary neurons. $M$ randomly chosen configurations,…

We study minority games in efficient regime. By incorporating the utility function and aggregating agents with similar strategies we develop an effective mesoscale notion of state of the game. Using this approach, the game can be…

In typical single-molecule force spectroscopy experiments the mechanical unfolding of molecular complexes or biomolecules is studied applying a force ramp to one end of the system while the other end is kept fixed in space. The…

We establish a connection between policy evaluation in Markov decision processes and PageRank in network analysis. For a fixed policy, we show that the value function of a discounted Markov decision process can be obtained, up to an…

Motivated by wide-ranging applications such as video delivery over networks using Multiple Description Codes, congestion control, and inventory management, we study the state-tracking of a Markovian random process with a known transition…

A Markov decision process can be parameterized by a transition kernel and a reward function. Both play essential roles in the study of reinforcement learning as evidenced by their presence in the Bellman equations. In our inquiry of various…

Web crawling, snowball sampling, and respondent-driven sampling (RDS) are three types of network sampling techniques used to contact individuals in hard-to-reach populations. This paper studies these procedures as a Markov process on the…

We study infinite horizon Markov decision processes (MDPs) with "fast-slow" structure, where some state variables evolve rapidly ("fast states") while others change more gradually ("slow states"). This structure commonly arises in practice…

In the nonlinear prediction of scalar time series, the common practice is to reconstruct the state space using time-delay embedding and apply a local model on neighborhoods of the reconstructed space. The method of false nearest neighbors…

This work is a continuation of [Kalikaeva, MPRF, 23(2):225-240]. The object of study is ``Markov-up processes'' on $\mathbb Z_+$ and the moment of downcrossing a certain barrier. The processes considered in this paper differ from Markov…