Related papers: Multi-step Uniformization with Steady-State Detect…
Continuous-time Markov chains are mathematical models that are used to describe the state-evolution of dynamical systems under stochastic uncertainty, and have found widespread applications in various fields. In order to make these models…
This paper contributes an in-depth study of properties of continuous time Markov chains (CTMCs) on non-negative integer lattices $\N_0^d$, with particular interest in one-dimensional CTMCs with polynomial transitions rates. Such stochastic…
Consistency Models (CMs) have shown promise for efficient one-step generation. However, most existing CMs rely on manually designed discretization schemes, which can cause repeated adjustments for different noise schedules and datasets. To…
We investigate sequential change point estimation and detection in univariate nonparametric settings, where a stream of independent observations from sub-Gaussian distributions with a common variance factor and piecewise-constant but…
In this paper, we study the simultaneous stability problem of a finite number of locally inter-connected linear subsystems under practical constraints, including asynchronous and aperiodic sampling, time-varying delays, and measurement…
This paper considers the problem of joint change detection and identification assuming multiple composite postchange hypotheses. We propose a multihypothesis changepoint detection-identification procedure that controls the probabilities of…
Given a finite sequence of graphs, e.g., coming from technological, biological, and social networks, the paper proposes a methodology to identify possible changes in stationarity in the stochastic process generating the graphs. In order to…
Sliding mode control (SMC) is a robust and computationally efficient solution for tracking control problems of highly nonlinear systems with a great deal of uncertainty. High frequency oscillations due to chattering phenomena and…
A spectral method for identifying lumping in large Markov chains is presented. Identification of meta stable states is treated as a special case. The method is based on spectral analysis of a self-adjoint matrix that is a function of the…
The problem of detection and possible estimation of a signal generated by a dynamic system when a variable number of noisy measurements can be taken is here considered. Assuming a Markov evolution of the system (in particular, the pair…
We study the concentration phenomenon for discrete-time random dynamical systems with an unbounded state space. We develop a heuristic approach towards obtaining exponential concentration inequalities for dynamical systems using an entirely…
A classical problem for Markov chains is determining their stationary (or steady-state) distribution. This problem has an equally classical solution based on eigenvectors and linear equation systems. However, this approach does not scale to…
This paper analyzes a (1, $\lambda$)-Evolution Strategy, a randomized comparison-based adaptive search algorithm, optimizing a linear function with a linear constraint. The algorithm uses resampling to handle the constraint. Two cases are…
Stochastic models are often used to help understand the behavior of intracellular biochemical processes. The most common such models are continuous time Markov chains (CTMCs). Parametric sensitivities, which are derivatives of expectations…
This paper is dedicated to the investigation of a new numerical method to approximate the optimal stopping problem for a discrete-time continuous state space Markov chain under partial observations. It is based on a two-step discretization…
This paper considers the stabilization of unknown switched linear systems using data. Instead of a full system model, we have access to a finite number of trajectories of each of the different modes prior to the online operation of the…
Learning a Markov Decision Process (MDP) from a fixed batch of trajectories is a non-trivial task whose outcome's quality depends on both the amount and the diversity of the sampled regions of the state-action space. Yet, many MDPs are…
This paper studies the cooperative global robust stabilization problem for a class of nonlinear multi-agent systems. The problem is motivated from the study of the cooperative global robust output regulation problem for the class of…
In this paper, we develop a stochastic algorithm based on Euler-Maruyama scheme to approximate the invariant measure of the limiting multidimensional diffusion of the $M/Ph/n+M$ queue. Specifically, we prove a non-asymptotic error bound…
The convergence behavior of Stochastic Gradient Descent (SGD) crucially depends on the stepsize configuration. When using a constant stepsize, the SGD iterates form a Markov chain, enjoying fast convergence during the initial transient…