Related papers: Efficient Sensitivity Analysis for Parametric Robu…

Stochastic gradient methods are the workhorse (algorithms) of large-scale optimization problems in machine learning, signal processing, and other computational sciences and engineering. This paper studies Markov chain gradient descent, a…

We develop a new bidirectional algorithm for estimating Markov chain multi-step transition probabilities: given a Markov chain, we want to estimate the probability of hitting a given target state in $\ell$ steps after starting from a given…

We present on-line policy gradient algorithms for computing the locally optimal policy of a constrained, average cost, finite state Markov Decision Process. The stochastic approximation algorithms require estimation of the gradient of the…

This paper deals with the problem of estimating second-order parameter sensitivities for stochastic reaction networks, where the reaction dynamics is modeled as a continuous time Markov chain over a discrete state space. Estimation of such…

Suppose we observe a geometrically ergodic semi-Markov process and have a parametric model for the transition distribution of the embedded Markov chain, for the conditional distribution of the inter-arrival times, or for both. The first two…

We study the causal distributionally robust optimization (DRO) in both discrete- and continuous- time settings. The framework captures model uncertainty, with potential models penalized in function of their adapted Wasserstein distance to a…

A random walk-based method is proposed to efficiently compute the solution of a large class of fractional in time linear systems of differential equations (linear F-ODE systems), along with the derivatives with respect to the system…

The parameters of a discrete stationary Markov model are transition probabilities between states. Traditionally, data consist in sequences of observed states for a given number of individuals over the whole observation period. In such a…

Estimating online the parameters of general state-space hidden Markov models is a topic of importance in many scientific and engineering disciplines. In this paper we present an online parameter estimation algorithm obtained by casting our…

In this paper, we consider the stability analysis of large-scale distributed networked control systems with random communication delays between linearly interconnected subsystems. The stability analysis is performed in the Markov jump…

We devise a Monte Carlo based method for detecting whether a non-negative Markov chain is stable for a given set of parameter values. More precisely, for a given subset of the parameter space, we develop an algorithm that is capable of…

There exist many methods for sensitivity analysis readily available to the practitioner. While each seeks to help the modeler answer the same general question -- How do sources of uncertainty or changes in the model inputs relate to…

We present a Markov-chain analysis of blockwise-stochastic algorithms for solving partially block-separable optimization problems. Our main contributions to the extensive literature on these methods are statements about the Markov operators…

We consider a robust approach to address uncertainty in model parameters in Markov Decision Processes (MDPs), which are widely used to model dynamic optimization in many applications. Most prior works consider the case where the uncertainty…

Semi-Markov models are widely used for survival analysis and reliability analysis. In general, there are two competing parameterizations and each entails its own interpretation and inference properties. On the one hand, a semi-Markov…

Parameter estimation is one of the most important tasks in statistics, and is key to helping people understand the distribution behind a sample of observations. Traditionally parameter estimation is done either by closed-form solutions…

We present a deep transformation model for probabilistic regression. Deep learning is known for outstandingly accurate predictions on complex data but in regression tasks, it is predominantly used to just predict a single number. This…

The two-parameter Macdonald polynomials are a central object of algebraic combinatorics and representation theory. We give a Markov chain on partitions of k with eigenfunctions the coefficients of the Macdonald polynomials when expanded in…

This report proposes a novel framework for a rigorous robustness analysis of stochastic biochemical systems. The technique is based on probabilistic model checking. We adapt the general definition of robustness introduced by Kitano to the…

Verification of infinite-state Markov chains is still a challenge despite several fruitful numerical or statistical approaches. For decisive Markov chains, there is a simple numerical algorithm that frames the reachability probability as…