Related papers: Recovering Markov Models from Closed-Loop Data

We study episodic reinforcement learning in Markov decision processes when the agent receives additional feedback per step in the form of several transition observations. Such additional observations are available in a range of tasks…

This paper introduces an algorithm for discovering implicit and delayed causal relations between events observed by a robot at arbitrary times, with the objective of improving data-efficiency and interpretability of model-based…

In this paper, we consider statistical estimation of time-inhomogeneous aggregate Markov models. Unaggregated models, which corresponds to Markov chains, are commonly used in multi-state life insurance to model the biometric states of an…

In order to evaluate and compare different recursive filters, simulation is a common tool and numerous simulation models are widely used as 'benchmark'. In the simulation, the continuous time dynamic system is converted into a discrete-time…

Critical infrastructure systems such as electric power networks, water networks, and transportation systems play a major role in the welfare of any community. In the aftermath of disasters, their recovery is of paramount importance; orderly…

Hidden Markov models have successfully been applied as models of discrete time series in many fields. Often, when applied in practice, the parameters of these models have to be estimated. The currently predominating identification methods,…

Feedback optimization has emerged as an effective strategy for steady-state optimization of dynamical systems. By exploiting models of the steady-state input-output sensitivity, methods of this type are often sample efficient, and their use…

Predicting the near-future delay with accuracy for trains is momentous for railway operations and passengers' traveling experience. This work aims to design prediction models for train delays based on Netherlands Railway data. We first…

We propose a Bayesian inference approach for a class of latent Markov models. These models are widely used for the analysis of longitudinal categorical data, when the interest is in studying the evolution of an individual unobservable…

The hidden Markov model (HMM) provides a powerful framework for inference in time-varying environments, where the underlying state evolves according to a Markov chain. To address the optimal filtering problem in general dynamic settings, we…

This paper studies the item-to-item recommendation problem in recommender systems from a new perspective of metric learning via implicit feedback. We develop and investigate a personalizable deep metric model that captures both the internal…

A standard model for Recommender Systems is the Matrix Completion setting: given partially known matrix of ratings given by users (rows) to items (columns), infer the unknown ratings. In the last decades, few attempts where done to handle…

We propose a method for learning cyclic causal models from a combination of observational and interventional equilibrium data. Novel aspects of the proposed method are its ability to work with continuous data (without assuming linearity)…

This report presents an Expectation-Maximization (EM) algorithm for estimation of the maximum-likelihood parameter values of constrained multivariate autoregressive Gaussian state-space (MARSS) models. The MARSS model can be written:…

The goal of recommender systems is to provide ordered item lists to users that best match their interests. As a critical task in the recommendation pipeline, re-ranking has received increasing attention in recent years. In contrast to…

Recommender systems play an essential role in the choices people make in domains such as entertainment, shopping, food, news, employment, and education. The machine learning models underlying these recommender systems are often enormously…

Algorithmic Recourse aims to provide actionable explanations, or recourse plans, to overturn potentially unfavourable decisions taken by automated machine learning models. In this paper, we propose an interaction paradigm based on a guided…

Performance evaluation of nursing homes is usually accomplished by the repeated administration of questionnaires aimed at measuring the health status of the patients during their period of residence in the nursing home. We illustrate how a…

Latent Markov (LM) models represent an important tool of analysis of longitudinal data when response variables are affected by time-varying unobserved heterogeneity, which is accounted for by a hidden Markov chain. In order to avoid bias…

We develop a recursion for hidden Markov model of any order h, which allows us to obtain the posterior distribution of the latent state at every occasion, given the previous h states and the observed data. With respect to the well-known…