Related papers: State-dependent marginal emission factors with aut…
Energy markets exhibit complex causal relationships between weather patterns, generation technologies, and price formation, with regime changes occurring continuously rather than at discrete break points. Current approaches model…
The participation of electric vehicle (EV) aggregators in real-time electricity markets offers promising revenue opportunities through price-responsive energy arbitrage. A central challenge in economic bidding lies in quantifying the…
We propose a non-linear state-space model to examine the relationship between CO$_2$ emissions, energy sources, and macroeconomic activity, using data from 1971 to 2019. CO$_2$ emissions are modeled as a weighted sum of fossil fuel use,…
Many applications -- including power systems, robotics, and economics -- involve a dynamical system interacting with a stochastic and hard-to-model environment. We adopt a reinforcement learning approach to control such systems.…
The most serious threat to ecosystems is the global climate change fueled by the uncontrolled increase in carbon emissions. In this project, we use mean field control and mean field game models to analyze and inform the decisions of…
Model-based offline reinforcement learning is brittle under distribution shift: policy improvement drives rollouts into state--action regions weakly supported by the dataset, where compounding model error yields severe value overestimation.…
We study Exo-MDPs, a structured class of Markov Decision Processes (MDPs) where the state space is partitioned into exogenous and endogenous components. Exogenous states evolve stochastically, independent of the agent's actions, while…
Because failures in distribution systems caused by extreme weather events directly result in consumers' outages, this paper proposes a state-based decision-making model with the objective of mitigating loss of load to improve the…
This paper studies some temporal dependence properties and addresses the issue of parametric estimation for a class of state-dependent autoregressive models for nonlinear time series in which we assume a stochastic autoregressive…
Accurate prediction of structural dynamics is imperative for preserving digital twin fidelity throughout operational lifetimes. Parametric models with fixed nominal parameters often omit critical physical effects due to simplifications in…
[This paper was initially published in PHME conference in 2016, selected for further publication in International Journal of Prognostics and Health Management.] This paper describes an Autoregressive Partially-hidden Markov model (ARPHMM)…
Markov state models (MSMs) have been broadly adopted for analyzing molecular dynamics trajectories, but the approximate nature of the models that results from coarse-graining into discrete states is a long-known limitation. We show…
We develop a semi-parametric state-space model for time-series data with latent regime transitions. Classical Markov-switching models use fixed parametric transition functions, such as logistic or probit links, which restrict flexibility…
This paper proposes a control strategy for a Reverse Fuel Cell used to manage a Renewable Energy Community. A two-stage scenario-based Model Predictive Control algorithm is designed to define the best economic strategy to be followed during…
Finite-state Markov models are widely used for modeling wireless channels affected by a variety of non-idealities, ranging from shadowing to interference. In an industrial environment, the derivation of a Markov model based on the wireless…
We show that nonequilibrium dynamics can play a constructive role in unsupervised machine learning by inducing the spontaneous emergence of latent-state cycles. We introduce a model in which visible and hidden variables interact through two…
This paper estimates free energy, average mutual information, and minimum mean square error (MMSE) of a linear model under two assumptions: (1) the source is generated by a Markov chain, (2) the source is generated via a hidden Markov…
We develop and apply an approach for analyzing multi-curve data where each curve is driven by a latent state process. The state at any particular point determines a smooth function, forcing the individual curve to switch from one function…
Conditional particle filters (CPFs) with backward/ancestor sampling are powerful methods for sampling from the posterior distribution of the latent states of a dynamic model such as a hidden Markov model. However, the performance of these…
New theoretical results are presented here on the recently introduced model called mixed states MRF. Such models were introduced in the context of image motion analysis and are useful to represent information which can take both discrete…