Related papers: Alternating Good-for-MDP Automata
Gradient-based methods for optimisation of objectives in stochastic settings with unknown or intractable dynamics require estimators of derivatives. We derive an objective that, under automatic differentiation, produces low-variance…
We present an efficient Angluin-style learning algorithm for weak deterministic B\"uchi automata (wDBAs). Different to ordinary deterministic B\"uchi and co-B\"uchi automata, wDBAs have a minimal normal form, and we show that we can learn…
We present a direct transformation of weak alternating $\omega$-automata into equivalent backward deterministic $\omega$-automata and show (1) how it can be used to obtain a transformation of non-deterministic B\"uchi automata into…
Regular decision processes (RDPs) are a subclass of non-Markovian decision processes where the transition and reward functions are guarded by some regular property of the past (a lookback). While RDPs enable intuitive and succinct…
A nondeterministic automaton is semantically deterministic (SD) if different nondeterministic choices in the automaton lead to equivalent states. Semantic determinism is interesting as it is a natural relaxation of determinism, and as some…
In this paper we revisit Safra's determinization constructions for automata on infinite words. We show how to construct deterministic automata with fewer states and, most importantly, parity acceptance conditions. Determinization is used in…
Abstract machines for strong evaluation of the $\lambda$-calculus enter into arguments and have a set of transitions for backtracking out of an evaluated argument. We study a new abstract machine which avoids backtracking by splitting the…
We give a new version of fuzzy alternating $\mathrm{B\ddot{u}chi}$ automata over distributive lattices: weights are putting in every leaf node of run trees rather than along with edges from every node to its children. Such settings are…
This paper studies optimal motion planning subject to motion and environment uncertainties. By modeling the system as a probabilistic labeled Markov decision process (PL-MDP), the control objective is to synthesize a finite-memory policy,…
Deterministic timed automata are strictly less expressive than their non-deterministic counterparts, which are again less expressive than those with silent transitions. As a consequence, timed automata are in general non-determinizable.…
We consider the representational state complexity of unranked tree automata. The bottom-up computation of an unranked tree automaton may be either deterministic or nondeterministic, and further variants arise depending on whether the…
A finite automaton is called bideterministic if it is both deterministic and codeterministic -- that is, if it is deterministic and its transpose is deterministic as well. The study of such automata in a weighted setting is initiated. All…
Multi-armed bandits (MAB) model sequential decision making problems, in which a learner sequentially chooses arms with unknown reward distributions in order to maximize its cumulative reward. Most of the prior work on MAB assumes that the…
Transformer-based trajectory optimization methods have demonstrated exceptional performance in offline Reinforcement Learning (offline RL). Yet, it poses challenges due to substantial parameter size and limited scalability, which is…
In this paper, we introduce rerailing automata for $\omega$-regular languages. They generalize both deterministic parity (DPW) and minimized history-deterministic co-B\"uchi automata (with transition based acceptance, HdTbcBW) while…
A Markov Decision Process (MDP) is a popular model for reinforcement learning. However, its commonly used assumption of stationary dynamics and rewards is too stringent and fails to hold in adversarial, nonstationary, or multi-agent…
A two-dimensional finite automaton has a read-only input head that moves in four directions on a finite array of cells labelled by symbols of the input alphabet. A three-way two-dimensional automaton is prohibited from making upward moves,…
We study alternating automata with qualitative semantics over infinite binary trees: alternation means that two opposing players construct a decoration of the input tree called a run, and the qualitative semantics says that a run of the…
The multi-armed bandit (MAB) problem is a classic example of the exploration-exploitation dilemma. It is concerned with maximising the total rewards for a gambler by sequentially pulling an arm from a multi-armed slot machine where each arm…
In this paper, we propose a novel algorithm to learn a B\"uchi automaton from a teacher who knows an $\omega$-regular language. The algorithm is based on learning a formalism named family of DFAs (FDFAs) recently proposed by Angluin and…