Related papers: Entropy Semiring Forward-backward Algorithm for HM…

The Hidden Markov Model (HMM) is one of the most widely used statistical models for sequential data analysis. One of the key reasons for this versatility is the ability of HMM to deal with missing data. However, standard HMM learning…

Large Reasoning Models (LRMs) excel at complex reasoning tasks through extended chain-of-thought generation, but their reliance on lengthy intermediate steps incurs substantial computational cost. We find that the entropy of the model's…

This paper presents a novel methodology for modelling precipitation patterns in a specific geographical region using Hidden Markov Models (HMMs). Departing from conventional HMMs, where the hidden state process is assumed to be Markovian,…

We derive the Baum-Welch algorithm for hidden Markov models (HMMs) through an information-theoretical approach using cross-entropy instead of the Lagrange multiplier approach which is universal in machine learning literature. The proposed…

Using supervised machine learning approaches to recognize human activities from on-body wearable accelerometers generally requires a large amount of labelled data. When ground truth information is not available, too expensive, time…

Based on the recently developed resummation-based quantum Monte Carlo method for the SU($N$) spin and loop-gas models, we develop a new algorithm, dubbed ResumEE, to compute the entanglement entropy (EE) with greatly enhanced efficiency.…

The long-tailed distribution of sequence lengths in LLM serving and reinforcement learning (RL) sampling causes significant computational waste due to excessive padding in batched inference. Existing methods rely on auxiliary models for…

Forecasting nonlinear time series with multi-scale temporal structures remains a central challenge in complex systems modeling. We present a novel reservoir computing framework that combines delay embedding with random Fourier feature (RFF)…

Recent masked diffusion models (MDMs) have shown competitive performance compared to autoregressive models (ARMs) for language modeling. While most literature has focused on performance enhancing sampling procedures, efficient sampling from…

This paper develops a new exponential forgetting algorithm that can prevent so-called the estimator windup problem, while retaining fast convergence speed. To investigate the properties of the proposed forgetting algorithm, boundedness of…

Energy-based models (EBMs) are a powerful class of probabilistic generative models due to their flexibility and interpretability. However, relationships between potential flows and explicit EBMs remain underexplored, while contrastive…

The hidden Markov model (HMM) is a widely-used generative model that copes with sequential data, assuming that each observation is conditioned on the state of a hidden Markov chain. In this paper, we derive a novel algorithm to cluster HMMs…

The inverse-free extreme learning machine (ELM) algorithm proposed in [4] was based on an inverse-free algorithm to compute the regularized pseudo-inverse, which was deduced from an inverse-free recursive algorithm to update the inverse of…

The objective of drug discovery is to identify chemical compounds that possess specific pharmaceutical properties toward a binding target. Existing large language models (LLMS) can achieve high token matching scores in terms of likelihood…

Reservoir computing (RC) is an effective method for predicting chaotic systems by using a high-dimensional dynamic reservoir with fixed internal weights, while keeping the learning phase linear, which simplifies training and reduces…

We study the generalized forward-reflected-backward (GFRB) method, an extension of the forward-reflected-backward (FRB) scheme due to Malitsky and Tam, for solving monotone inclusion problems in real Hilbert spaces. We first analyze GFRB…

Reduced Basis Methods (RBMs) are frequently proposed to approximate parametric problem solutions. They can be used to calculate solutions for a large number of parameter values (e.g. for parameter fitting) as well as to approximate a…

Matrix factorization methods are linear models, with limited capability to model complex relations. In our work, we use tropical semiring to introduce non-linearity into matrix factorization models. We propose a method called Sparse…

Entanglement entropy is a useful probe of compressible quantum matter because it can detect the existence of Fermi surfaces, both of microscopic fermionic degrees of freedom and of "hidden" gauge charged fermions. Much recent attention has…

This paper presents an entropy based adaptive measurement sequence strategy for quantum sensing of magnetic fields. To physically ground our ideas we consider a sensor employing a nitrogen vacancy center in diamond, however our approach is…