Related papers: Soft-to-Hard Routing in Sparse Mixture-of-Experts …

A dynamic routing policy, referred to as Heat-Diffusion (HD), is developed for multihop uniclass wireless networks subject to random traffic, time-varying topology and inter-channel interference.The policy uses only current condition of…

With an eye towards human-centered automation, we contribute to the development of a systematic means to infer features of human decision-making from behavioral data. Motivated by the common use of softmax selection in models of human…

Decision trees are widely used for classification and regression tasks in a variety of application fields due to their interpretability and good accuracy. During the past decade, growing attention has been devoted to globally optimized…

Ram\'irez and Rider (2009) established that the hard edge of the spectrum of the $\beta$-Laguerre ensemble converges, in the high-dimensional limit, to the bottom of the spectrum of the stochastic Bessel operator. Using stochastic analysis…

We generalize the technique of smoothed analysis to distributed algorithms in dynamic network models. Whereas standard smoothed analysis studies the impact of small random perturbations of input values on algorithm performance metrics,…

Mixture-of-Experts (MoE) models can scale parameter capacity by routing each token to a subset of experts through a learned gate function. While conditional routing reduces training costs, it shifts the burden on inference memory: expert…

Three-dimensional icosahedral random tilings are studied in the semi-entropic model. We introduce a global energy measure defined by the variance of the quasilattice points in orthogonal space. The specific heat shows a pronounced Schottky…

Generalizing to out-of-distribution (OOD) data or unseen domain, termed OOD generalization, still lacks appropriate theoretical guarantees. Canonical OOD bounds focus on different distance measurements between source and target domains but…

In this paper, we utilize results from convex analysis and monotone operator theory to derive additional properties of the softmax function that have not yet been covered in the existing literature. In particular, we show that the softmax…

Surface parametrization is a crucial part in various fields, having applications in computer graphic, medical imaging, scientific computing and computational engineering. The majority of surface parametrization approaches are performed on…

Multi-task learning has gained popularity due to the advantages it provides with respect to resource usage and performance. Nonetheless, the joint optimization of parameters with respect to multiple tasks remains an active research topic.…

Diffusion models have emerged as mainstream framework in visual generation. Building upon this success, the integration of Mixture of Experts (MoE) methods has shown promise in enhancing model scalability and performance. In this paper, we…

We develop a novel optimization method for NLPbackdoor inversion. We leverage a dynamically reducing temperature coefficient in the softmax function to provide changing loss landscapes to the optimizer such that the process gradually…

The present paper is devoted to estimating the speed of convergence towards consensus for a general class of discrete-time multi-agent systems. In the systems considered here, both the topology of the interconnection graph and the weight of…

This paper considers the problem of matrix-variate logistic regression. It derives the fundamental error threshold on estimating low-rank coefficient matrices in the logistic regression problem by obtaining a lower bound on the minimax…

The logistic loss function is often advocated in machine learning and statistics as a smooth and strictly convex surrogate for the 0-1 loss. In this paper we investigate the question of whether these smoothness and convexity properties make…

We study the problem of deterministic approximate counting of matchings and independent sets in graphs of bounded connective constant. More generally, we consider the problem of evaluating the partition functions of the monomer-dimer model…

We propose soft Hoeffding trees (SoHoT) as a new differentiable and transparent model for possibly infinite and changing data streams. Stream mining algorithms such as Hoeffding trees grow based on the incoming data stream, but they…

A fundamental question in random matrix theory is to quantify the optimal rate of convergence to universal laws. We take up this problem for the Laguerre $\beta$ ensemble, characterised by the Dyson parameter $\beta$, and the Laguerre…

For two-dimensional last-passage time models of weakly increasing paths, interesting scaling limits have been proved for points close the axis (the hard edge). For strictly increasing paths of Bernoulli($p$) marked sites, the relevant…