Related papers: Local Softmax and Global Weights in Non-Boolean Ev…
Causal inference in modern largescale systems faces growing challenges, including highdimensional covariates, multi-valued treatments, massive observational (OBS) data, and limited randomized controlled trial (RCT) samples due to cost…
Local projections (LPs) are widely used in empirical macroeconomics to estimate impulse responses to policy interventions. Yet, in many ways, they are black boxes. It is often unclear what mechanism or historical episodes drive a particular…
In meta-analysis, the random-effects models are standard tools to address between-study heterogeneity in evidence synthesis analyses. For the random-effects distribution models, the normal distribution model has been adopted in most…
Softmax is widely used in neural networks for multiclass classification, gate structure and attention mechanisms. The statistical assumption that the input is normal distributed supports the gradient stability of Softmax. However, when used…
Link weight is crucial in weighted complex networks. It provides additional dimension for describing and adjusting the properties of networks. The topological role of weight is studied by the effects of random redistribution of link weights…
Probabilistic smoothing is a standard tool for global optimization, but existing methods rely on Gaussian kernels and specific transforms, often resulting in strong hyperparameter sensitivity and limited robustness. We propose a general…
Search-optimization problems are plentiful in scientific and engineering domains. Artificial intelligence has long contributed to the development of search algorithms and declarative programming languages geared toward solving and modeling…
A special normalization is proposed for strongly intensive quantities used in the study of event-by-event fluctuations in high energy collisions. It ensures that these measures are dimensionless and yields a common scale required for a…
In many studies, it is common to use binary (i.e., unweighted) edges to examine networks of entities that are either adjacent or not adjacent. Researchers have generalized such binary networks to incorporate edge weights, which allow one to…
The plethora of scalar fields participating in the formulation of a softly broken supersymmetric theory can threat the stability of the standard vacuum. The generic situation is twofold. Directions in scalar field space may exist along…
A framework (a straight-line embedding of a graph into a normed space allowing edges to cross) is globally rigid if any other framework with the same edge lengths with respect to the chosen norm is an isometric copy. We investigate global…
We study local asymptotic normality of M-estimates of convex minimization in an infinite dimensional parameter space. The objective function of M-estimates is not necessary differentiable and is possibly subject to convex constraints. In…
Self-normalizing discriminative models approximate the normalized probability of a class without having to compute the partition function. In the context of language modeling, this property is particularly appealing as it may significantly…
In this paper, we study large-scale convex optimization algorithms based on the Newton method applied to regularized generalized self-concordant losses, which include logistic regression and softmax regression. We first prove that our new…
Parameter estimation via unbinned maximum likelihood fits is central for many analyses performed in high energy physics. Unbinned maximum likelihood fits using event weights, for example to statistically subtract background contributions…
The plasticity of amorphous solids undergoing shear is characterized by quasi-localized rearrangements of particles. While many models of plasticity exist, the precise relationship between plastic dynamics and the structure of a particle's…
Nonlocal (spatial-dispersion) effects in multilayered metamaterials composed of periodic stacks of alternating, deeply subwavelength dielectric layers are known to be negligibly weak. Counterintuitively, under certain critical conditions,…
The softmax function combined with a cross-entropy loss is a principled approach to modeling probability distributions that has become ubiquitous in deep learning. The softmax function is defined by a lone hyperparameter, the temperature,…
In Transformer models, non-GEMM (non-General Matrix Multiplication) operations -- especially Softmax and Layer Normalization (LayerNorm) -- often dominate hardware cost due to their nonlinear nature. To address this, previous approximation…
Robust model fitting is a fundamental problem in computer vision: used to pre-process raw data in the presence of outliers. Maximisation of Consensus (MaxCon) is one of the most popular robust criteria and widely used. Recently (Tennakoon…