Related papers: Measuring the Instability of Fine-Tuning

The dynamical behavior of switched affine systems is known to be more intricate than that of the well-studied switched linear systems, essentially due to the existence of distinct equilibrium points for each subsystem. First, under…

Given the significance of physical measures in understanding the complexity of dynamical systems as well as the noisy nature of real-world systems, investigating the stability of physical measures under noise perturbations is undoubtedly a…

We study a class of singularly perturbed impulsive linear switched systems exhibiting switching between slow and fast dynamics. To analyze their behavior, we construct auxiliary switched systems evolving in a single time scale. We prove…

The solution of a fine tuning problem is one of the principal motivations of Supersymmetry. However experimental constraints indicate that many Supersymmetric models are also fine tuned (although to a much lesser extent). We review the…

Invariance-based randomization tests -- such as permutation tests, rotation tests, or sign changes -- are an important and widely used class of statistical methods. They allow drawing inferences under weak assumptions on the data…

In this article, two types of methods from different perspectives based on spectral normalization are described for ensuring the stability of the system controlled by a neural network. The first one is that the L2 gain of the feedback…

Despite advances in large language models (LLMs) on reasoning and instruction-following tasks, it is unclear whether they can reliably produce outputs aligned with a variety of user goals, a concept called steerability. Two gaps in current…

As large language models achieve impressive scores on traditional benchmarks, an increasing number of researchers are becoming concerned about benchmark data leakage during pre-training, commonly known as the data contamination problem. To…

Reliable evaluation is fundamental to the progress of Large Language Models (LLMs), yet the evaluation process during pre-training is plagued by significant instability that obscures true learning dynamics. In this work, we systematically…

Recent pretrained language models extend from millions to billions of parameters. Thus the need to fine-tune an extremely large pretrained model with a limited training corpus arises in various downstream tasks. In this paper, we propose a…

This work studies robustness to system disturbance and measurement noise of some popular general practical stabilization techniques, namely, Dini aiming, optimization-based stabilization and inf-convolution stabilization. Common to all…

We present a theoretical analysis of stochastic optimization methods in terms of their sensitivity with respect to the step size. We identify a key quantity that, for each method, describes how the performance degrades as the step size…

Pre-trained language models (PLMs) may fail in giving reliable estimates of their predictive uncertainty. We take a close look into this problem, aiming to answer two questions: (1) Do PLMs learn to become calibrated in the training…

Large Language Models (LLMs) are increasingly deployed for structured data generation, yet output consistency remains critical for production applications. We introduce a comprehensive framework for evaluating and improving consistency in…

Nonlinearity in many systems is heavily dependent on component variation and environmental factors such as temperature. This is often overcome by keeping signals close enough to the device's operating point that it appears approximately…

Structured pruning has been extensively studied on monolingual pre-trained language models and is yet to be fully evaluated on their multilingual counterparts. This work investigates three aspects of structured pruning on multilingual…

Despite having high accuracy, neural nets have been shown to be susceptible to adversarial examples, where a small perturbation to an input can cause it to become mislabeled. We propose metrics for measuring the robustness of a neural net…

Soft dynamic time warping (SDTW) is a differentiable loss function that allows for training neural networks from weakly aligned data. Typically, SDTW is used to iteratively compute and refine soft alignments that compensate for temporal…

Results reported in large-scale multilingual evaluations are often fragmented and confounded by factors such as target languages, differences in experimental setups, and model choices. We propose a framework that disentangles these…

Robustness checks are routine in empirical work, but there is no standard statistical procedure to formally measure what one can learn from them. I propose a "robustness radius" measure to quantify the amount by which the robustness checks…