Related papers: Optimal Protocols for Nonlocality Distillation
Deploying large and complex deep neural networks on resource-constrained edge devices poses significant challenges due to their computational demands and the complexities of non-convex optimization. Traditional compression methods such as…
This paper reexamines the research on out-of-distribution (OOD) robustness in the field of NLP. We find that the distribution shift settings in previous studies commonly lack adequate challenges, hindering the accurate evaluation of OOD…
We apply the inseparability criterion for $2 \times 2$ systems, local filtering and Bennett et al. purification protocol [Phys. Rev. Lett. {\bf 76}, 722 (1996)] to show how to distill {\it any} inseparable $2\times 2$ system. The extended…
Spatial self-attention layers, in the form of Non-Local blocks, introduce long-range dependencies in Convolutional Neural Networks by computing pairwise similarities among all possible positions. Such pairwise functions underpin the…
Cluster analysis, or clustering, plays a crucial role across numerous scientific and engineering domains. Despite the wealth of clustering methods proposed over the past decades, each method is typically designed for specific scenarios and…
We present an approach to handle Dirichlet type nonlocal boundary conditions for nonlocal diffusion models with a finite range of nonlocal interactions. Our approach utilizes a linear extrapolation of prescribed boundary data. A novelty is,…
A noiseless linear amplifier (NLA) performs the highest quality amplification allowable under the rules of quantum physics. Unfortunately, these same rules conspire against us via the no-cloning theorem, which constrains NLA operations to…
In this paper, we present BitNet Distillation (BitDistill), a lightweight pipeline that fine-tunes off-the-shelf full-precision LLMs (e.g., Qwen) into 1.58-bit precision (i.e., ternary weights {-1, 0, 1}) for specific downstream tasks,…
Compared to traditional electrodynamic loudspeakers, the parametric array loudspeaker (PAL) offers exceptional directivity for audio applications but suffers from significant nonlinear distortions due to its inherent intricate demodulation…
We study the oracle complexity of nonsmooth nonconvex optimization, with the algorithm assumed to have access only to local function information. It has been shown by Davis, Drusvyatskiy, and Jiang (2023) that for nonsmooth Lipschitz…
Adversarial Distillation aims to enhance student robustness by guiding the student with a robust teacher's soft labels within the min-max adversarial training framework, yet its success is notoriously inconsistent: a more robust teacher…
Recent natural language processing (NLP) techniques have accomplished high performance on benchmark datasets, primarily due to the significant improvement in the performance of deep learning. The advances in the research community have led…
The success of deep learning has brought forth a wave of interest in computer hardware design to better meet the high demands of neural network inference. In particular, analog computing hardware has been heavily motivated specifically for…
We seek for the optimal strategy to infer the width $a$ of an infinite potential wells by performing measurements on the particle(s) contained in the well. In particular, we address quantum estimation theory as the proper framework to…
Label-noise learning (LNL) aims to increase the model's generalization given training data with noisy labels. To facilitate practical LNL algorithms, researchers have proposed different label noise types, ranging from class-conditional to…
Distribution Matching Distillation (DMD) is a powerful acceleration paradigm, yet its stability is often compromised in Forbidden Zone, regions where the real teacher provides unreliable guidance while the fake teacher exerts insufficient…
We study the mathematical theory of second order systems with two species, arising in the dynamics of interacting particles subject to linear damping, to nonlocal forces and to external ones, and resulting into a nonlocal version of the…
We present a demo of DQLoRA, an Adapter-Guided Distillation framework for robust speech recognition under low-resource and noisy conditions. Our method employs a frozen Whisper model as the teacher to provide semantic supervision, and a…
Neural network-based medium access control (MAC) protocol models (NPMs) improve goodput through site-specific operations but are vulnerable to shifts from their training network environments, such as changes in the number of user equipments…
We construct an optimal state merging protocol by adapting a recently-discovered optimal entanglement distillation protcol [Renes and Boileau, Phys. Rev. A . 73, 032335 (2008)]. The proof of optimality relies only on directly establishing…