Related papers: Gradient-Descent Based Optimization of Multi-Tone …
Various gradient compression schemes have been proposed to mitigate the communication cost in distributed training of large scale machine learning models. Sign-based methods, such as signSGD, have recently been gaining popularity because of…
Gradient-based hyperparameter optimization has earned a widespread popularity in the context of few-shot meta-learning, but remains broadly impractical for tasks with long horizons (many gradient steps), due to memory scaling and gradient…
Recent deep-learning approaches have shown that Frequency Transformation (FT) blocks can significantly improve spectrogram-based single-source separation models by capturing frequency patterns. The goal of this paper is to extend the FT…
Time series foundation models (TSFMs) demonstrate impressive zero-shot performance for time series forecasting. However, an important yet underexplored challenge is how to effectively finetune TSFMs on specific downstream tasks. While naive…
In this work, we present an optimization framework for sparse variable-length stop-feedback (VLSF) codes based on a saddlepoint approximation, which jointly optimizes the decoding configuration parameters. Thanks to the analytical…
Gradient-based bilevel optimisation is a powerful technique with applications in hyperparameter optimisation, task adaptation, algorithm discovery, meta-learning more broadly, and beyond. It often requires differentiating through the…
Fluid antenna is a new reconfigurable antenna technology that can dynamically adjust the positions or ports of radiating elements and therefore provides a new degree of freedom for wireless communications. However, the associated port…
In this paper, we propose a novel image calibration algorithm for a twofold time-interleaved DAC (TIDAC). The algorithm is based on simulated annealing, which is often used in the field of machine learning to solve derivative free…
This paper considers the problem of asynchronous stochastic nonconvex optimization with heavy-tailed gradient noise and arbitrarily heterogeneous computation times across workers. We propose an asynchronous normalized stochastic gradient…
Dielectric metasurfaces provide a unique platform for efficient harmonic generation and optical wavefront manipulation at the nanoscale. Tailoring phase and amplitude of a nonlinearly generated wave with a high emission efficiency using…
This work proposes a mixed learning-based and optimization-based approach to the weighted-sum-rates beamforming problem in a multiple-input multiple-output (MIMO) wireless network. The conventional methods, i.e., the fractional programming…
Invariance to microphone array configuration is a rare attribute in neural beamformers. Filter-and-sum (FS) methods in this class define the target signal with respect to a reference channel. However, this not only complicates formulation…
We developed a novel reduced-order multi-scale method for solving large time-domain wavefield simulation problems. Our algorithm consists of two main stages. During the first "off-line" stage the fine-grid operator (of the graph Laplacian…
This paper studies the coordinated beamforming (CoBF) design for the multiple-input single-output interference channel, provided that only channel distribution information is known to the transmitters. The problem under consideration is a…
Stochastic multi-level compositional optimization problems cover many new machine learning paradigms, e.g., multi-step model-agnostic meta-learning, which require efficient optimization algorithms for large-scale data. This paper studies…
Stochastic gradient-based descent (SGD), have long been central to training large language models (LLMs). However, their effectiveness is increasingly being questioned, particularly in large-scale applications where empirical evidence…
Applying a sparse constraint on the beam pattern has been suggested to suppress the sidelobe of the minimum variance distortionless response (MVDR) beamformer recently. To further improve the performance, we add a mixed norm constraint on…
The success of current Large-Language Models (LLMs) hinges on extensive training data that is collected and stored centrally, called Centralized Learning (CL). However, such a collection manner poses a privacy threat, and one potential…
Decentralized optimization algorithms have received much attention due to the recent advances in network information processing. However, conventional decentralized algorithms based on projected gradient descent are incapable of handling…
Aims : We describe MS-MFS, a multi-scale multi-frequency deconvolution algorithm for wide-band synthesis-imaging, and present imaging results that illustrate the capabilities of the algorithm and the conditions under which it is feasible…