Related papers: High dimensional theory of two-phase optimizers

Conventional works generally employ a two-phase model in which a generator selects the most important pieces, followed by a predictor that makes predictions based on the selected pieces. However, such a two-phase model may incur the…

The SPS-LASSO has recently been introduced as a solution to the problem of regularization parameter selection in the complex-valued LASSO problem. Still, the dependence on the grid size and the polynomial time of performing convex…

We investigate the network stability problem when two users are scheduled simultaneously. The key idea is to simultaneously transmit to more than one users experiencing different channel conditions by employing hierarchical modulation. For…

Traditionally, the output noise and power supply rejection of low-dropout regulators (LDOs) are optimized to minimize power supply fluctuations, reducing their impact on the low-frequency noise of target voltage-controlled oscillators…

Optimization is an integral part of modern deep learning. Recently, the concept of learned optimizers has emerged as a way to accelerate this optimization process by replacing traditional, hand-crafted algorithms with meta-learned…

The concept of learning to optimize involves utilizing a trainable optimization strategy rather than relying on manually defined full gradient estimations such as ADAM. We present a framework that jointly trains the full gradient estimator…

Query optimizer is a crucial module for database management systems. Existing optimizers exhibit two flawed paradigms: (1) cost-based optimizers use dynamic programming with cost models but face search space explosion and heuristic pruning…

Linear Attention (LA) offers a promising paradigm for scaling large language models (LLMs) to long sequences by avoiding the quadratic complexity of self-attention. Recent LA models such as Mamba2 and GDN interpret linear recurrences as…

This study addresses the challenge of noise in training datasets for Direct Preference Optimization (DPO), a method for aligning Large Language Models (LLMs) with human preferences. We categorize noise into pointwise noise, which includes…

Many information systems employ lossy compression as a crucial intermediate stage among other processing components. While the important distortion is defined by the system's input and output signals, the compression usually ignores the…

Distributed optimization is the standard way of speeding up machine learning training, and most of the research in the area focuses on distributed first-order, gradient-based methods. Yet, there are settings where some…

This paper introduces a dual-signal transformation LSTM network (DTLN) for real-time speech enhancement as part of the Deep Noise Suppression Challenge (DNS-Challenge). This approach combines a short-time Fourier transform (STFT) and a…

The Least Absolute Shrinkage and Selection Operator (LASSO) has gained attention in a wide class of continuous parametric estimation problems with promising results. It has been a subject of research for more than a decade. Due to the…

Vision-Language-Action (VLA) models are receiving increasing attention for their ability to enable robots to perform complex tasks by integrating visual context with linguistic commands. However, achieving efficient real-time performance…

Even though high-level synthesis (HLS) tools mitigate the challenges of programming domain-specific accelerators (DSAs) by raising the abstraction level, optimizing hardware directive parameters remains a significant hurdle. Existing…

We consider stochastic convex optimization problems where the objective is an expectation over smooth functions. For this setting we suggest a novel gradient estimate that combines two recent mechanism that are related to notion of…

This article presents a novel methodology for detecting multiple biomarkers in high-dimensional mediation models by utilizing a modified Least Absolute Shrinkage and Selection Operator (LASSO) alongside Pathway LASSO. This approach…

This paper explores the benefit of using some of the machine learning techniques and Big data optimization tools in approximating maximum likelihood (ML) detection of Large Scale MIMO systems. First, large scale MIMO detection problem is…

In this work, we study optimization methods that leverage the linear minimization oracle (LMO) over a norm-ball. We propose a new stochastic family of algorithms that uses the LMO to adapt to the geometry of the problem and, perhaps…

Can synchronization properties of a network of identical oscillators in the presence of noise be improved through appropriate rewiring of its connections? What are the optimal network architectures for a given total number of connections?…