Related papers: On convergence and optimality of maximum-likelihoo…

A block sparse memory improved proportionate affine projection sign algorithm (BS-MIP-APSA) is proposed for block sparse system identification under impulsive noise. The new BS-MIP-APSA not only inherits the performance improvement for…

In the past decades, model averaging (MA) has attracted much attention as it has emerged as an alternative tool to the model selection (MS) statistical approach. Hansen [Econometrica 75 (2007) 1175--1189] introduced a Mallows model…

This paper analyzes the correlation matrix between the a priori error and measurement noise vectors for affine projection algorithms (APA). This correlation stems from the dependence between the filter tap estimates and the noise samples,…

Magnetic Resonance Imaging (MRI) is a powerful technique employed for non-invasive in vivo visualization of internal structures. Sparsity is often deployed to accelerate the signal acquisition or overcome the presence of motion artifacts,…

The proximal point algorithm (PPA) has been well studied in the literature. In particular, its linear convergence rate has been studied by Rockafellar in 1976 under certain condition. We consider a generalized PPA in the generic setting of…

The maximum likelihood (ML) and maximum a posteriori (MAP) estimation techniques are widely used to address the direction-of-arrival (DOA) estimation problems, an important topic in sensor array processing. Conventionally the ML estimators…

Proximal Policy Optimization (PPO) is a widely used reinforcement learning algorithm that heavily relies on accurate advantage estimates for stable and efficient training. However, raw advantage signals can exhibit significant variance,…

Fine-tuning and inference with large Language Models (LM) are generally known to be expensive. Parameter-efficient fine-tuning over pretrained LMs reduces training memory by updating a small number of LM parameters but does not improve…

This paper aims to predict optimal solutions for combinatorial optimization problems (COPs) via machine learning (ML). To find high-quality solutions efficiently, existing work uses a ML prediction of the optimal solution to guide heuristic…

We propose a framework for online meta-optimization of parameters that govern optimization, called Amortized Proximal Optimization (APO). We first interpret various existing neural network optimizers as approximate stochastic proximal point…

In addressing the computational and memory demands of fine-tuning Large Language Models(LLMs), we propose LoRA-SP(Streamlined Partial Parameter Adaptation), a novel approach utilizing randomized half-selective parameter freezing within the…

In this paper, we propose an inexact Augmented Lagrangian Method (ALM) for the optimization of convex and nonsmooth objective functions subject to linear equality constraints and box constraints where errors are due to fixed-point data. To…

We analyse the convergence of an approximate, fully inexact, ADMM algorithm under additive, deterministic and probabilistic error models. We consider the generalized ADMM scheme that is derived from generalized Lagrangian penalty with…

We consider partially-specified optimization problems where the goal is to actively, but efficiently, acquire missing information about the problem in order to solve it. An algorithm designer wishes to solve a linear program (LP), $\max…

Aligning large language models (LLMs) to diverse human preferences is fundamentally challenging since criteria can often conflict with each other. Inference-time alignment methods have recently gained popularity as they allow LLMs to be…

Approximate linear programming (ALP) and its variants have been widely applied to Markov Decision Processes (MDPs) with a large number of states. A serious limitation of ALP is that it has an intractable number of constraints, as a result…

A primary challenge facing modern scientific research is the limited availability of gold-standard data which can be costly, labor-intensive, or invasive to obtain. With the rapid development of machine learning (ML), scientists can now…

The accelerated proximal point algorithm (APPA), also known as "Catalyst", is a well-established reduction from convex optimization to approximate proximal point computation (i.e., regularized minimization). This reduction is conceptually…

We introduce a recursive algorithm of conveniently general form for estimating the coefficient of a moving average model of order one and obtain convergence results for both correct and misspecified MA(1) models. The algorithm encompasses…

Despite its popularity, several recent works question the effectiveness of MAML when test tasks are different from training tasks, thus suggesting various task-conditioned methodology to improve the initialization. Instead of searching for…