Related papers: Weight Norm Control
Decentralised learning enables the training of deep learning algorithms without centralising data sets, resulting in benefits such as improved data privacy, operational efficiency and the fostering of data ownership policies. However,…
Regularization is a critical technique for ensuring well-posedness in solving inverse problems with incomplete measurement data. Traditionally, the regularization term is designed based on prior knowledge of the unknown signal's…
Maximum hands-off control is a control that has the minimum L0 norm among all feasible controls. It is known that the maximum hands-off (or L0-optimal) control problem is equivalent to the L1-optimal control under the assumption of…
A risk measure that is consistent with the second-order stochastic dominance and additive for sums of independent random variables can be represented as a weighted entropic risk measure (WERM). The expected utility maximization problem with…
The Homotopy Analysis Method (HAM) is a widely used analytical approach for solving nonlinear problems, yet its theoretical foundation lacks rigorous justification, and its intrinsic correlation with perturbation theory remains ambiguous,…
Standard conformal anomaly detection provides marginal finite-sample guarantees under the assumption of exchangeability . However, real-world data often exhibit distribution shifts, necessitating a weighted conformal approach to adapt to…
We introduce a new normalization technique that exhibits the fast convergence properties of batch normalization using a transformation of layer weights instead of layer outputs. The proposed technique keeps the contribution of positive and…
Clinical trials with a hybrid control arm (a control arm constructed from a combination of randomized patients and real-world data on patients receiving usual care in standard clinical practice) have the potential to decrease the cost of…
This paper addresses two crucial problems of learning disentangled image representations, namely controlling the degree of disentanglement during image editing, and balancing the disentanglement strength and the reconstruction quality. To…
Neural networks are often over-parameterized and hence benefit from aggressive regularization. Conventional regularization methods, such as Dropout or weight decay, do not leverage the structures of the network's inputs and hidden states.…
A major challenge facing adiabatic quantum computing is that algorithm design and error correction can be difficult for adiabatic quantum computing. Recent work has considered addressing his challenge by using coherently controlled…
In decentralized optimization, several nodes connected by a network collaboratively minimize some objective function. For minimization of Lipschitz functions lower bounds are known along with optimal algorithms. We study a specific class of…
Model merging has recently gained attention as an economical and scalable approach to incorporate task-specific weights from various tasks into a unified multi-task model. For example, in Task Arithmetic (TA), adding the fine-tuned weights…
A variety of problems in distributed control involve a networked system of autonomous agents cooperating to carry out some complex task in a decentralized fashion, e.g., orienting a flock of drones, or aggregating data from a network of…
Recently theoretical guarantees have been obtained for matrix completion in the non-uniform sampling regime. In particular, if the sampling distribution aligns with the underlying matrix's leverage scores, then with high probability nuclear…
The unbounded knapsack problem with bounded weights is a variant of the well-studied variant of the traditional binary knapsack problem; key changes being the relaxation of the binary constraint and allowing the unit weights of each item to…
Inspired by Double Q-learning algorithm, the Double-DQN (DDQN) algorithm was originally proposed in order to address the overestimation issue in the original DQN algorithm. The DDQN has successfully shown both theoretically and empirically…
Questions remain on the robustness of data-driven learning methods when crossing the gap from simulation to reality. We utilize weight anchoring, a method known from continual learning, to cultivate and fixate desired behavior in Neural…
Binary constant-weight codes have been extensively studied, due to both their numerous applications and to their theoretical significance. In particular, constant-weight codes have been proposed for error correction in store and forward. In…
With the accumulation of resources in the era of big data and the rise of pre-trained models in deep learning, optimizing neural networks for various tasks often involves different strategies for fine-tuning pre-trained models versus…