Related papers: Powers and Limitations of Synchronous Self-Assembl…
Large language models (LLMs) benefit from test-time scaling but are often hampered by high inference latency. Speculative decoding is a natural way to accelerate the scaling process; however, scaling along both the parallel and sequential…
Through numerical simulations we analyze the synchronization time and the Lyapunov dimension of a coupled map lattice consisting of a chain of chaotic logistic maps exhibiting power law interactions. From the observed behaviors we find a…
The Transformer architecture, underpinned by the self-attention mechanism, has become the de facto standard for sequence modeling tasks. However, its core computational primitive scales quadratically with sequence length (O(N^2)), creating…
In this paper, we consider a network of agents that jointly aim to minimise the sum of local functions subject to coupling constraints involving all local variables. To solve this problem, we propose a novel solution based on a primal-dual…
Adapter based fine-tuning has been studied for improving the performance of SAM on downstream tasks. However, there is still a significant performance gap between fine-tuned SAMs and domain-specific models. To reduce the gap, we propose…
Sharpness-Aware Minimization (SAM) is an optimization method that improves generalization performance of machine learning models. Despite its superior generalization, SAM has not been actively used in real-world applications due to its…
Generalized synchronization is plausibly the most complex form of synchronization. Previous studies have revealed the existence of weak or strong forms of generalized synchronization depending on the multi- or mono-valued nature of the…
The stability analysis of synchronization in time-varying higher-order networked structures (simplicial complexes) is one of the challenging problem due to the presence of time-varying group interactions. In this context, most of the…
Synchronization resulting in unified collective behavior of the individual elements of a system that are weakly coupled to each other has long fascinated scientists. Examples range from the periodic oscillation of coupled pendulum clocks to…
The alternating direction method of multipliers (ADMM) has been popular for solving many signal processing problems, convex or nonconvex. In this paper, we study an asynchronous implementation of the ADMM for solving a nonconvex nonsmooth…
Anomalies often occur in real-world information networks/graphs, such as malevolent users, malicious comments, banned users, and fake news in social graphs. The latest graph anomaly detection methods use a novel mechanism called truncated…
Recently, a detailed epistemic reasoning framework for multi-agent systems with byzantine faulty asynchronous agents and possibly unreliable communication was introduced. We have developed a modular extension framework implemented on top of…
Discrete ordinates $S_N$ transport solvers on unstructured meshes pose a challenge to scale due to complex data dependencies, memory access patterns and a high-dimensional domain. In this paper, we review the performance bottlenecks within…
Many state-of-the-art LLMs are trained to think before giving their answer. Reasoning can greatly improve language model capabilities, but it also makes them less interactive: given a new input, a model must stop thinking before it can…
Correctly capturing the symmetry transformations of data can lead to efficient models with strong generalization capabilities, though methods incorporating symmetries often require prior knowledge. While recent advancements have been made…
In this Letter, we propose a growing network model that can generate scale-free networks with a tunable community strength. The community strength, $C$, is directly measured by the ratio of the number of external edges to internal ones; a…
We study the relative effects of data augmentations, pretraining algorithms, and model architectures in Self-Supervised Learning (SSL). While the recent literature in this space leaves the impression that the pretraining algorithm is of…
A novel approach is being developed to introduce a parallel asynchronous implementation of non-intrusive global-local coupling. This study examines scenarios involving numerous patches, including those covering the entire structure. By…
Recent works have shown that self-supervised learning can achieve remarkable robustness when integrated with adversarial training (AT). However, the robustness gap between supervised AT (sup-AT) and self-supervised AT (self-AT) remains…
We first pose the Unsupervised Progressive Learning (UPL) problem: an online representation learning problem in which the learner observes a non-stationary and unlabeled data stream, learning a growing number of features that persist over…