Related papers: FishBack: Pullback Fisher Geometry for Optimal Act…
Hierarchical neural networks are exponentially more efficient than their corresponding "shallow" counterpart with the same expressive power, but involve huge number of parameters and require tedious amounts of training. Our main idea is to…
Analyzing time series data is crucial to a wide spectrum of applications, including economics, online marketplaces, and human healthcare. In particular, time series classification plays an indispensable role in segmenting different phases…
Electromechanical switching devices such as relays, solenoid valves, and contactors offer several technical and economic advantages that make them widely used in industry. However, uncontrolled operations result in undesirable…
Iterative approximation methods using backpropagation enable the optimization of neural networks, but they remain computationally expensive, especially when used at scale. This paper presents an efficient alternative for optimizing neural…
The natural gradient method is widely used in statistical optimization, but its standard formulation assumes a Euclidean parameter space. This paper proposes an inversion-free stochastic natural gradient method for probability distributions…
Automatic assembly lines have increasingly replaced human labor in various tasks; however, the automation of Flexible Flat Cable (FFC) insertion remains unrealized due to its high requirement for effective feedback and dynamic operation,…
Geometric properties of Transformer weights, particularly the unembedding matrix, have been widely useful in language model interpretability research. Yet, their utility for estimating downstream performance remains unclear. In this work,…
Backpropagation is widely used to train artificial neural networks, but its relationship to synaptic plasticity in the brain is unknown. Some biological models of backpropagation rely on feedback projections that are symmetric with…
This paper investigates the conformal isometry hypothesis as a potential explanation for the hexagonal periodic patterns in grid cell response maps. We posit that grid cell activities form a high-dimensional vector in neural space, encoding…
Non-Hermitian systems have attracted considerable interest over the last few decades due to their unique spectral and dynamical properties not encountered in Hermitian counterparts. An intensely debated question is whether non-Hermitian…
This paper investigates the problem of straight-line path following for magnetic helical microswimmers. The control objective is to make the helical microswimmer to converge to a straight line without violating the step-out frequency…
This paper considers the problem of localizing a set of nodes in a wireless sensor network when both their positions and the parameters of the communication model are unknown. We assume that a single agent moves through the environment,…
Ultrasonic imaging methods often assume linear direct models, while in reality, many nonlinear phenomena are present, e.g. multiple reflections. A family of imaging methods called Full Waveform Inversion (FWI), which has been developed in…
Training large-scale neural networks requires solving nonconvex optimization where the choice of optimizer fundamentally determines both convergence behavior and computational efficiency. While adaptive methods like Adam have long dominated…
We show that when a dynamic-weight AMM rebalances by creating arbitrage opportunities, the per-step log loss is the KL divergence between successive weight vectors. The Fisher-Rao metric is therefore the natural Riemannian metric on the…
The Forward-Forward algorithm is an alternative learning method which consists of two forward passes rather than a forward and backward pass employed by backpropagation. Forward-Forward networks employ layer local loss functions which are…
Self-propelling organisms locomote via generation of patterns of self-deformation. Despite the diversity of body plans, internal actuation schemes and environments in limbless vertebrates and invertebrates, such organisms often use similar…
This work proposes a quaternion-based sliding variable that describes exponentially convergent error dynamics for any forward complete desired attitude trajectory. The proposed sliding variable directly operates on the non-Euclidean space…
The recently proposed Activation Relaxation (AR) algorithm provides a simple and robust approach for approximating the backpropagation of error algorithm using only local learning rules. Unlike competing schemes, it converges to the exact…
Markov Random Field models are powerful tools for the study of complex systems. However, little is known about how the interactions between the elements of such systems are encoded, especially from an information-theoretic perspective. In…