Related papers: Transfer Learning for Transformer-Based Modeling o…

Transfer learning, also referred as knowledge transfer, aims at reusing knowledge from a source dataset to a similar target one. While many empirical studies illustrate the benefits of transfer learning, few theoretical results are…

Transformer-based pre-trained models with millions of parameters require large storage. Recent approaches tackle this shortcoming by training adapters, but these approaches still require a relatively large number of parameters. In this…

Understanding Transformer-based models has attracted significant attention, as they lie at the heart of recent technological advances across machine learning. While most interpretability methods rely on running models over inputs, recent…

In this work, we propose a distributed adaptive observer for a class of nonlinear networked systems inspired by biophysical neural network models. Neural systems learn by adjusting intrinsic and synaptic weights in a distributed fashion,…

Real-world machine learning applications often involve deploying neural networks to domains that are not seen in the training time. Hence, we need to understand the extrapolation of nonlinear models -- under what conditions on the…

Motivated by the recent success of end-to-end training of communications in the wireless domain, we strive to adapt the end-to-end-learning idea from the wireless case (i.e., linear) to coherent optical fiber links (i.e., nonlinear).…

We propose a novel type of nonlinear solver acceleration for systems of nonlinear partial differential equations (PDEs) that is based on online/adaptive learning. It is applied in the context of multiphase flow in porous media. The proposed…

The paper proposes the use of structured neural networks for reinforcement learning based nonlinear adaptive control. The focus is on partially observable systems, with separate neural networks for the state and feedforward observer and the…

Previous research on late-reverberation modeling has mainly focused on exponentially decaying room impulse responses, whereas methods for accurately modeling non-exponential reverberation remain challenging. This paper extends the…

Machine learning methods are being introduced at all stages of data reconstruction and analysis in various high-energy physics experiments. We present the development and application of convolutional neural networks with modified…

Transfer learning refers to the promising idea of initializing model fits based on pre-training on other data. We particularly consider regression modeling settings where parameter estimates from previous data can be used as anchoring…

Deep learning has shown substantial progress in image analysis. However, the computational demands of large, fully trained models remain a consideration. Transfer learning offers a strategy for adapting pre-trained models to new tasks.…

We investigate end-to-end optimized optical transmission systems based on feedforward or bidirectional recurrent neural networks (BRNN) and deep learning. In particular, we report the first experimental demonstration of a BRNN auto-encoder,…

A new way to design parameter estimators with enhanced performance is proposed in the paper. The procedure consists of two stages, first, the generation of new regression forms via the application of a dynamic operator to the original…

Linear and nonlinear distortions in optical communication signals are equalized using an integrated feed-forward Photonic Neural Network (PNN). The PNN is based on a linear stage made of an 8-tap Finite Impulse Response (FIR) filter,…

Existing works on building a soliton transmission system only encode information using the imaginary part of the eigenvalue, which fails to make full use of the signal degree-of-freedoms. Motivated by this observation, we make the first…

Domain adaptation deals with adapting classifiers trained on data from a source distribution, to work effectively on data from a target distribution. In this paper, we introduce the Nonlinear Embedding Transform (NET) for unsupervised…

We have developed an algorithm that constructs a model of a reconfigurable optical interferometer, independent of specific architectural constraints. The programming of unitary transformations on the interferometer's optical modes relies on…

Transfer learning (TL) is becoming a powerful tool in scientific applications of neural networks (NNs), such as weather/climate prediction and turbulence modeling. TL enables out-of-distribution generalization (e.g., extrapolation in…

The fundamental lemma from behavioral systems theory yields a data-driven non-parametric system representation that has shown great potential for the data-efficient control of unknown linear and weakly nonlinear systems, even in the…