Related papers: Parameter-Efficient Fine-Tuning of Machine-Learnin…
With the ever-increasing number of pretrained models, machine learning practitioners are continuously faced with which pretrained model to use, and how to finetune it for a new dataset. In this paper, we propose a methodology that jointly…
We introduce a machine learning prediction workflow to study the impact of defects on the Raman response of 2D materials. By combining the use of machine-learned interatomic potentials, the Raman-active $\Gamma$-weighted density of states…
We propose fine-tuning large language models for generation of stable materials. While unorthodox, fine-tuning large language models on text-encoded atomistic data is simple to implement yet reliable, with around 90% of sampled structures…
Vision-Language Models (VLMs) have shown strong performance in tasks like visual question answering and multimodal text generation, but their effectiveness in scientific domains such as materials science remains limited. While some machine…
This paper introduces an active learning approach to the fitting of machine learning interatomic potentials. Our approach is based on the D-optimality criterion for selecting atomic configurations on which the potential is fitted. It is…
The appearance of generative models has opened vast chemical spaces in the design of functional materials. Although machine learning interatomic potentials (MLIPs) have substantially accelerated phonon calculations, high-fidelity prediction…
While traditional trial-and-error methods for designing amorphous alloys are costly and inefficient, machine learning approaches based solely on composition lack critical atomic structural information. Machine learning interatomic…
In this paper, we move towards combining large parametric models with non-parametric prototypical networks. We propose prototypical fine-tuning, a novel prototypical framework for fine-tuning pretrained language models (LM), which…
Multilingual neural machine translation models support fine-tuning hundreds of languages simultaneously. However, fine-tuning on full parameters solely is inefficient potentially leading to negative interactions among languages. In this…
Parameter-efficient fine-tuning aims to achieve performance comparable to fine-tuning, using fewer trainable parameters. Several strategies (e.g., Adapters, prefix tuning, BitFit, and LoRA) have been proposed. However, their designs are…
The accuracy of classical physical property predictions using molecular dynamics simulations is determined by the quality of the interatomic potentials. Here we introduce a training approach for empirical interatomic potentials (EIPs) which…
Human Activity Recognition is a foundational task in pervasive computing. While recent advances in self-supervised learning and transformer-based architectures have significantly improved HAR performance, adapting large pretrained models to…
Universal machine-learned interatomic potentials (U-MLIPs) have demonstrated effectiveness across diverse atomistic systems but often require fine-tuning for task-specific accuracy. We investigate the fine-tuning of two MACE-based…
Overcoming the challenge of limited data availability within materials science is crucial for the broad-based applicability of machine learning within materials science. One pathway to overcome this limited data availability is to use the…
We present an equivariant neural network for predicting vibrational and phonon modes of molecules and periodic crystals, respectively. These predictions are made by evaluating the second derivative Hessian matrices of the learned energy…
Here, we report the development of a detailed "Materials Informatics" framework for the design of acoustic coatings for underwater sound attenuation through integrating Machine Learning (ML) and statistical optimization algorithms with a…
In this paper we test two strategies to improving the accuracy of machine-learning potentials, namely adding more fitting parameters thus making use of large volumes of available quantum-mechanical data, and adding a charge-equilibration…
A recent family of techniques, dubbed lightweight fine-tuning methods, facilitates parameter-efficient transfer learning by updating only a small set of additional parameters while keeping the parameters of the pretrained language model…
The learning of the physical world relies on sensing and data post-processing. When the signals are weak, multidimensional and correlated, the performance of learning is often bottlenecked by the quality of sensors, calling for integrating…
Parameter fine tuning is a transfer learning approach whereby learned parameters from pre-trained source network are transferred to the target network followed by fine-tuning. Prior research has shown that this approach is capable of…