Transfer Learning in Physics-Informed Neural Networks: Full Fine-Tuning, Lightweight Fine-Tuning, and Low-Rank Adaptation

Yizheng Wang; Jinshuai Bai; Mohammad Sadegh Eshaghi; Cosmin Anitescu; Xiaoying Zhuang; Timon Rabczuk; Yinghua Liu

Transfer Learning in Physics-Informed Neural Networks: Full Fine-Tuning, Lightweight Fine-Tuning, and Low-Rank Adaptation

Machine Learning 2025-02-04 v1

Authors: Yizheng Wang , Jinshuai Bai , Mohammad Sadegh Eshaghi , Cosmin Anitescu , Xiaoying Zhuang , Timon Rabczuk , Yinghua Liu

View on arXiv ↗ PDF ↗

Abstract

AI for PDEs has garnered significant attention, particularly Physics-Informed Neural Networks (PINNs). However, PINNs are typically limited to solving specific problems, and any changes in problem conditions necessitate retraining. Therefore, we explore the generalization capability of transfer learning in the strong and energy form of PINNs across different boundary conditions, materials, and geometries. The transfer learning methods we employ include full finetuning, lightweight finetuning, and Low-Rank Adaptation (LoRA). The results demonstrate that full finetuning and LoRA can significantly improve convergence speed while providing a slight enhancement in accuracy.

Keywords

physics-informed neural networks transfer learning full fine-tuning lightweight fine-tuning low-rank adaptation partial differential equations convergence speed

Cite

@article{arxiv.2502.00782,
  title  = {Transfer Learning in Physics-Informed Neural Networks: Full Fine-Tuning, Lightweight Fine-Tuning, and Low-Rank Adaptation},
  author = {Yizheng Wang and Jinshuai Bai and Mohammad Sadegh Eshaghi and Cosmin Anitescu and Xiaoying Zhuang and Timon Rabczuk and Yinghua Liu},
  journal= {arXiv preprint arXiv:2502.00782},
  year   = {2025}
}

Transfer Learning in Physics-Informed Neural Networks: Full Fine-Tuning, Lightweight Fine-Tuning, and Low-Rank Adaptation

Abstract

Keywords

Cite

Related papers