This article details our participation (L3iTC) in the FinLLM Challenge Task 2024, focusing on two key areas: Task 1, financial text classification, and Task 2, financial text summarization. To address these challenges, we fine-tuned several large language models (LLMs) to optimize performance for each task. Specifically, we used 4-bit quantization and LoRA to determine which layers of the LLMs should be trained at a lower precision. This approach not only accelerated the fine-tuning process on the training data provided by the organizers but also enabled us to run the models on low GPU memory. Our fine-tuned models achieved third place for the financial classification task with an F1-score of 0.7543 and secured sixth place in the financial summarization task on the official test datasets.
@article{arxiv.2408.03033,
title = {L3iTC at the FinLLM Challenge Task: Quantization for Financial Text Classification & Summarization},
author = {Elvys Linhares Pontes and Carlos-Emiliano González-Gallardo and Mohamed Benjannet and Caryn Qu and Antoine Doucet},
journal= {arXiv preprint arXiv:2408.03033},
year = {2024}
}
Comments
Joint Workshop of the 8th Financial Technology and Natural Language Processing (FinNLP) and the 1st Agent AI for Scenario Planning (AgentScen), 2024