Low-Rank Adaptation of Pre-Trained Large Vision Models for Improved Lung Nodule Malignancy Classification

Abstract

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Goal:</i> This paper investigates using Low-Rank Adaptation (LoRA) to adapt large vision models (LVMs) pretrained with self-supervised learning (SSL) for lung nodule malignancy classification. Inspired by LoRA's success in the field of Natural Language Processing, we hypothesized that such an adaptation technique can significantly improve classification performance, parameter efficiency, and training speed for the novel application of lung image cancer diagnostic. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Methods:</i> Utilizing two comprehensive lung nodule datasets, NLSTx and LIDC, which together encompass a diverse array of biopsy- and radiologist-confirmed lung CT scans, our rigorous experimental setup demonstrates that LoRA-adapted models markedly surpass traditional fine-tuning methods. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Results:</i> The best LoRA-adapted model achieved a 3% increase in ROC AUC over the state-of-the-art model, utilized 89.9% fewer parameters, and reduced training times by 36.5%. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Conclusions:</i> Integrating LoRA with out-of-domain pretrained LVMs offers a promising avenue for enhancing performance of lung nodule malignancy classification. The annotations for the NLSTx dataset are also released with this paper on GitHub at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/benVZ/NLSTx</uri>.