Exploring connections of spectral analysis and transfer learning in medical imaging.

TitleExploring connections of spectral analysis and transfer learning in medical imaging.
Publication TypeJournal Article
Year of Publication2025
AuthorsLu Y, Juodelyte D, Victor JD, Cheplygina V
JournalProc SPIE Int Soc Opt Eng
Volume13406
Date Published2025 Feb
ISSN0277-786X
Abstract

In this paper, we use spectral analysis to investigate transfer learning and study model sensitivity to frequency shortcuts in medical imaging. By analyzing the power spectrum density of both pre-trained and fine-tuned model gradients, as well as artificially generated frequency shortcuts, we observe notable differences in learning priorities between models pre-trained on natural vs medical images, which generally persist during fine-tuning. We find that when a model's learning priority aligns with the power spectrum density of an artifact, it results in overfitting to that artifact. Based on these observations, we show that source data editing can alter the model's resistance to shortcut learning. Code available at: https://github.com/YCL92/Shortcut-PSD.

DOI10.1117/12.3047670
Alternate JournalProc SPIE Int Soc Opt Eng
PubMed ID40575597
PubMed Central IDPMC12201968
Grant ListR01 EY007977 / EY / NEI NIH HHS / United States