IMSE: Intrinsic Mixture of Spectral Experts Fine-tuning for Test-Time Adaptation

arXiv:2603.07926v3 Announce Type: replace-cross Abstract: Test-time adaptation (TTA) has been widely explored to prevent performance degradation when test data differ from the training distribution. However, fully leveraging the rich representations of large pretrained models with minimal parameter updates remains underexplored. In this paper, we propose Intrinsic Mixture of Spectral Experts (IMSE) that leverages the spectral experts inherently embedded in Vision Transformers. We decompose each