VISTA: Validation-Informed Trajectory Adaptation via Self-Distillation

arXiv:2604.12044v1 Announce Type: cross Abstract: Deep learning models may converge to suboptimal solutions despite strong validation accuracy, masking an optimization failure we term Trajectory Deviation. This is because as training proceeds, models can abandon high generalization states for specific data sub-populations, thus discarding previously learned latent features without triggering classical overfitting signals. To address this problem we introduce VISTA, an online self-distillation fr