Grokking as Dimensional Phase Transition in Neural Networks

1. 1Analyze gradient avalanche dynamics across multiple model scales to identify the phase transition
2. 2Apply finite-size scaling to understand the dimensional phase transition
3. 3Investigate the relationship between effective dimensionality and generalization onset
4. 4Develop new training methods that take into account the dimensional phase transition

ML researchers and AI engineers benefit from understanding grokking as a dimensional phase transition to improve neural network training and generalization, and to develop more efficient learning algorithms

💡 Grokking is a dimensional phase transition where effective dimensionality crosses from sub-diffusive to diffusive at generalization onset