Categorical Perception in Large Language Model Hidden States: Structural Warping at Digit-Count Boundaries

1. 1Analyze hidden-state representations of LLMs using representational similarity analysis
2. 2Identify geometric warping at category boundaries, such as digit-count boundaries
3. 3Apply CP-additive models to quantify the effect of categorical perception on LLM representations
4. 4Use findings to inform model design and fine-tuning decisions, such as adjusting model architecture or training data

ML researchers and AI engineers can benefit from understanding how LLMs process and represent categorical information, which can inform model design and fine-tuning decisions

💡 LLMs exhibit enhanced discriminability at category boundaries, similar to human perceptual psychology

🤖 LLMs exhibit categorical perception in hidden states when processing Arabic numerals! #LLMs #CategoricalPerception