Dimension-Corrected Hitting Times for Heavy-Tailed Spectral Emergence in Neural Optimizer Dynamics

RESUMO

Heavy-tailed empirical spectral densities of neural-network weight matrices are widely used as diagnostics of implicit self-regularization, but the step complexity of heavy-tail emergence remains poorly understood. We formulate spectral heavy-tail formation as a right-censored hitting-time problem: a run that does not reach a heavy-tail diagnostic within the observation horizon is treated as censored rather than discarded. In controlled full-batch teacher--student dynamics, we find that the first-step spike--bulk gap alone does not explain onset time. Instead, finite-onset regression supports a dimension-corrected spectral-gap law, (\tau_{\mathrm{HT}}\approx C\Delta_1^{-\gamma}d^\rho), with (R^2=0.683), (\gamma=0.626), and (\rho=0.772) across 330 completed runs. Right-censored lognormal accelerated-failure-time models further favor the dimension-corrected model over a gap-only model, improving AIC from 706.62 to 628.70. Theoretically, we prove that exact early loss dynamics in linear networks do not determine factor spectral tails, that Adam recurrences alone do not imply spectral redistribution, and that projected singular-basis spreading implies contraction of a spectral-tail potential and hence a dimension-corrected hitting-time bound. Empirically, projected-kernel profiles support the sufficient spreading mechanism, Adam and AdamW agree under tested grids, GD and signGD do not reach onset in the same regimes, and real pretrained Qwen2.5-0.5B and Pythia-70M transformer weights show non-Gaussian spectral-tail structure relative to matched Gaussian nulls. The result is a reproducible spectral hitting-time law with rigorous conditional theory, not a claim that Adam necessarily generates heavy tails from first principles.