Same Function, Different Dynamics: Understanding Muon’s Response to Reparameterization

摘要

We study how function-preserving changes in parameter representation affect optimizer updates and subsequent predictions. Focusing on Muon in SwiGLU language models, we develop Libra, a state-resolved framework that connects reciprocal channel scaling to local predictive response. Under consistent optimizer-state transport, update homogeneity explains the uniform-scale response. After scale compensation, exact-polar derivatives separate column-space motion from singular-frame rotation, while finite Newton--Schulz updates also change spectral amplitudes. We decompose the difference between their derivatives and connect the original-rule update defect to full-state sensitivity and Jacobian/Fisher readouts. Experiments on 106.57M-parameter language models find that off-diagonal response contributes 65.3--82.1% of the five-step Newton--Schulz (NS5)--polar derivative-difference energy across checkpoint groups in the tested random directions. Observable projection lowers transported one-step median relative Kullback--Leibler (KL) error against calibrated scalar defect proxies in the larger-dose exploratory cohort. In addition, retained-state measurements reveal memory interference. These measurements characterize local, short-horizon predictive responses under the specified optimizer-history conventions.