Skip to content

Add analytic and numerical MuonClip polar Jacobian baseline - #84

Draft
charlesmartin14 wants to merge 7 commits into
mainfrom
agent/mnist-mlp3-muonclip-polar-jacobian-baseline
Draft

Add analytic and numerical MuonClip polar Jacobian baseline#84
charlesmartin14 wants to merge 7 commits into
mainfrom
agent/mnist-mlp3-muonclip-polar-jacobian-baseline

Conversation

@charlesmartin14

@charlesmartin14 charlesmartin14 commented Aug 18, 2026

Copy link
Copy Markdown
Member

Adds an MNIST MLP3 MuonClip baseline for the local polar-projection Jacobian at single matrix checkpoints.

For each layer matrix W, the notebook computes J = D Pi(W) and compares the Gram spectrum G = J*J in two ways: (1) analytically from the closed-form polar Frechet derivative and its exact eigenvalues 4/(sigma_i+sigma_j)^2 plus rectangular 1/sigma_i^2 modes, and (2) numerically from isotropic random perturbations, centered finite differences of Pi(W), and the empirical response Gram X^T X.

Both raw positive spectra are passed directly to WeightWatcher's WW_powerlaw.pl_fit. The notebook compares WeightWatcher alpha, KS D, xmin, and tail size, and includes a probe-count convergence helper for a single checkpoint. No spectrum normalization, trace normalization, log-binning, KDE, entropy, participation-ratio, or hand-written power-law fit is used.

The reusable operator code lives in baseline/rg_baselines/polar_jacobian.py; the experiment driver and mathematical derivation live in baseline/notebooks/MNIST_MLP3_MuonClip_Polar_Jacobian_Baseline.ipynb.

@charlesmartin14 charlesmartin14 changed the title Add MuonClip polar quotient Gram baseline notebook Add analytic and numerical MuonClip polar Jacobian baseline Aug 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant