feat: model converter scripts

[mikasenghaas]

Summary

• Add tools/convert_dcp_to_bf16.py: convert a DCP trainer checkpoint to HF sharded safetensors offline. Recovers the converter deferred out of feat: remove weight checkpointing #3316 and adapts it: required positional ckpt_dir (accepts <run>/checkpoints/step_{n} or .../step_{n}/trainer), output defaults to <ckpt_dir>/weights (or a second positional arg), model/tokenizer configs read from the run's resolved configs/trainer.json / sft.json (the single source of truth — the checkpoint must live in its run directory), single-process (uv run python) and multi-rank (torchrun) invocations. Local trust_remote_code models get their custom code files copied next to the weights. Full fine-tunes only — LoRA checkpoints are rejected up front. Export replaces training-only attention, fused-MoE, and DeepEP settings with portable model-construction values.
• Add tools/convert_bf16_to_fp8.py: quantize an HF safetensors dir to blockwise FP8 (DeepSeek/GLM format: e4m3 weights + weight_scale_inv per-128×128-block scales, quantization_config block vLLM loads natively). Streams shard-by-shard; norms, embeddings, lm_head, router gates, GatedDeltaNet low-rank projections, MTP projection, and vision towers stay bf16. Reuses quantize_to_fp8_blockwise from prime_rl.trainer.models.fp8. Inspired by slime/miles tools/convert_hf_to_fp8.py.
• Add tools/convert_dcp_to_fp8.py: DCP straight to fp8 — same gather as dcp_to_bf16, but each rank quantizes its slice on GPU and only fp8 shards are written to <ckpt_dir>/weights-FP8 (no intermediate bf16 export on disk).
• Add tools/convert_fp8_to_bf16.py: the inverse (dequantize blockwise-fp8 releases like GLM-5-FP8 to bf16 for training). Inspired by slime/miles tools/fp8_cast_bf16.py.
• Fold converter e2e checks into the existing reverse-text CI runs (no new CI matrix entries): both the SFT and RL integration tests convert their final DCP checkpoint to bf16 and fp8 and assert the exports reload in HF with a clean loading report; the RL test additionally asserts direct dcp_to_fp8 output is byte-identical to chained bf16_to_fp8 on a real trainer checkpoint. Shared helpers convert_checkpoint/check_hf_load in tests/utils.py. (A standalone per-arch byte-exactness matrix over the mini_moe archs — glm4_moe, laguna, minimax_m2, qwen3.5-MoE VLM; bf16 exports byte-identical to their random-init sources, both fp8 paths byte-identical — was validated during development and dropped from CI in favor of these hooks.) Also fixes the mini_moe VLM preset mrope_section for its shrunken head_dim.
• Read the run config from the configs/resolved/ layout via get_config_dir (adapted after merging main).
• Document the export flow in docs/training.md and skills/training/start-run/SKILL.md, and note the CLI convention in AGENTS.md: tools/ scripts use plain argparse, [project.scripts] entrypoints use pydantic-config.

Verification

All run on 2× RTX PRO 6000 Blackwell (SM120), vLLM 0.26.

dcp→hf, real RL run (qwen3 dense) — 20-step reverse-text RL run with --ckpt.interval 10 --weight-broadcast.type filesystem:

• step 10 converted single-process, step 20 with torchrun --nproc-per-node 2.
• Both exports are bit-identical (all 310 tensors) to the trainer's own filesystem-broadcast weights at the same step; tied lm_head.weight is dropped per HF convention and verified equal to embed_tokens.
• uv run inference --vllm.model .../step_10/weights serves the export; reverse-text eval (uv run eval, n=64 r=4) scores 0.754 vs 0.744 training reward at step 10. Step 20 export scores 0.801 vs ~0.80 at end of training.

dcp→hf, architecture matrix — per arch: tiny random model (scripts/mini_moe.py) → 3-step fake-data SFT → convert (exercises configs/sft.json auto-detection) → verify:

arch	strict HF load	prime reload + forward	notes
glm4_moe (MoE)	✓	✓	plus direct HF→DCP→HF round trip: 889/889 tensors bit-exact (expert stack/unstack)
laguna	✓ (trust_remote_code=True)	✓	HF→DCP→HF round trip 433/433 tensors bit-exact modulo the conversion chain's canonical per-expert layout (the same layout the live broadcast path emits; poolside's HF class stores fused experts.gate_up_proj and re-inits experts it doesn't recognize — see converting_laguna.py docstring)
minimax_m2 (MoE)	✓	✓	mini built via prime impl (MiniMax remote code incompatible with transformers v5)
qwen3_5_moe VLM	✓ (Qwen3_5MoeForConditionalGeneration)	key-set identity vs source	vision tower + linear attention round-trip

bf16→fp8 + fp8→bf16 — on the real step-20 Qwen3-0.6B export and all four minis: dequant error bounded (worst median relative error ~2.2%, e4m3-consistent), skip-list modules bit-identical, index/quantization_config consistent; fp8→bf16 round trip drops all *_scale_inv keys and quantization_config. Structure matches the official Qwen/Qwen3-0.6B-FP8 release (same scale shapes/names). Serving the fp8→bf16 round-tripped weights scores 0.798 (vs 0.801 bf16 baseline), confirming the stored values.

fp8 serving on SM120 (vLLM kernel bug, not the script): serving the fp8 checkpoint scores 0.31 with vLLM's default CutlassFp8BlockScaledMMKernel on this box, but 0.790 with the same checkpoint after forcing the Triton kernel (VLLM_DISABLED_KERNELS=...CutlassFp8BlockScaledMMKernel,MarlinFP8ScaledMMLinearKernel → TritonFp8BlockScaledMMKernel), and vLLM's online fp8 of the same weights scores 0.79. vLLM's CUTLASS blockwise-fp8 GEMM silently degrades on SM120 (RTX PRO 6000); upstream already tracks other SM120 blockwise-fp8 breakage (vllm-project/vllm#47990, #47436). Noted the workaround in skills/training/start-run/SKILL.md; H100/H200/B200 (SM90/SM100) use different kernels.

Not ported from slime/miles

Megatron converters (convert_torch_dist_to_hf*, convert_hf_to_torch_dist) — prime-rl is FSDP/DCP; int4 converters (Kimi-K2-specific); mxfp8/nvfp4 (no checkpoint-level use case yet); param_name_remap.py (covered by prime conversion chains); profiling tools (prime-rl has its own monitors).

🤖 Generated with Claude Code

---

Note

Medium Risk
Converters touch distributed gather, HF weight layout, and quantization paths that must match live training/broadcast behavior; mistakes could ship broken checkpoints, though integration tests and LoRA rejection reduce exposure.

Overview
Adds offline checkpoint export and FP8 conversion tooling under tools/convert_*.py, plus docs/skills updates and integration-test coverage.

DCP → Hugging Face: convert_dcp_to_bf16.py loads FSDP DCP trainer checkpoints using the run’s resolved configs/resolved/trainer.json or sft.json, gathers weights in parallel (single GPU or torchrun), writes sharded bf16 safetensors and tokenizer/config assets to <ckpt_dir>/weights, and rejects LoRA checkpoints.

FP8 round-trip: convert_bf16_to_fp8.py blockwise-quantizes HF safetensors (e4m3 + weight_scale_inv, vLLM-native quantization_config); convert_dcp_to_fp8.py quantizes each rank’s gathered slice straight to <ckpt_dir>/weights-FP8 without a bf16 export on disk; convert_fp8_to_bf16.py dequantizes fp8-only releases for training.

CI: Reverse-text RL and SFT integration tests convert the final checkpoint, assert clean AutoModelForCausalLM loads, and (RL) byte-identity between direct dcp_to_fp8 and chained bf16_to_fp8. Shared helpers convert_checkpoint / check_hf_load live in tests/utils.py. scripts/mini_moe.py fixes mrope_section for the shrunk VLM preset.

Docs: docs/training.md and training skills document the export flow; AGENTS.md notes tools/ scripts use plain argparse vs pydantic-config entrypoints. Run artifact paths use step_{n} consistently.

^{Reviewed by Cursor Bugbot for commit fe1241d. Bugbot is set up for automated code reviews on this repo. Configure here.}

[samsja]

we should write test for this, we should have one mini moe class that does

random-weight -> hf -> load -> dcp -> hf ->load ? maybe even with fp8

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 1627ce8. Configure here.}

[garrett361]

One high-level question I have is whether it's overly annoying to require gpus for model conversion. I think the more usual pattern is to do the conversion on CPU. Can be a future work item, though.

[mikasenghaas]

ya agreed, i think for really large models conversion on cpu might be tough? or can you stream it so you dont materialize the full thing in ram?

[garrett361]

or can you stream it so you dont materialize the full thing in ram?

Yeah, that's the platonic ideal to shoot for I think: CPU only and realizing only single weights (maybe layers) in memory at a time. I thought that this is what torchtitan does and that we could borrow from them, but they do actually realize the full model (but convert only on CPU). So achieving this would require a bit more custom code

Looks like NVIDIA's megatron bridge achieves this, though: https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/babf9533db1c492c0eb0ad9267f26e5028de6123/docs/bridge-tech-details.md

Not a goal for this PR, but a future one.