Non-record: notes on the recurrence band (mixing parameters, MLP sizing, loop sizing) - #2137

Open

leon2k2k2k wants to merge 84 commits into

openai:mainfrom

leon2k2k2k:writeup/recurrence-band-notes

Commits on May 1, 2026

Non-record writeup: notes on the recurrence band

leon2k2k2k
and
claude
committed
cccbe67

Commits on May 2, 2026

chore: consolidate unsaved research notes + run from worktrees before cleanup
leon2k2k2k
committed
0376b4e
fix: correct author handle for PR #2050 — @someone114514 → @AidenGeunGeun
leon2k2k2k
committed
8943d54

Commits on May 4, 2026

writeup: add blog post outline (5-part structure)

leon2k2k2k
and
claude
committed
3e3f5d3
writeup: fix technique descriptions after source cross-check

leon2k2k2k
and
claude
committed
717546c
writeup: fix parallel residuals layer (7→8), add three-expert n-gram detail

leon2k2k2k
and
claude
committed
dc74ccb
writeup: respond to two outline comments

leon2k2k2k
and
claude
committed
3575bd7
writeup: rewrite Part 0 — competition setup, scoring, C1-C4

leon2k2k2k
and
claude
committed
c5e4d7c
writeup: draft Part 1 baseline section

leon2k2k2k
and
claude
committed
99b75d4
writeup: add tokenizer and model architecture sections to Part 1

leon2k2k2k
and
claude
committed
76f979e
writeup: add training, quantization, and TTT sections to Part 1

leon2k2k2k
and
claude
committed
cec9e59

Commits on May 5, 2026

writeup: add closing paragraph to Part 1 final model section

leon2k2k2k
and
claude
committed
62b2ac3
writeup: add Part 2 disqualifications — n-gram tilt and PPM-D sections

leon2k2k2k
and
claude
committed
407830f
writeup: fact-check corrections across Part 1 and Part 2

leon2k2k2k
and
claude
committed
ac24777
writeup: fix internal inconsistency in global SGD doc count

leon2k2k2k
and
claude
committed
94d1d09
writeup: add CaseOps leak section and closing Looking Back paragraph

leon2k2k2k
and
claude
committed
1fe9390
writeup: tighten closing paragraph to single sentence

leon2k2k2k
and
claude
committed
b43a4ea
writeup: fix "picture-book" → "picture-perfect" finish

leon2k2k2k
and
claude
committed
ef98897
writeup: rename "The Final Model" section to "The Evolution"

leon2k2k2k
and
claude
committed
8ebcc08
writeup: promote "Drama on the Last Day" to top-level section, fold in closing

leon2k2k2k
and
claude
committed
6f6ba33
writeup: remove all em-dashes throughout draft

leon2k2k2k
and
claude
committed
1bf836c
writeup: rename 060A Research Model to Near-SOTA Model in table

leon2k2k2k
and
claude
committed
3633339
writeup: add "Let's break it down" transition in baseline section

leon2k2k2k
and
claude
committed
bf0b915
writeup: tighten baseline quantization paragraph

leon2k2k2k
and
claude
committed
25aa472
writeup: reformat model architecture section as bullet points

leon2k2k2k
and
claude
committed
b7ca759
writeup: reformat training, quantization, TTT sections as bullet points

leon2k2k2k
and
claude
committed
0258c80
writeup: add v2 outline and draft skeleton with hook
leon2k2k2k
committed
d6d1b9d
writeup: fix title and remove section 0 header
leon2k2k2k
committed
a0fcfd2
writeup: fix byte count and scoring description in hook
leon2k2k2k
committed
bb1815a
writeup: trim hook sentence
leon2k2k2k
committed
1d54375
writeup: rewrite hook second paragraph
leon2k2k2k
committed
e0693ce
writeup: add model comparison tables to top of v2
leon2k2k2k
committed
7192dfa
writeup: add closing sentence to hook paragraph
leon2k2k2k
committed
e687784
Update draft_v2.md
leon2k2k2k
authored
a965a06
writeup: draft section 1 - the competition
leon2k2k2k
committed
214b44f
writeup: expand section 1 with probability distribution explanation and analogy
leon2k2k2k
committed
c82e267
Update draft_v2.md
leon2k2k2k
authored
dd9fe34
writeup: expand section 1 with per-token cost and uniform baseline comparison
leon2k2k2k
committed
f42ab0f
writeup: draft section 2 - model evolution
leon2k2k2k
committed
f19fdd6
writeup: clarify block structure and add block chain diagram
leon2k2k2k
committed
7532ef6
writeup: add attention/MLP role description
leon2k2k2k
committed
be8d300
writeup: add U-Net skip connection code block to baseline
leon2k2k2k
committed
517d6de
writeup: restore baseline-to-evolution transition from v1
leon2k2k2k
committed
7834905
writeup: add PR number to depth recurrence, add parallel residuals section
leon2k2k2k
committed
7e328c9
writeup: remove redundant layers/parameters table
leon2k2k2k
committed
597bb87
Update draft_v2.md
leon2k2k2k
authored
5b1382d
writeup: rewrite training closing as choreographed dance
leon2k2k2k
committed
fe493f7
writeup: add quantization intro paragraph from v1
leon2k2k2k
committed
29f3879
writeup: restore v1 baseline prose, keep v2 U-Net code block
leon2k2k2k
committed
ec7dae2
writeup: tighten TTT opening sentence
leon2k2k2k
committed
fb5aad5
writeup: add TTT two-step transition sentence
leon2k2k2k
committed
d9eb241
writeup: remove incorrect TTT row from other changes table
leon2k2k2k
committed
14b8455
writeup: add intro sentence to Other Changes table
leon2k2k2k
committed
6a7f20e
writeup: remove redundant transition sentence before section 3
leon2k2k2k
committed
754f8c5
writeup: add transition line into section 3
leon2k2k2k
committed
4dcc044
writeup: remove transition line before section 3
leon2k2k2k
committed
12969bc
writeup: soften other changes intro
leon2k2k2k
committed
59a88c0
writeup: draft section 3 hook
leon2k2k2k
committed
012fbe7
writeup: add closing line to section 2
leon2k2k2k
committed
d445053
writeup: add ablations mention to section 2 closing line
leon2k2k2k
committed
55a90b0
writeup: draft section 3 - too good to be true
leon2k2k2k
committed
2f177bf
writeup: connect hook sentences with while
leon2k2k2k
committed
4673875
writeup: restructure n-gram section, move C1-C4 footnote, improve exam analogy
leon2k2k2k
committed
835ecda
Refine explanation of token prediction and PR #1514
leon2k2k2k
authored
237f935
writeup: rewrite PPM-D section with flashier hook and Russian novel analogy
leon2k2k2k
committed
a1219ba
writeup: restructure PPM-D section, tighten C2 explanation, defer PR #1905 to footnote
leon2k2k2k
committed
9bb020e
writeup: add expected entropy footnote to lesson section
leon2k2k2k
committed
a37cf00
writeup: implement user's PPM-D edits from GitHub
leon2k2k2k
committed
8d2029d
Update draft_v2.md
leon2k2k2k
authored
c649d91
writeup: draft section 4 - drama on the last day
leon2k2k2k
committed
c78613b
writeup: add hook line to section 4, clarify PR #2014, add gap values
leon2k2k2k
committed
8fc21c5
writeup: rewrite data leak discovery with cleaner explanation and exam analogy
leon2k2k2k
committed
55971bd
Update draft_v2.md
leon2k2k2k
authored
5834765
writeup: rephrase leaky PRs paragraph, change fifteen to many
leon2k2k2k
committed
0cbea32
writeup: simplify the ironic detail in section 4
leon2k2k2k
committed
b8d4051
writeup: rewrite section 4 closing with proper drama arc
leon2k2k2k
committed
3e7e24e
writeup: fix timing of leaky PRs - shortly after not final hours
leon2k2k2k
committed
46ffdc3
Update draft_v2.md
leon2k2k2k
authored
5f7e97d
writeup: add closing paragraph
leon2k2k2k
committed
754522d
writeup: remove all em-dashes from draft_v2
leon2k2k2k
committed
e00c3ab
writeup: fix PR #1344 loop attribution and GPTQ scope description

leon2k2k2k
and
claude
committed
833c99a
writeup: fix 8 incorrect PR attributions after leaderboard cross-check

leon2k2k2k
and
claude
committed
e128626
writeup: revert 4 PR attributions to true first appearance (pre-leaderboard)

leon2k2k2k
and
claude
committed
15dbf5e
writeup: fix GPTQ (#374→#535) and SmearGate (#162→#65) after direct PR inspection

leon2k2k2k
and
claude
committed
76b45f3