Repository navigation
Add momentum and the momentum coefficient - #559
Conversation
Moved from the lds gradient descent chapter, reworded to mds notation (f, step length, iterates); lds will link here. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019xfc9ow6EykAkrucpc8Fyw
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019xfc9ow6EykAkrucpc8Fyw
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019xfc9ow6EykAkrucpc8Fyw
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
… example Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019xfc9ow6EykAkrucpc8Fyw
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019xfc9ow6EykAkrucpc8Fyw
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
…cters Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019xfc9ow6EykAkrucpc8Fyw
|
Claude finished review — View run I fetched PR #559 live: it is open, not a draft, not merged, and its head is Review of PR #559 at
|
2026-10-31 07:09 UTC |
Requested by Ezra · project thread
Part 4 of moving general gradient-descent content from lds to mds, after mds#558. This PR adds momentum (the heavy-ball method) and the momentum coefficient to the optimization gradient descent page, each with an example in R and Python.
The lds chapter's cost
C, parameters, and learning rate become mds'sf, iterates, and step length. The lds definition of momentum sets the learning rate outside the velocity, and this PR keeps that form. The lds figure of momentum paths on the narrow valley stays in lds, since it belongs to the lds exercise.After this merges and the site publishes, a follow-up lds PR will replace the lds copies with links. Adaptive methods and Adam, then the step-count schedules, are the next moves.
Not rendered locally (the container has no Quarto or R packages). I ran the R and Python logic on its own and got the same values: after 20 updates plain descent is at 0.1216, momentum at 0.3447, and momentum's lowest point is -0.6037 at update 9. The PR build is the render check.
Generated by Claude Code