[dem-extra1]

Requested by Ezra · project thread

Closes #263.

What changed

• New def-surprisal (surprisal, also called self-information and information content) using the existing \surprisalf macro. The definition states the unit (nats with the natural logarithm, bits with base 2). A worked coin example exm-surprisal-coin and a short remark rem-surprisal-rarity (rarer values are more surprising) follow. Sources: Goodfellow et al. printed p. 73 (the definition of self-information, eq. 3.48) and Bishop printed p. 49, checked against the PDFs in mlr. The printed page numbers differ from the PDF page index (Goodfellow's is offset by 15).
• def-entropy now defines entropy as the expectation of the surprisal, using \entropyf and \surprisalf.
• bishop2006prml and goodfellow2016deep are added to references.bib. Neither book has a DOI in the lab's other bibliographies, so both join the DOI check's exclude-keys as the other books without one already are.

Checks

quarto render expectation.qmd --to html succeeds locally with no unresolved cross-references, and the term-link check is clean locally. typos is not installed in this session, so spelling is left to CI.

Split out of #262, which now holds only the cross-entropy theorem. The entropy corollary follows in a PR stacked on both.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JeChvMkJZfAotVCTze7Tct

[dem-extra1]

🤖 Antigravity agent report

Cross-harness review (Argon: Gemini 4 Argon (Medium) via Antigravity agy 1.3.5, read-only plan mode) at e11e5dd

These come from the other model and I have not independently verified each one against the files, so treat them as leads. The review used the diff only (long diffs are truncated at 24000 characters); nothing was rendered.

Findings

1. Page-number contradiction between PR description and text (_subfiles/_def-surprisal.qmd):

   The surprisal is also called the **self-information** [@goodfellow2016deep, p. 73]
   The PR body claims "Sources: Goodfellow et al. p. 72 and Bishop p. 49", whereas _def-surprisal.qmd cites p. 73 (and in Goodfellow et al. 2016, self-information $I(x) = -\log P(x)$ is defined in Section 3.13 on p. 72).

2. Forward reference to logarithmic units (_subfiles/def-surprisal/_exm-surprisal-coin.qmd vs. _rem-surprisal-units.qmd):

   Using logarithms to base 2, the [surprisal](#def-surprisal) of heads is $\surprisalf{1} = -\log_2 \tfrac{1}{4} = 2$ bits,
_exm-surprisal-coin.qmd uses the unit "bits" on the slide before _rem-surprisal-units.qmd introduces and defines how the logarithm base determines the unit ("nats for the natural logarithm and bits for base 2"). Include _rem-surprisal-units.qmd before _exm-surprisal-coin.qmd.

3. Mismatched ID/title and duplicate claim in remark (_subfiles/def-surprisal/_rem-surprisal-units.qmd):

   ::: {#rem-surprisal-units} / #### Rare outcomes are more surprising
   The ID #rem-surprisal-units and PR description ("a remark on units") describe logarithm units, but the heading and first sentence are about monotonicity/asymptotics, and the heading duplicates the closing sentence of _exm-surprisal-coin.qmd ("The rarer outcome is the more surprising one.").

4. Undefined mapping between coin outcomes and $1, 0$ (_subfiles/def-surprisal/_exm-surprisal-coin.qmd):

   Let $X$ be the result of one flip of a coin with $\P(X=1) = \tfrac{1}{4}$ and $\P(X=0) = \tfrac{3}{4}$. Using logarithms to base 2, the [surprisal](#def-surprisal) of heads is $\surprisalf{1}
   $X$ is never stated to be the indicator of heads ($1 =$ heads, $0 =$ tails) before identifying $\surprisalf{1}$ as "heads" and $\surprisalf{0}$ as "tails".

5. Terminology slip: outcome vs. value in range (_subfiles/_def-surprisal.qmd):

   The **surprisal** of an [outcome](probability-basics.qmd#def-experiment-outcome) $x$ of a [discrete random variable](random-variables.qmd#def-discrete-rv) $X$
   Under #def-experiment-outcome, an outcome $\omega \in \Omega$ is an element of the sample space, whereas $x \in \rangef{X}$ is a value (or realization) of the random variable $X$ corresponding to the event ${X = x}$.

6. Missing space after = (_subfiles/_def-entropy.qmd):

   $$\entropyf{X} \eqdef \E{\surprisalf{X}} =-\sum_{x \in \rangef{X}} \P(X=x) \log \P(X=x)$$
   The space between = and -\sum that was present in the original line (= -\sum) was accidentally deleted.

Verdict

VERDICT: FINDINGS

Reviewed-Commit: e11e5dd

Posted by Claude Code (AI agent) --- not written by a human.

[dem-extra1]

🤖 Antigravity agent report

Cross-harness review (Argon: Gemini 4 Argon (Medium) via Antigravity agy 1.3.5, read-only plan mode) at b8dd5d7

These come from the other model and I have not independently verified each one against the files, so treat them as leads. The review used the diff only (long diffs are truncated at 24000 characters); nothing was rendered.

Findings

1. _subfiles/def-surprisal/_exm-surprisal-coin.qmd uses forward references to the unit ("measured in bits") and base-2 logarithm convention before _rem-surprisal-units.qmd introduces them; either move _rem-surprisal-units.qmd before _exm-surprisal-coin.qmd or introduce the unit/base in _def-surprisal.qmd.

   Using logarithms to base 2, so that [surprisal](#def-surprisal) is measured in bits, the surprisal of heads is $\surprisalf{1} = -\log_2 \tfrac{1}{4} = 2$ bits,

2. _subfiles/_def-surprisal.qmd defines $\surprisalf{x}$ on realization values $x$ and then overwrites the same symbol $\surprisalf{X}$ for the random variable without noting the abuse of notation (or distinguishing the PMF $p_X(x)$ from $\P(X)$, where $\P(X)$ is non-standard notation for $p_X(X)$).

   Applying $\surprisalf{\cdot}$ to the [random variable](random-variables.qmd#def-random-variable) $X$ itself gives the random variable $\surprisalf{X} = -\log \P(X).$

3. _subfiles/def-surprisal/_exm-surprisal-coin.qmd shifts terminology from "possible value $x$" (used in def-surprisal) to "outcomes" (which in probability-basics.qmd#def-experiment-outcome are elements $\omega$ of the sample space $\Omega$, whereas $1$ and $0$ are values of $X$, and "heads"/"tails" are outcomes):

   the surprisal of heads is $\surprisalf{1} = -\log_2 \tfrac{1}{4} = 2$ bits, and the surprisal of tails is $\surprisalf{0} = -\log_2 \tfrac{3}{4} = \log_2 \tfrac{4}{3} \approx 0.415$ bits.

4. _subfiles/def-surprisal/_rem-surprisal-units.qmd likewise conflates an experiment outcome $\omega \in \Omega$ with a value $x$ of random variable $X$ when writing $\P(X=x) \to 0$ for an "outcome":

   An [outcome](probability-basics.qmd#def-experiment-outcome) with [probability](probability-basics.qmd#def-probability) $1$ has [surprisal](#def-surprisal) $0$, and the surprisal grows without bound as $\P(X=x) \to 0$.

5. references.bib omits address = {Cambridge, MA} and isbn = {978-0-262-03561-3} on goodfellow2016deep, which is inconsistent with bishop2006prml immediately above it.

   @book{goodfellow2016deep, author = {Goodfellow, Ian and Bengio, Yoshua and Courville, Aaron}, title = {Deep Learning}, publisher = {MIT Press}, year = {2016},

Verdict

VERDICT: FINDINGS

Reviewed-Commit: b8dd5d7

Posted by Claude Code (AI agent) --- not written by a human.

[dem-extra1]

Response to the cross-harness review at b8dd5d7, addressed in 6d90ab0:

1. Unit before the example: the definition now says the surprisal is measured in nats with the natural logarithm and in bits with base 2, so the example no longer uses a unit that comes later.
2. Notation of \surprisalf{X}: the definition now says the random variable \surprisalf{X} equals \surprisalf{x} when X = x, and no longer writes -\log \P(X).
3. and 4. Outcome versus value: the example and the remark now say "value" (a value of X, a possible value x with \P(X=x) = 1) and no longer link "outcome".
4. goodfellow2016deep: address and isbn (978-0-262-03561-3) added to match the Bishop entry.

---

Posted by Claude Code (AI agent) --- not written by a human.
Generated by Claude Code

---

Generated by Claude Code

[dem-extra1]

🤖 Antigravity agent report

Cross-harness review (Argon: Gemini 4 Argon (Medium) via Antigravity agy 1.3.5, read-only plan mode) at 6d90ab0

These come from the other model and I have not independently verified each one against the files, so treat them as leads. The review used the diff only (long diffs are truncated at 24000 characters); nothing was rendered.

Findings

1. Duplicated logarithm-base/unit explanation between #def-surprisal and #rem-surprisal-units:

   With the natural logarithm the surprisal is measured in nats, and with base 2 in bits. (_subfiles/_def-surprisal.qmd)
   The base of the logarithm determines the unit: nats for the natural logarithm and bits for base 2. (_subfiles/def-surprisal/_rem-surprisal-units.qmd)
   Both blocks state the same rule on back-to-back slides; keep it in one place (preferably #rem-surprisal-units, or in #def-surprisal and drop the second half of the remark).

2. Undefined shorthand $\P(X)$ in _subfiles/_def-entropy.qmd:

   is the [expectation](#def-expectation) of its [surprisal](#def-surprisal) $\surprisalf{X} = -\log \P(X)$
#def-surprisal defines $\surprisalf{x} \eqdef -\log \P(X=x)$ and then defines the random variable $\surprisalf{X}$ without introducing the notation $\P(X)$, so writing $\surprisalf{X} = -\log \P(X)$ introduces an undefined shorthand that #def-surprisal avoided.

3. Outcome vs. value wording in _subfiles/def-surprisal/_exm-surprisal-coin.qmd:

   the surprisal of heads is $\surprisalf{1} = -\log_2 \tfrac{1}{4} = 2$ bits, and the surprisal of tails is $\surprisalf{0} = -\log_2 \tfrac{3}{4}
#def-surprisal defines the surprisal of a value $x$ of $X$ (and the example heading says "values"), whereas the prose attributes $\surprisalf{1}$ and $\surprisalf{0}$ to the outcomes "heads" and "tails" rather than to the values $1$ (heads) and $0$ (tails).

4. Relative fragment links in subfiles omit expectation.qmd:

   so that [surprisal](#def-surprisal) is measured in bits (_exm-surprisal-coin.qmd) and has [surprisal](#def-surprisal) $0$ (_rem-surprisal-units.qmd)
   Other cross-references in these subfiles include the target filename (random-variables.qmd#def-discrete-rv, probability-basics.qmd#def-probability); using expectation.qmd#def-surprisal (and expectation.qmd#def-expectation in _def-entropy.qmd) keeps the subfiles portable if included outside expectation.qmd.

Verdict

VERDICT: FINDINGS

Reviewed-Commit: 6d90ab0

Posted by Claude Code (AI agent) --- not written by a human.

[dem-extra1]

Response to the cross-harness review at 6d90ab0, addressed in e0bb126:

1. Duplicated unit sentence: the unit now appears only in the definition. The remark is retitled "Rarer values are more surprising" and its id and file are renamed to rem-surprisal-rarity to match (nothing else referenced the old id).
2. -\log \P(X) in _def-entropy.qmd: removed. The entropy definition now says "its surprisal \surprisalf{X}".
3. Heads and tails: the example now speaks of "the value 1 (heads)" and "the value 0 (tails)".
4. Fragment links without expectation.qmd (not changed): same-page anchors such as #def-surprisal are the form the other subfiles included only from expectation.qmd use, and the Claude reviews accepted it. Prefixing the page would make them cross-page links to the page they are on.

---

Posted by Claude Code (AI agent) --- not written by a human.
Generated by Claude Code

---

Generated by Claude Code

[dem-extra1]

🤖 Antigravity agent report

Cross-harness review (Argon: Gemini 4 Argon (Medium) via Antigravity agy 1.3.5, read-only plan mode) at e0bb126

These come from the other model and I have not independently verified each one against the files, so treat them as leads. The review used the diff only (long diffs are truncated at 24000 characters); nothing was rendered.

Findings

1. PR body contradicts actual diff regarding rem-surprisal-rarity:

   - New def-surprisal (surprisal, also called self-information and information content) using the existing \surprisalf macro, a worked coin example exm-surprisal-coin, and a remark on the logarithm base and unit.
   The logarithm base and unit (nats/bits) are in def-surprisal itself, whereas the added remark is actually rem-surprisal-rarity ("Rarer values are more surprising").

2. Subfile fragility with bare #def-* anchors:

   is the [expectation](#def-expectation) of its [surprisal](#def-surprisal) \surprisalf{X}
   And in _exm-surprisal-coin.qmd / _rem-surprisal-rarity.qmd: [surprisal](#def-surprisal). While currently included in expectation.qmd, _subfiles/ are designed for reuse across pages/decks; using expectation.qmd#def-surprisal (matching random-variables.qmd#def-pmf on the line above) prevents broken links if a subfile is included elsewhere.

3. Notation overload / dependence on $X$ in $\surprisalf{x}$:

   $$\surprisalf{x} \eqdef -\log \P(X=x).$$
   Writing $\surprisalf{x}$ without subscripting $X$ (e.g., $\surprisalf[X]{x}$) leaves the random variable $X$ implicit on the LHS, and makes $\surprisalf{X}$ vs. $\surprisalf{x}$ an abuse of notation (especially when $X$ is numerical, as in $\surprisalf{1}$ and $\surprisalf{0}$ in exm-surprisal-coin).

4. Missing comma after introductory prepositional phrase (style nit):

   With the natural logarithm the surprisal is measured in nats, and with base 2 in bits.
   Consider adding a comma after "logarithm" ("With the natural logarithm, the surprisal...").

Verdict

VERDICT: FINDINGS

Reviewed-Commit: e0bb126

Posted by Claude Code (AI agent) --- not written by a human.

[dem-extra1]

Response to the review of e0bb126, fixed in 397818b:

1. PR body contradicted the diff: fixed. It now describes the definition's unit sentence and the rem-surprisal-rarity remark.
2. Bare #def-* anchors in subfiles: not changed. Of the subfiles here, 60 use bare same-page anchors and 12 use a page-prefixed form. Subfiles are included into one parent page, so bare anchors resolve there, and the term-link check passes with them.
3. \surprisalf{x} notation overload (random variable vs value): not changed. The macro has no subscript form, and the definition states that evaluating at the random variable X equals the value's surprisal when X = x, which is the textbook usage.
4. Missing comma after "With the natural logarithm": fixed.

---

Generated by Claude Code

[dem-extra1]

🤖 Antigravity agent report

Cross-harness review (Argon: Gemini 4 Argon (Medium) via Antigravity agy 1.3.5, read-only plan mode) at 397818b

These come from the other model and I have not independently verified each one against the files, so treat them as leads. The review used the diff only (long diffs are truncated at 24000 characters); nothing was rendered.

Findings

1. Subfile portability broken by bare anchors (#def-surprisal, #def-expectation):

is the [expectation](#def-expectation) of its [surprisal](#def-surprisal) $\surprisalf{X}$ (in _subfiles/_def-entropy.qmd; also [surprisal](#def-surprisal) in _exm-surprisal-coin.qmd and _rem-surprisal-rarity.qmd)
Files in _subfiles/ are reusable across chapters/decks; bare #def-... anchors break whenever a subfile is included outside expectation.qmd (unlike random-variables.qmd#def-pmf and probability-basics.qmd#def-probability on the surrounding lines). Rebutting this by assuming subfiles will only ever be included in expectation.qmd defeats the purpose of _subfiles/; qualify with expectation.qmd#def-surprisal and expectation.qmd#def-expectation.

2. Notation overloading of \surprisalf{x} vs. \surprisalf{X} leaves distribution Dependence implicit:

$$\surprisalf{x} \eqdef -\log \P(X=x).$$ and Evaluating $\surprisalf{\cdot}$ at the [random variable](random-variables.qmd#def-random-variable) $X$ itself gives the random variable $\surprisalf{X}$, which equals $\surprisalf{x}$ when $X = x$.
Writing $\surprisalf{x}$ without subscripting $X$ (or $p_X$) hides that the function depends on the distribution of $X$, and makes $\surprisalf{X}$ an abuse of notation (evaluating $x \mapsto -\log \P(X=x)$ at $X$, rather than a property of the distribution of $X$ like $\entropyf{X}$). If keeping unsubscripted \surprisalf, explicitly note that the dependence on $X$'s distribution is suppressed in the notation when $X$ is clear from context.

3. Inconsistent bolding of defined unit terms (nats, bits):

With the natural logarithm, the surprisal is measured in nats, and with base 2 in bits.
When introducing named units of information for the first time in a definition block, bold nats and bits (or shannons) to match standard term-introduction style.

4. Strict inequality $\P(X=x) > 0$ in def-surprisal conflicts with $\E{\surprisalf{X}}$ when $\P(X=x)=0$:

The **surprisal** of a possible value $x$ of a [discrete random variable](random-variables.qmd#def-discrete-rv) $X$ with $\P(X=x) > 0$ is $$\surprisalf{x} \eqdef -\log \P(X=x).$$
If $\rangef{X}$ includes values with $\P(X=x)=0$ (which _def-entropy.qmd explicitly accommodates via "A term with $\P(X=x) = 0$ is taken to be $0$"), then $\surprisalf{x}$ is undefined there unless either $\rangef{X}$ is restricted to $\P(X=x)>0$ or $\surprisalf{x} \eqdef \infty$ for $\P(X=x)=0$.

Verdict

VERDICT: FINDINGS

Reviewed-Commit: 397818b

Posted by Claude Code (AI agent) --- not written by a human.

[dem-extra1]

Response to the review of 397818b, fixed in ee7dbe5 where noted:

1. Bare #def-* anchors: not changed, as in my earlier reply. This is not an assumption that the subfiles stay on one page. The repository's convention is bare anchors in subfiles (60 subfiles use them, 12 use a page prefix), and check-term-links passes with them. A page-prefixed link would also break the other way, because the bare form is what resolves when a subfile is reused on a page that includes the definition too. If the maintainer wants prefixed anchors, that is a repo-wide change for its own PR.
2. Implicit distribution: fixed. The definition now says "The notation leaves the distribution of $X$ implicit."
3. Bold units: fixed, nats and bits are bold.
4. Strict inequality: not changed. The definition covers exactly the values with $\P(X=x) > 0$, and $\surprisalf{X}$ is then defined with probability 1, so its expectation is well defined. def-entropy states the convention for the terms with $\P(X=x) = 0$ separately, and those terms never occur as values of $X$.

Posted by Claude Code (AI agent) --- not written by a human.

---

Generated by Claude Code

[dem-extra1]

Correction to my previous reply: finding 3 (bold nats and bits) is not applied after all. check-term-links treats a bolded word as a defined term and then failed on the first use of "bits" in exm-surprisal-coin (run 40308714999). Reverted in 410e682. Nats and bits are units named in passing, not terms this page defines, so they stay plain. Findings 2 (distribution note) stays fixed.

Posted by Claude Code (AI agent) --- not written by a human.

---

Generated by Claude Code

[dem-extra1]

🤖 Antigravity agent report

Cross-harness review (Argon: Gemini 4 Argon (Medium) via Antigravity agy 1.3.5, read-only plan mode) at 5e3efc0

These come from the other model and I have not independently verified each one against the files, so treat them as leads. The review used the diff only (long diffs are truncated at 24000 characters); nothing was rendered.

Findings

1. Unbolded unit terms (nats, bits) contradict the review rebuttal (lost commit ee7dbe5):

   With the natural logarithm, the surprisal is measured in nats, and with base 2 in bits.
   Rebuttal item 3 states that **nats** and **bits** were bolded in ee7dbe5, but at HEAD (5e3efc0) both defined unit terms remain unbolded (suggesting ee7dbe5 was not pushed or was overwritten).

2. Wording of implicit distribution sentence contradicts the review rebuttal:

   The notation leaves the [probability mass function](random-variables.qmd#def-pmf) of $X$ implicit.
   Rebuttal item 2 claims ee7dbe5 updated this sentence to "The notation leaves the distribution of $X$ implicit.", but HEAD (5e3efc0) still reads probability mass function. (Rebuttal items 1 and 4 are sound.)

Verdict

VERDICT: FINDINGS

Reviewed-Commit: 5e3efc0

Posted by Claude Code (AI agent) --- not written by a human.

[dem-extra1]

Response to the review of 5e3efc0: both findings compare the head with my first reply, which the later commits overtook. The head is correct as it stands.

1. Unbolded units: intentional. ee7dbe5 bolded nats and bits, and check-term-links then failed on the first use of "bits" in exm-surprisal-coin, because the check treats bold as a defined term. 410e682 reverted it, as my correction comment says. Nats and bits are units named in passing, not terms this page defines.
2. Wording of the implicit-distribution sentence: also intentional. ee7dbe5 wrote "the distribution of $X$", and check-term-links then flagged "distribution" as a defined term. 5e3efc0 names the linked probability mass function instead, which is the same claim and passes the check (clean locally and on the head).

Nothing was lost: ee7dbe5, 410e682 and 5e3efc0 are all in the branch history.

Posted by Claude Code (AI agent) --- not written by a human.

---

Generated by Claude Code

[dem-extra1]

🤖 Antigravity agent report

Cross-harness review (Argon: Gemini 4 Argon (Medium) via Antigravity agy 1.3.5, read-only plan mode) at 5e3efc0

Checked for correctness, Quarto syntax and crossrefs, consistency with the PR description, lost or duplicated content, and style, from the diff alone. Not checked: the rendered page.

Verdict

VERDICT: CLEAN

Reviewed-Commit: 5e3efc0

Posted by Claude Code (AI agent) --- not written by a human.

[github-actions]

Claude finished review — View run

---

Commit 995df9f has no new findings. I read the sources and the diff and did not render the site.

The only unreviewed commit in 5e3efc03..995df9fb is the merge of main into the branch. Its one hand-resolved file is references.bib, where both sides added entries at the end of the file. The resolution is correct:

• Entries kept: bishop2006prml and goodfellow2016deep from this branch, plus lange2024applied from main.
• Markup: no conflict markers remain, each entry closes its own }, and the three entries are separated by blank lines.
• Duplicates: no duplicate keys, and lange2024applied carries a DOI.
• Other merged changes: the cross-entropy theorem, exercises, solutions, expectation.qmd and probability-basics.qmd edits, and the latex-macros bump come from main's own commits (the cross-entropy work and Point to Lange's Applied Probability in Further reading #269). They are not this PR's own changes, so I did not re-review them.

The Goodfellow p. 73 and Bishop p. 49 page numbers in _subfiles/_def-surprisal.qmd are still unconfirmed against the books. That is non-blocking and carried over from earlier rounds.

My last review, round 10 at 5e3efc0, gave Ready for merge (comment), and that verdict still stands.

Verdict

Ready for merge

Structured Review Data (JSON)

{
  "schema_version": "1.1",
  "reviewer": "claude",
  "commit_sha": "995df9fb4e5be2b26a966659a63e3510597a1055",
  "verdict": "CLEAN",
  "findings": [],
  "detailed_assessment": "The merge commit 995df9f hand-resolved an add/add conflict in references.bib; the result keeps bishop2006prml, goodfellow2016deep and lange2024applied with no conflict markers, balanced braces and no duplicate keys.",
  "holistic_assessment": "The change since the last round is only a merge of main with a correct bibliography resolution, so scope and regression risk are unchanged, leaving only the unverified book page citations as a non-blocking carry-over."
}

Reviewed commit: 995df9f

[github-actions]

💰 Cost: $0.2042 (review) — run

[github-actions]

PR Preview Action v1.8.1
Preview removed because the pull request was closed.

_{2026-10-26 18:34 UTC}