Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions _subfiles/_def-prediction-error.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ the observed [outcome](#def-outcome) $\Outvar$ of a [new observation](#def-new-o
not the [conditional mean](#def-cond-expectation-function) of that outcome.
The **prediction error** of $\predfn$ is the prediction minus that outcome:

$$\esterr{\predfnf{\Predvar}}{\Outvar} \eqdef \predfnf{\Predvar} - \Outvar$$
$$\prederrf{\predfn} \eqdef \predfnf{\Predvar} - \Outvar$$

:::

Expand All @@ -17,7 +17,7 @@ $$\esterr{\predfnf{\Predvar}}{\Outvar} \eqdef \predfnf{\Predvar} - \Outvar$$

In @exr-best-predictor,
consider the constant [prediction](#def-prediction) $\predfn_1\paren{\Predvar} = 3$.
Its [prediction error](#def-prediction-error) is $\esterr{\predfn_1\paren{\Predvar}}{\Outvar} = 3 - \Outvar$.
Its [prediction error](#def-prediction-error) is $\prederrf{\predfn_1} = 3 - \Outvar$.
This is $3$ when $\Outvar = 0$.
It is $-3$ when $\Outvar = 6$.

Expand Down
2 changes: 1 addition & 1 deletion _subfiles/_def-squared-error-risk.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ The **squared-error risk** of a prediction function $\predfnf{\Predvar}$,
also called its **expected prediction error** [@hastie2009elements, p. 18],
is the expected square of its [prediction error](#def-prediction-error):

$$\popriskf{\predfn} \eqdef \E{\sqf{\esterr{\predfnf{\Predvar}}{\Outvar}}}$$
$$\popriskf{\predfn} \eqdef \E{\sqf{\prederrf{\predfn}}}$$

It is the [risk](#def-risk) under [squared error loss](#def-squared-error-loss).

Expand Down
14 changes: 7 additions & 7 deletions _subfiles/_lem-cross-term-zero.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
Let $\truemeanf{\Predvar} \eqdef \E{\Outvar \mid \Predvar}$ be the [conditional expectation function](#def-cond-expectation-function) of $\Outvar$ given $\Predvar$.
For any [prediction function](#def-prediction) $\predfnf{\Predvar}$ with $\E{\predfnf{\Predvar}^2} < \infty$:

$$\E{\esterr{\truemeanf{\Predvar}}{\Outvar} \cd \paren{\predfnf{\Predvar} - \truemeanf{\Predvar}}} = 0$$
$$\E{\prederrf{\truemean} \cd \paren{\predfnf{\Predvar} - \truemeanf{\Predvar}}} = 0$$

:::

Expand All @@ -15,11 +15,11 @@ by @thm-cond-pull-out it factors out of the [conditional expectation](#def-cond-

$$
\ba
\E{\esterr{\truemeanf{\Predvar}}{\Outvar}\paren{\predfnf{\Predvar} - \truemeanf{\Predvar}} \mid \Predvar}
&= \paren{\predfnf{\Predvar} - \truemeanf{\Predvar}} \cd \E{\esterr{\truemeanf{\Predvar}}{\Outvar} \mid \Predvar}
\E{\prederrf{\truemean}\paren{\predfnf{\Predvar} - \truemeanf{\Predvar}} \mid \Predvar}
&= \paren{\predfnf{\Predvar} - \truemeanf{\Predvar}} \cd \E{\prederrf{\truemean} \mid \Predvar}
&& \text{(a function of } \Predvar \text{ factors out)} \\
&= \paren{\predfnf{\Predvar} - \truemeanf{\Predvar}} \cd \E{\truemeanf{\Predvar} - \Outvar \mid \Predvar}
&& \text{(definition of } \esterr{\truemeanf{\Predvar}}{\Outvar} \text{)} \\
&& \text{(definition of } \prederrf{\truemean} \text{)} \\
&= \paren{\predfnf{\Predvar} - \truemeanf{\Predvar}} \cd \paren{\E{\truemeanf{\Predvar} \mid \Predvar} - \E{\Outvar \mid \Predvar}}
&& \text{(linearity of conditional expectation)} \\
&= \paren{\predfnf{\Predvar} - \truemeanf{\Predvar}} \cd \paren{\truemeanf{\Predvar} - \truemeanf{\Predvar}}
Expand All @@ -33,8 +33,8 @@ By the law of iterated expectations (@thm-lie):

$$
\ba
\E{\esterr{\truemeanf{\Predvar}}{\Outvar}\paren{\predfnf{\Predvar} - \truemeanf{\Predvar}}}
&= \E{\E{\esterr{\truemeanf{\Predvar}}{\Outvar}\paren{\predfnf{\Predvar} - \truemeanf{\Predvar}} \mid \Predvar}}
\E{\prederrf{\truemean}\paren{\predfnf{\Predvar} - \truemeanf{\Predvar}}}
&= \E{\E{\prederrf{\truemean}\paren{\predfnf{\Predvar} - \truemeanf{\Predvar}} \mid \Predvar}}
&& \text{(law of iterated expectations)} \\
&= \E{0}
&& \text{(the conditional expectation is } 0 \text{)} \\
Expand All @@ -52,7 +52,7 @@ $$

In @exr-best-predictor, take the constant [prediction](#def-prediction) $\predfn_1\paren{\Predvar} = 3$.
When $\Predvar = 0$, $\truemeanf{0} = 1$, so $\predfn_1\paren{0} - \truemeanf{0} = 2$,
and $\esterr{\truemeanf{\Predvar}}{\Outvar} = 1 - \Outvar$ is $1$ or $-1$ with equal [probability](#def-probability),
and $\prederrf{\truemean} = 1 - \Outvar$ is $1$ or $-1$ with equal [probability](#def-probability),
so the product has conditional mean $0$.
The same cancellation happens when $\Predvar = 1$,
so the expectation of the product is $0$.
Expand Down
14 changes: 7 additions & 7 deletions _subfiles/_lem-prediction-error-split.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -5,21 +5,21 @@ Let $\truemeanf{\Predvar} \eqdef \E{\Outvar \mid \Predvar}$ be the [conditional
For any [prediction function](#def-prediction) $\predfnf{\Predvar}$,
the [prediction error](#def-prediction-error) splits as:

$$\esterr{\predfnf{\Predvar}}{\Outvar} = \esterr{\truemeanf{\Predvar}}{\Outvar} + \paren{\predfnf{\Predvar} - \truemeanf{\Predvar}}$$
$$\prederrf{\predfn} = \prederrf{\truemean} + \paren{\predfnf{\Predvar} - \truemeanf{\Predvar}}$$

:::

::: proof

$$
\ba
\esterr{\predfnf{\Predvar}}{\Outvar}
\prederrf{\predfn}
&= \predfnf{\Predvar} - \Outvar
&& \text{(definition of } \esterr{\predfnf{\Predvar}}{\Outvar} \text{)} \\
&& \text{(definition of } \prederrf{\predfn} \text{)} \\
&= \paren{\truemeanf{\Predvar} - \Outvar} + \paren{\predfnf{\Predvar} - \truemeanf{\Predvar}}
&& \text{(add and subtract } \truemeanf{\Predvar} \text{)} \\
&= \esterr{\truemeanf{\Predvar}}{\Outvar} + \paren{\predfnf{\Predvar} - \truemeanf{\Predvar}}
&& \text{(definition of } \esterr{\truemeanf{\Predvar}}{\Outvar} \text{)}
&= \prederrf{\truemean} + \paren{\predfnf{\Predvar} - \truemeanf{\Predvar}}
&& \text{(definition of } \prederrf{\truemean} \text{)}
\ea
$$

Expand All @@ -32,9 +32,9 @@ $$

In @exr-best-predictor,
when $\Predvar = 0$ and $\Outvar = 0$,
the constant [prediction](#def-prediction) $\predfn_1\paren{\Predvar} = 3$ has [prediction error](#def-prediction-error) $\esterr{\predfn_1\paren{\Predvar}}{\Outvar} = 3$.
the constant [prediction](#def-prediction) $\predfn_1\paren{\Predvar} = 3$ has [prediction error](#def-prediction-error) $\prederrf{\predfn_1} = 3$.
The conditional mean is $\truemeanf{0} = 1$,
so $\esterr{\truemeanf{\Predvar}}{\Outvar} = 1$ and $\predfn_1\paren{0} - \truemeanf{0} = 2$,
so $\prederrf{\truemean} = 1$ and $\predfn_1\paren{0} - \truemeanf{0} = 2$,
and $3 = 1 + 2$.

:::
15 changes: 10 additions & 5 deletions _subfiles/_thm-risk-decomposition.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -19,15 +19,20 @@ and @thm-linearity-expectation:
$$
\ba
\popriskf{\predfn}
&= \E{\sqf{\esterr{\predfnf{\Predvar}}{\Outvar}}}
&= \E{\sqf{\prederrf{\predfn}}}
&& \text{(definition of the squared-error risk)} \\
&= \E{\sqf{\esterr{\truemeanf{\Predvar}}{\Outvar} + \paren{\predfnf{\Predvar} - \truemeanf{\Predvar}}}}
&= \E{\sqf{\prederrf{\truemean} + \devof{\predfnf{\Predvar}}{\truemeanf{\Predvar}}}}
&& \text{(the prediction error splits at } \truemean \text{)} \\
&= \E{\sqf{\esterr{\truemeanf{\Predvar}}{\Outvar}}} + 2 \cd \E{\esterr{\truemeanf{\Predvar}}{\Outvar}\paren{\predfnf{\Predvar} - \truemeanf{\Predvar}}} + \E{\sqf{\predfnf{\Predvar} - \truemeanf{\Predvar}}}
&= \E{\sqf{\prederrf{\truemean}}}
+ 2 \cd \E{\prederrf{\truemean} \cd \devof{\predfnf{\Predvar}}{\truemeanf{\Predvar}}}
+ \E{\sqf{\devof{\predfnf{\Predvar}}{\truemeanf{\Predvar}}}}
&& \text{(expand the square; linearity of expectation)} \\
&= \E{\sqf{\esterr{\truemeanf{\Predvar}}{\Outvar}}} + 2 \cd 0 + \E{\sqf{\predfnf{\Predvar} - \truemeanf{\Predvar}}}
&= \E{\sqf{\prederrf{\truemean}}}
+ 2 \cd 0
+ \E{\sqf{\devof{\predfnf{\Predvar}}{\truemeanf{\Predvar}}}}
&& \text{(the cross term is } 0 \text{)} \\
&= \popriskf{\truemean} + \E{\sqf{\predfnf{\Predvar} - \truemeanf{\Predvar}}}
&= \popriskf{\truemean}
+ \E{\sqf{\devof{\predfnf{\Predvar}}{\truemeanf{\Predvar}}}}
&& \text{(definition of the squared-error risk of } \truemean \text{)}
\ea
$$
Expand Down
2 changes: 1 addition & 1 deletion latex-macros
Loading