Revalidate fetch redirects against network policy (#647)

docs/adr-020-revalidate-fetch-redirects.md

@@ -0,0 +1,83 @@
+# Architectural decision record (ADR) 020: Revalidate every fetch redirect
+
+## Status
+
+Accepted.
+
+## Date
+
+2026-09-02.
+
+## Context and issue
+
+`fetch()` evaluates the caller-supplied URL against `NetworkPolicy`, which
+limits schemes and hosts before opening a connection. The HTTP client formerly
+followed redirects itself, so an allowed origin could redirect a manifest fetch
+to a link-local address, blocked host, or non-allowlisted host without another
+policy decision. That gap turns a permitted request into a server-side request
+forgery opportunity. Issue #647 requires the least-privilege policy to cover
+each outbound hop rather than only the initial URL.
+
+## Decision
+
+Disable ureq's automatic redirects and follow redirects in the fetch adapter.
+Before every redirected connection, resolve `Location` relative to the current
+URL, remove URL credentials when the origin changes, and evaluate the resolved
+target against `NetworkPolicy`. The adapter accepts at most five redirects and
+rejects a repeated target.
+
+The cache identity remains the original caller-supplied URL. A cache miss
+validates every redirect hop before its response body is written under that
+original key. A cache hit opens no outbound connection; the original URL is
+still evaluated before the entry is read.
+
+## Rationale
+
+- **Policy is an outbound-hop invariant.** Checking the target before each
+  request ensures a redirect cannot bypass scheme, allowlist, blocklist, or
+  missing-host checks. An HTTPS-to-HTTP downgrade is therefore rejected unless
+  `http` is explicitly allowed and its host passes the same policy.
+- **Manual handling makes ordering auditable.** Disabling ureq redirects makes
+  the policy check visibly precede every redirected network operation.
+- **The loop is finite and deterministic.** Relative `Location` values resolve
+  against the preceding URL, five accepted redirects is the upper bound, and a
+  repeated resolved URL produces a loop error rather than another request.
+- **Telemetry and diagnostics stay redacted.** Redirect decisions emit only
+  operation, outcome, reason, and hop fields. They never emit a location, URL,
+  host, or userinfo. Error URLs remove userinfo before localization, following
+  ADR-009's bounded-redaction contract.
+
+## Consequences
+
+- Redirect responses without `Location`, invalid locations, policy rejections,
+  loops, and over-limit chains now have distinct localized diagnostics.
+- GET remains the request method at every accepted hop. No caller-derived
+  headers are configured on redirected requests, and cross-origin URL
+  credentials are stripped before the next request.
+- Redirected responses have one cache entry per original fetch URL, not one per
+  final destination. Cached and uncached cache-miss paths therefore apply the
+  same hop policy before a body can be stored.
+
+## Alternatives considered
+
+- **Retain ureq automatic redirects.** Rejected because the default redirect
+  handler has no Netsuke policy callback before each destination connection.
+- **Check only a final response URL.** Rejected because the disallowed request
+  has already occurred by the time a final URL is available.
+- **Use redirect destinations as cache keys.** Rejected because callers request
+  the original URL and an allowed endpoint can legitimately change its final
+  location. Recording the original request as the identity preserves existing
+  cache semantics without allowing an unchecked hop.
+
+## Implementation references
+
+- Redirect adapter and redacted diagnostics:
+  [`src/stdlib/network/redirect.rs`](../src/stdlib/network/redirect.rs)
+- Policy evaluation:
+  [`src/stdlib/network/policy/mod.rs`](../src/stdlib/network/policy/mod.rs)
+- Original-URL cache key:
+  [`src/stdlib/network/cache.rs`](../src/stdlib/network/cache.rs)
+- Two-server and cache coverage:
+  [`tests/std_filter_tests/network_redirect_tests.rs`](../tests/std_filter_tests/network_redirect_tests.rs)
+  and
+  [`src/stdlib/network/redirect_tests.rs`](../src/stdlib/network/redirect_tests.rs)

docs/contents.md

@@ -148,6 +148,8 @@ operator, user, and contributor references are easier to find.
 - [ADR-019](adr-019-structured-command-shell-selection.md): Allow-listed
   structured-command shell selection, trusted configuration authority,
   resolution, lowering, diagnostics, and safety boundaries.
+- [ADR-020](adr-020-revalidate-fetch-redirects.md): Redirect policy decision
+  record, making network policy an invariant of every outbound fetch hop.
 
 ## Proposals
 

docs/developers-guide.md

@@ -3703,6 +3703,26 @@ ambient boundary stays where the lint expects it. Prefer that shape — pass in
 what the operation needs and keep the handle here — over widening an exclusion
 to a module that wants a raw `File`.
 
+### `test_support::http`
+
+`test_support::http` owns the local HTTP server fixtures used by unit,
+integration, and behavioural tests that exercise network-facing helpers. Its
+public response model, `HttpResponse`, is composed with `spawn_http_server`,
+`spawn_http_server_with_config`, or `spawn_http_server_responses`. The first
+two preserve the one-request fixture contract and emit `200 OK` by default; the
+response-sequence helper is the composition point for redirect chains and
+returns a request counter for asserting which requests were received.
+
+Only test code may call these helpers. Use separate fixture instances for a
+redirecting origin and its target, and use the target counter when a policy
+decision must prove that no connection was attempted. Configure response
+status, headers, and body through `HttpResponse`; do not add protocol-specific
+server logic to individual tests when the response sequence already expresses
+the scenario. Keep one-off fixtures for behaviour that cannot be represented by
+this local server, and do not use the fixture as a production HTTP adapter. The
+server's bounded accept and read deadlines, plus its drop-time cleanup, keep
+expected zero-request cases from stalling the suite.
+
 ### `test_support::ensure_manifest_exists`
 
 `test_support::ensure_manifest_exists` (`test_support/src/manifest.rs`) never

docs/netsuke-design.md

@@ -1864,7 +1864,34 @@ Implementation details:
   `--fetch-allow-scheme <SCHEME>`, declare explicit host allowlists via
   `--fetch-allow-host <HOST>` and `--fetch-default-deny`, and block individual
   hosts through `--fetch-block-host <HOST>`. Policy failures abort before a
-  network call and leave the template marked pure.
+  network call and leave the template marked pure. Redirect handling applies
+  the same policy before every outbound hop; the decision, including bounded
+  redirect handling and cache identity, is recorded in
+  [ADR-020](adr-020-revalidate-fetch-redirects.md).
+
+For screen readers: `fetch` dispatches the current hop until it receives a
+non-redirect response. For a redirect, it resolves the location and rejects a
+missing or invalid location. It then rejects a target that fails policy, has
+already appeared in the chain, or would exceed the five-hop limit; only an
+allowed unseen target becomes the next current hop.
+
+```mermaid
+stateDiagram-v2
+    [*] --> CurrentHop
+    CurrentHop --> FinalResponse: non-redirect response
+    CurrentHop --> ResolveLocation: redirect response
+    ResolveLocation --> Reject: missing or invalid Location
+    ResolveLocation --> CheckTarget: resolved target
+    CheckTarget --> Reject: NetworkPolicy rejects
+    CheckTarget --> Reject: repeated target
+    CheckTarget --> Reject: five-hop limit reached
+    CheckTarget --> CurrentHop: allowed unseen target
+    FinalResponse --> [*]
+    Reject --> [*]
+```
+
+*Figure: Policy-checked `fetch` redirect state transitions.*
+
 - `manifest::from_path` derives the workspace root from the manifest file's
   directory before registering the stdlib. This keeps caches scoped to the
   manifest tree even when the CLI evaluates a manifest from another working

docs/security-network-command-audit.md

@@ -36,6 +36,15 @@ introduces, and concrete remediation tasks that would harden the helpers.
     `--fetch-block-host` CLI options. `--fetch-default-deny` switches the
     policy to "block by default" with an explicit allowlist. Policy violations
     surface as `InvalidOperation` errors without opening a network connection.
+- [x] **Redirects bypass outbound request policy.** *(Status: remediated in
+  issue #647.)* An allowed HTTP endpoint could redirect `fetch` to a target
+  that the initial policy would block, including link-local metadata services.
+  **Remediation:** ureq automatic redirects are disabled. Netsuke resolves and
+  evaluates every `Location` target before opening its connection, bounds
+  chains to five hops, detects loops, strips cross-origin URL credentials, and
+  redacts redirect diagnostics. Cached responses retain the original requested
+  URL as their key; every cache miss validates its complete redirect chain
+  before a response body is stored.
 - [x] **Response bodies are read without a size limit.** `fetch_remote` reads
   the entire HTTP response into memory before returning or caching it. An
   attacker controlling the endpoint can stream unbounded data and exhaust

docs/users-guide.md

@@ -956,6 +956,26 @@ every helper's signature, defaults, purity, platform caveats, and executable
 examples. Host-observing helpers belong only in trusted manifests: Netsuke
 bounds command and network output, but does not sandbox template evaluation.
 
+### Network fetch policy
+
+`fetch()` applies the configured `NetworkPolicy` to the caller-supplied URL and
+to every redirect destination before opening a connection. The default policy
+allows only HTTPS; `--fetch-allow-scheme`, `--fetch-allow-host`,
+`--fetch-default-deny`, and `--fetch-block-host` adjust the scheme and host
+rules. A redirect from HTTPS to HTTP therefore succeeds only when `http` is
+explicitly allowed and the destination host also passes the policy.
+
+Supported redirects (`301`, `302`, `303`, `307`, and `308`) retain GET
+semantics, resolve relative `Location` values against the current URL, and stop
+after five redirects. Repeated destinations, missing or invalid `Location`
+values, and policy-rejected destinations fail with distinct localized
+diagnostics. Credentials in the URL are removed when a redirect changes origin,
+and redirect diagnostics do not disclose URL userinfo.
+
+When `cache=true`, the cache entry is identified by the original URL. A cache
+miss applies the same policy checks to every redirect before storing the body;
+a cache hit performs no network request.
+
 When a Boolean is interpolated into a string field, Netsuke renders it as
 lowercase `true` or `false`. For example, this writes `true` to `status.txt`:
 

locales/ar/messages.ftl

@@ -227,6 +227,11 @@ stdlib.config.cwd_non_utf8 = يتضمّن الدليل الحالي أجزاءً
 stdlib.fetch.url_invalid = عنوان URL غير صالح «{ $url }»: { $details }.
 stdlib.fetch.disallowed = العنوان URL «{ $url }» غير مسموح به: { $details }.
 stdlib.fetch.failed = تعذّر جلب «{ $url }»: { $details }.
+stdlib.fetch.redirect_loop = ‏Redirect loop detected at '{ $url }'.
+stdlib.fetch.redirect_limit_exceeded = ‏Redirect limit of { $limit } exceeded while fetching '{ $url }'.
+stdlib.fetch.redirect_location_invalid = ‏Invalid redirect location '{ $location }' from '{ $url }': { $details }.
+stdlib.fetch.redirect_disallowed = ‏Redirect URL '{ $url }' is disallowed: { $details }.
+stdlib.fetch.redirect_location_missing = ‏Redirect response from '{ $url }' did not include a Location header.
 stdlib.fetch.cache_read_failed = تعذّرت قراءة مدخل الذاكرة المخبّأة «{ $name }»: { $details }.
 stdlib.fetch.cache_open_failed = تعذّر فتح مدخل الذاكرة المخبّأة «{ $name }»: { $details }.
 stdlib.fetch.response_read_failed = تعذّرت قراءة الاستجابة من «{ $url }»: { $details }.

locales/cs/messages.ftl

@@ -227,6 +227,11 @@ stdlib.config.cwd_non_utf8 = Aktuální adresář obsahuje části, které nejso
 stdlib.fetch.url_invalid = Neplatná adresa URL „{ $url }“: { $details }.
 stdlib.fetch.disallowed = Adresa URL „{ $url }“ není povolena: { $details }.
 stdlib.fetch.failed = Z adresy „{ $url }“ se nepodařilo stáhnout data: { $details }.
+stdlib.fetch.redirect_loop = Redirect loop detected at '{ $url }'.
+stdlib.fetch.redirect_limit_exceeded = Redirect limit of { $limit } exceeded while fetching '{ $url }'.
+stdlib.fetch.redirect_location_invalid = Invalid redirect location '{ $location }' from '{ $url }': { $details }.
+stdlib.fetch.redirect_disallowed = Redirect URL '{ $url }' is disallowed: { $details }.
+stdlib.fetch.redirect_location_missing = Redirect response from '{ $url }' did not include a Location header.
 stdlib.fetch.cache_read_failed = Položku mezipaměti „{ $name }“ se nepodařilo přečíst: { $details }.
 stdlib.fetch.cache_open_failed = Položku mezipaměti „{ $name }“ se nepodařilo otevřít: { $details }.
 stdlib.fetch.response_read_failed = Odpověď z „{ $url }“ se nepodařilo přečíst: { $details }.

locales/cy/messages.ftl

@@ -227,6 +227,11 @@ stdlib.config.cwd_non_utf8 = Mae'r cyfeiriadur cyfredol yn cynnwys rhannau nad y
 stdlib.fetch.url_invalid = URL annilys ‘{ $url }’: { $details }.
 stdlib.fetch.disallowed = Ni chaniateir yr URL ‘{ $url }’: { $details }.
 stdlib.fetch.failed = Methwyd â nôl ‘{ $url }’: { $details }.
+stdlib.fetch.redirect_loop = Redirect loop detected at '{ $url }'.
+stdlib.fetch.redirect_limit_exceeded = Redirect limit of { $limit } exceeded while fetching '{ $url }'.
+stdlib.fetch.redirect_location_invalid = Invalid redirect location '{ $location }' from '{ $url }': { $details }.
+stdlib.fetch.redirect_disallowed = Redirect URL '{ $url }' is disallowed: { $details }.
+stdlib.fetch.redirect_location_missing = Redirect response from '{ $url }' did not include a Location header.
 stdlib.fetch.cache_read_failed = Methwyd â darllen cofnod y storfa ‘{ $name }’: { $details }.
 stdlib.fetch.cache_open_failed = Methwyd ag agor cofnod y storfa ‘{ $name }’: { $details }.
 stdlib.fetch.response_read_failed = Methwyd â darllen yr ymateb o ‘{ $url }’: { $details }.

locales/da/messages.ftl

@@ -227,6 +227,11 @@ stdlib.config.cwd_non_utf8 = Den aktuelle mappe indeholder dele, der ikke er UTF
 stdlib.fetch.url_invalid = Ugyldig URL-adresse "{ $url }": { $details }.
 stdlib.fetch.disallowed = URL-adressen "{ $url }" er ikke tilladt: { $details }.
 stdlib.fetch.failed = "{ $url }" kunne ikke hentes: { $details }.
+stdlib.fetch.redirect_loop = Redirect loop detected at '{ $url }'.
+stdlib.fetch.redirect_limit_exceeded = Redirect limit of { $limit } exceeded while fetching '{ $url }'.
+stdlib.fetch.redirect_location_invalid = Invalid redirect location '{ $location }' from '{ $url }': { $details }.
+stdlib.fetch.redirect_disallowed = Redirect URL '{ $url }' is disallowed: { $details }.
+stdlib.fetch.redirect_location_missing = Redirect response from '{ $url }' did not include a Location header.
 stdlib.fetch.cache_read_failed = Opslaget "{ $name }" i mellemlageret kunne ikke læses: { $details }.
 stdlib.fetch.cache_open_failed = Opslaget "{ $name }" i mellemlageret kunne ikke åbnes: { $details }.
 stdlib.fetch.response_read_failed = Svaret fra "{ $url }" kunne ikke læses: { $details }.

locales/de/messages.ftl

@@ -227,6 +227,11 @@ stdlib.config.cwd_non_utf8 = Das aktuelle Verzeichnis enthält Komponenten ohne
 stdlib.fetch.url_invalid = Ungültige URL „{ $url }“: { $details }.
 stdlib.fetch.disallowed = Die URL „{ $url }“ ist nicht zugelassen: { $details }.
 stdlib.fetch.failed = „{ $url }“ konnte nicht abgerufen werden: { $details }.
+stdlib.fetch.redirect_loop = Redirect loop detected at '{ $url }'.
+stdlib.fetch.redirect_limit_exceeded = Redirect limit of { $limit } exceeded while fetching '{ $url }'.
+stdlib.fetch.redirect_location_invalid = Invalid redirect location '{ $location }' from '{ $url }': { $details }.
+stdlib.fetch.redirect_disallowed = Redirect URL '{ $url }' is disallowed: { $details }.
+stdlib.fetch.redirect_location_missing = Redirect response from '{ $url }' did not include a Location header.
 stdlib.fetch.cache_read_failed = Der Cache-Eintrag „{ $name }“ konnte nicht gelesen werden: { $details }.
 stdlib.fetch.cache_open_failed = Der Cache-Eintrag „{ $name }“ konnte nicht geöffnet werden: { $details }.
 stdlib.fetch.response_read_failed = Die Antwort von „{ $url }“ konnte nicht gelesen werden: { $details }.

locales/el/messages.ftl

@@ -228,6 +228,11 @@ stdlib.config.cwd_non_utf8 = Ο τρέχων κατάλογος περιέχει
 stdlib.fetch.url_invalid = Μη έγκυρη διεύθυνση URL «{ $url }»: { $details }.
 stdlib.fetch.disallowed = Η διεύθυνση URL «{ $url }» δεν επιτρέπεται: { $details }.
 stdlib.fetch.failed = Δεν ήταν δυνατή η λήψη του «{ $url }»: { $details }.
+stdlib.fetch.redirect_loop = Redirect loop detected at '{ $url }'.
+stdlib.fetch.redirect_limit_exceeded = Redirect limit of { $limit } exceeded while fetching '{ $url }'.
+stdlib.fetch.redirect_location_invalid = Invalid redirect location '{ $location }' from '{ $url }': { $details }.
+stdlib.fetch.redirect_disallowed = Redirect URL '{ $url }' is disallowed: { $details }.
+stdlib.fetch.redirect_location_missing = Redirect response from '{ $url }' did not include a Location header.
 stdlib.fetch.cache_read_failed = Δεν ήταν δυνατή η ανάγνωση της καταχώρισης κρυφής μνήμης «{ $name }»: { $details }.
 stdlib.fetch.cache_open_failed = Δεν ήταν δυνατό το άνοιγμα της καταχώρισης κρυφής μνήμης «{ $name }»: { $details }.
 stdlib.fetch.response_read_failed = Δεν ήταν δυνατή η ανάγνωση της απόκρισης από «{ $url }»: { $details }.

locales/en-GB/messages.ftl

@@ -227,6 +227,11 @@ stdlib.config.cwd_non_utf8 = Current directory contains non-UTF-8 components: {
 stdlib.fetch.url_invalid = Invalid URL '{ $url }': { $details }.
 stdlib.fetch.disallowed = URL '{ $url }' is disallowed: { $details }.
 stdlib.fetch.failed = Failed to fetch '{ $url }': { $details }.
+stdlib.fetch.redirect_loop = Redirect loop detected at '{ $url }'.
+stdlib.fetch.redirect_limit_exceeded = Redirect limit of { $limit } exceeded while fetching '{ $url }'.
+stdlib.fetch.redirect_location_invalid = Invalid redirect location '{ $location }' from '{ $url }': { $details }.
+stdlib.fetch.redirect_disallowed = Redirect URL '{ $url }' is disallowed: { $details }.
+stdlib.fetch.redirect_location_missing = Redirect response from '{ $url }' did not include a Location header.
 stdlib.fetch.cache_read_failed = Failed to read fetch cache entry '{ $name }': { $details }.
 stdlib.fetch.cache_open_failed = Failed to open fetch cache entry '{ $name }': { $details }.
 stdlib.fetch.response_read_failed = Failed to read response from '{ $url }': { $details }.

locales/en-US/messages.ftl

@@ -227,6 +227,11 @@ stdlib.config.cwd_non_utf8 = Current directory contains non-UTF-8 components: {
 stdlib.fetch.url_invalid = Invalid URL '{ $url }': { $details }.
 stdlib.fetch.disallowed = URL '{ $url }' is disallowed: { $details }.
 stdlib.fetch.failed = Failed to fetch '{ $url }': { $details }.
+stdlib.fetch.redirect_loop = Redirect loop detected at '{ $url }'.
+stdlib.fetch.redirect_limit_exceeded = Redirect limit of { $limit } exceeded while fetching '{ $url }'.
+stdlib.fetch.redirect_location_invalid = Invalid redirect location '{ $location }' from '{ $url }': { $details }.
+stdlib.fetch.redirect_disallowed = Redirect URL '{ $url }' is disallowed: { $details }.
+stdlib.fetch.redirect_location_missing = Redirect response from '{ $url }' did not include a Location header.
 stdlib.fetch.cache_read_failed = Failed to read fetch cache entry '{ $name }': { $details }.
 stdlib.fetch.cache_open_failed = Failed to open fetch cache entry '{ $name }': { $details }.
 stdlib.fetch.response_read_failed = Failed to read response from '{ $url }': { $details }.