diff --git a/.changeset/headless-static-override.md b/.changeset/headless-static-override.md new file mode 100644 index 0000000..bb16ed5 --- /dev/null +++ b/.changeset/headless-static-override.md @@ -0,0 +1,5 @@ +--- +"@bootnodedev/cbn": patch +--- + +Headless validators are now driven by the static `templates/runtime-overrides.yaml` via env vars: each validator's nginx route template is mounted from `${*_NGINX_ROUTES}`, pointing at Splice's real routing config when the UI is on or at an empty file when it is off. `writeGeneratedOverride` and the generated `.generated/service-overrides.yaml` are gone — no YAML is generated from JS anymore. diff --git a/README.md b/README.md index 38f5cff..9ab1e7c 100644 --- a/README.md +++ b/README.md @@ -112,8 +112,8 @@ Rules: | ------------------------ | ------------------------------------------------------------------------------------------------------------------- | | always | `--profile sv` runs the SV fully and, with it, the shared postgres/canton/splice/nginx | | `validators.*.enabled` | switches that validator's backend on/off via an env var | -| `validators.*.ui` | on → also starts that validator's UIs; off (but enabled) → nginx is told to skip that validator so it runs headless | -| an `sv` UI flag off | env vars pin that UI to 0 replicas and alias its hostname onto nginx (static `templates/runtime-overrides.yaml`) | +| `validators.*.ui` | on → also starts that validator's UIs; off (but enabled) → an env var blanks its nginx routes so it runs headless | +| an `sv` UI flag off | env vars pin that UI to 0 replicas and alias its hostname onto nginx | | a `networkTools` flag on | starts that tool via its profile | So the default config launches the SV plus a headless `appUser`, and each flag you flip adds more. (For _why_ a headless validator needs special handling, see [Design notes](#design-notes).) @@ -231,7 +231,7 @@ Non-obvious behaviors worth knowing before automating against the stack: | ------------------------- | ---------------------------------------------------------------------------------------------------- | | `bin/canton-barebones.js` | CLI entry point: parses the command and `--json`, dispatches to `src/*` | | `src/config.js` | Loads and zod-validates the config; resolves runtime paths and the pinned Splice checkout | -| `src/compose.js` | Turns the config into the Docker Compose invocation (profiles, env, generated overrides) and runs it | +| `src/compose.js` | Turns the config into the Docker Compose invocation (profiles, env vars, override chain) and runs it | | `src/splice.js` | Downloads and pins the Splice LocalNet source into `.generated/` | | `src/init.js` | Scaffolds the bundled templates into the project (`init`) | | `src/output.js` | Centralizes CLI output: human text vs `--json`, stdout vs stderr | @@ -277,20 +277,21 @@ Running a validator with its backend on but its UIs off (`enabled: true, ui: fal 2. Splice mounts each validator's routing config there (e.g. `app-user.conf` → `/etc/nginx/templates/app-user.conf.template`). That config proxies to the validator's UI containers as fixed upstreams, and nginx **refuses to start** if an upstream host does not exist. 3. Because the validator's backend is on, Splice renders its nginx config — so nginx would look for UI containers that we did not start, and crash. -CantonBarebones works around this without editing any Splice file. Docker Compose deduplicates volume mounts by their target path, and the **last `-f` wins**. So the wrapper writes a generated override (`.generated/service-overrides.yaml`, passed last) that mounts an **empty file** over that exact template path: +CantonBarebones works around this without editing any Splice file and without generating YAML. Docker Compose deduplicates volume mounts by their target path, and the **later `-f` wins**. The static override shipped inside the package (`templates/runtime-overrides.yaml`, always applied after Splice's files) mounts an env-var-selected **source file** over that exact template path: ``` -Splice: conf/nginx/app-user.conf → /etc/nginx/templates/app-user.conf.template -Generated override: (empty file) → /etc/nginx/templates/app-user.conf.template ← wins +Splice: conf/nginx/app-user.conf → /etc/nginx/templates/app-user.conf.template +Static override: ${APP_USER_NGINX_ROUTES} → /etc/nginx/templates/app-user.conf.template ← wins -nginx renders an empty app-user.conf → no routes for it → starts fine. +ui: true → the var points at Splice's own app-user.conf → identical routes, nothing changes. +ui: false → it points at an empty file in .generated/ → nginx renders no routes → starts fine. ``` -The validator's backend still runs (it is driven by an env var, not nginx) and stays reachable on its direct API ports; only its web routing is dropped. See `templates/splice-localnet-overrides.yaml` and `src/compose.js` for the full detail. +The wrapper only writes the `APP_*_NGINX_ROUTES` env vars to `.generated/localnet.env`. The validator's backend still runs (it is driven by an env var, not nginx) and stays reachable on its direct API ports; only its web routing is dropped. See `templates/runtime-overrides.yaml` and `writeLocalnetEnv` in `src/compose.js` for the full detail. ### How a disabled SV web UI works (replicas + alias) -Turning off an SV web UI (`sv.scanUI` / `sv.svUI` / `sv.walletUI`) cannot reuse the empty-template trick above: Splice's `sv.conf` mixes the UI routes with **API routes** (scan API, SV admin API, canton JSON API) that proxy to the always-running `splice`/`canton` containers and must stay up, and the flags are per-UI rather than all-or-nothing. Instead, a **static** override shipped inside the package (`templates/runtime-overrides.yaml`, always applied) uses two other compose levers, driven purely by env vars that the wrapper writes to `.generated/localnet.env`: +Turning off an SV web UI (`sv.scanUI` / `sv.svUI` / `sv.walletUI`) cannot reuse the empty-template trick above: Splice's `sv.conf` mixes the UI routes with **API routes** (scan API, SV admin API, canton JSON API) that proxy to the always-running `splice`/`canton` containers and must stay up, and the flags are per-UI rather than all-or-nothing. The same static override uses two other compose levers instead, also driven by env vars from `.generated/localnet.env`: 1. `deploy.replicas: ${…_REPLICAS}` — `0` for a disabled UI, so compose never starts its container. (An override entry rather than `docker compose --scale`, which errors for services whose profile is not selected.) 2. nginx still resolves that UI's hostname at startup (`proxy_pass http://scan-web-ui:8080/` …) and would crash with "host not found in upstream" once no container owns the name. So nginx carries a **network alias** per UI whose _value_ comes from `${…_NGINX_ALIAS}`: for a disabled UI it is the real hostname — the name resolves onto the nginx container itself, nginx boots, and since nothing there listens on the UI port, browsing the disabled UI answers **502** while every API route on port 4000 keeps working. For an enabled UI it is an inert `-unused` (static YAML has no conditionals, so the list entry always exists and only its value changes). diff --git a/scripts/config-validation.test.js b/scripts/config-validation.test.js index d76d68a..bfe998d 100644 --- a/scripts/config-validation.test.js +++ b/scripts/config-validation.test.js @@ -24,7 +24,7 @@ function assertRejects(raw, expectedFragment) { // The scaffolded default shipped by `init`: version 1, a pinned Splice source, // persistent volumes, app-provider off, app-user enabled headless (backend on, -// UIs off), every SV web UI on (the SV backend itself is not in the config — it +// UIs off), every SV web UI flag present (the SV backend itself is not in the config — it // is required infrastructure that always runs), and all network tools off. This // is the baseline; every negative case below clones it and breaks a single rule, // so a failure points to one validation concern. diff --git a/scripts/runtime-plan.test.js b/scripts/runtime-plan.test.js index a96ea7f..4fdbd73 100644 --- a/scripts/runtime-plan.test.js +++ b/scripts/runtime-plan.test.js @@ -10,7 +10,9 @@ import { deriveRuntimePlan, writeLocalnetEnv } from '../src/compose.js'; // writeLocalnetEnv read: the validator flags, the SV UI flags, the network tool // flags, the identifiers echoed into the env file, and a directory to generate // into. The baseline mirrors the scaffolded default — app-provider off, app-user -// headless, all SV UIs on, tools off — and each case below flips one lever. +// headless, tools off — and each case below flips one lever. The SV UIs are all +// on here (the scaffolded default ships them off) so the negative cases can +// disable flags one at a time from a fully-on baseline. function baseConfig(generatedDir) { return { imageTag: '0.6.11', @@ -69,6 +71,37 @@ describe('deriveRuntimePlan sv UI flags', () => { }); }); +// Scenario: the validator route-source env vars. templates/runtime-overrides.yaml +// statically mounts `${*_NGINX_ROUTES}` over each validator's nginx route +// template, so the var's value decides what nginx renders: Splice's real routing +// config when the UI is on, or an empty file — no routes, so nginx never tries +// to resolve UI containers that are not running (headless or disabled validators). +describe('writeLocalnetEnv validator route sources', () => { + // The scaffolded default: app-user enabled without UI (headless) and + // app-provider fully disabled. Both must get the empty routes file, because in + // both cases the validator's UI containers do not run. + it('points ui-less validators at the empty routes file', () => { + const env = readEnvFile(writeLocalnetEnv(baseConfig(generatedDir))); + assert.equal(env.APP_USER_NGINX_ROUTES.endsWith('empty-nginx-routes.conf'), true); + assert.equal(env.APP_PROVIDER_NGINX_ROUTES, env.APP_USER_NGINX_ROUTES); + // The mount source must exist and be empty, or docker would create a + // directory in its place / nginx would render stale routes. + assert.equal(fs.existsSync(env.APP_USER_NGINX_ROUTES), true); + assert.equal(fs.readFileSync(env.APP_USER_NGINX_ROUTES, 'utf8'), ''); + }); + + // A validator with its UI on must mount Splice's real routing config — the + // exact file Splice's own compose mounts — so its routes stay identical. + it("points a ui-enabled validator at Splice's real routing config", () => { + const config = baseConfig(generatedDir); + config.validators.appUser = { enabled: true, ui: true }; + const env = readEnvFile(writeLocalnetEnv(config)); + // baseConfig pins localnetDir to /tmp/localnet, so the resolved source is + // that checkout's nginx config for app-user. + assert.equal(env.APP_USER_NGINX_ROUTES, '/tmp/localnet/conf/nginx/app-user.conf'); + }); +}); + // Scenario: the SV UI env vars. templates/runtime-overrides.yaml is static and // consumes one replicas + alias pair per UI, so these vars ARE the runtime // contract: replicas 0/1 decides whether the container starts, and the alias diff --git a/scripts/smoke.js b/scripts/smoke.js index e98c1bf..ff1dfda 100644 --- a/scripts/smoke.js +++ b/scripts/smoke.js @@ -20,19 +20,22 @@ assert.deepEqual(config.validators, { appProvider: { enabled: false, ui: false }, appUser: { enabled: true, ui: false }, }); -assert.deepEqual(config.sv, { scanUI: true, svUI: true, walletUI: true }); +// The scaffolded default ships every SV web UI off, matching the barebones +// philosophy of the rest of the defaults (turn on what you need). +assert.deepEqual(config.sv, { scanUI: false, svUI: false, walletUI: false }); assert.deepEqual(config.networkTools, { console: false, multiSync: false, swaggerUI: false }); // With the scaffolded default (app-provider off, app-user enabled headless, tools // off), only the SV profile is started: a headless validator adds no `--profile` // (its backend is switched via env), so `upProfiles` stays `['sv']`. That profile // also brings up the shared postgres/canton/splice/nginx. app-user being headless -// means it lands in `headlessValidators`, which triggers the generated nginx override. +// means it lands in `headlessValidators`, and the static runtime override blanks +// its nginx routes: APP_USER_NGINX_ROUTES points at the empty routes file. const plan = deriveRuntimePlan(config); assert.deepEqual(plan.upProfiles, ['sv']); assert.deepEqual(plan.headlessValidators, ['appUser']); -// All SV web UIs are on by default, so nothing is pinned to 0 replicas. -assert.deepEqual(plan.disabledSvUIs, []); +// All SV web UIs are off by default, so all three are pinned to 0 replicas. +assert.deepEqual(plan.disabledSvUIs, ['scan-web-ui', 'sv-web-ui', 'wallet-web-ui-sv']); assert.equal(runtimeEnvPath.endsWith('.generated/localnet.env'), true); assert.equal(config.localnetOverridePath.endsWith('splice-localnet-overrides.yaml'), true); assert.match(localnetOverride, /max-size: "25m"/); diff --git a/src/compose.js b/src/compose.js index f8ca380..76aa818 100644 --- a/src/compose.js +++ b/src/compose.js @@ -9,12 +9,14 @@ // and nginx. // - Each validator's backend switches on/off via an env var (*_PROFILE). // - A validator with its backend on but UIs off would still make nginx try to -// route to its (absent) UIs and fail. To run it headless we blank out its -// nginx route with a generated override (see writeGeneratedOverride). +// route to its (absent) UIs and fail. To run it headless its nginx route +// template is blanked out with an empty file. // - The SV's web UIs ride the always-on `sv` profile, so turning one off is not -// a profile decision either: a static override shipped with this package -// (templates/runtime-overrides.yaml) pins it to 0 replicas and keeps nginx -// bootable, driven purely by env vars written here (see writeLocalnetEnv). +// a profile decision either: the disabled UI is pinned to 0 replicas while +// nginx keeps its hostname resolvable. +// Both UI levers live in a static override shipped with this package +// (templates/runtime-overrides.yaml, see its header for the mechanics), driven +// purely by env vars written here (see writeLocalnetEnv). import fs from 'node:fs'; import path from 'node:path'; import { spawnSync } from 'node:child_process'; @@ -28,14 +30,6 @@ export const allLocalnetProfiles = ['app-provider', 'app-user', 'sv', 'swagger-u // Compose profile that starts each validator's UI bundle. const PARTICIPANT_PROFILE = { appProvider: 'app-provider', appUser: 'app-user' }; -// nginx template file (inside the container) that holds each validator's routes. -// Blanking it out with an empty file makes nginx skip that validator, so it can -// run headless without nginx failing on the missing UI upstreams. -const NGINX_ROUTE_TEMPLATE = { - appProvider: '/etc/nginx/templates/app-provider.conf.template', - appUser: '/etc/nginx/templates/app-user.conf.template', -}; - // Compose profile for each network tool. const TOOL_PROFILE = { console: 'console', multiSync: 'multi-sync', swaggerUI: 'swagger-ui' }; @@ -61,8 +55,8 @@ function profileFlags(profiles) { // - upProfiles: `--profile` flags to start — always `sv` (which also brings up the // shared postgres/canton/splice/nginx), plus a validator's profile when its UIs // are wanted, plus a profile per enabled network tool. -// - headlessValidators: validators whose backend is on but UIs are off; nginx must -// be told to skip their routes (see writeGeneratedOverride). +// - headlessValidators: validators whose backend is on but UIs are off; nginx is +// told to skip their routes via env vars (see writeLocalnetEnv). // - disabledSvUIs: SV web UI services turned off in the config; the static // runtime override pins them to 0 replicas via env vars (see writeLocalnetEnv). export function deriveRuntimePlan(config) { @@ -113,6 +107,25 @@ export function writeLocalnetEnv(config) { const { nodeEnv } = deriveRuntimePlan(config); + // Mounted as a validator's nginx route template whenever its UI is off + // (headless or disabled): empty routes mean nginx has nothing to resolve, so + // it boots without that validator's UI containers. + const emptyRoutesPath = path.resolve(config.generatedDir, 'empty-nginx-routes.conf'); + fs.writeFileSync(emptyRoutesPath, ''); + + // The file the static runtime override mounts as a validator's nginx route + // template: Splice's real routing config when the UI is on (same file Splice + // itself mounts, so routes stay identical), the empty file otherwise. + const routesSource = (key, confName) => + config.validators[key].ui + ? path.resolve(config.localnetDir, 'conf', 'nginx', confName) + : emptyRoutesPath; + + const validatorRoutesEnv = [ + envLine('APP_PROVIDER_NGINX_ROUTES', routesSource('appProvider', 'app-provider.conf')), + envLine('APP_USER_NGINX_ROUTES', routesSource('appUser', 'app-user.conf')), + ]; + // One replicas + alias pair per SV web UI, consumed by the static // templates/runtime-overrides.yaml (see its header for how the two work). The // env prefix is the service name upper-snake-cased, e.g. SCAN_WEB_UI. @@ -135,6 +148,7 @@ export function writeLocalnetEnv(config) { envLine('SV_PROFILE', nodeEnv.SV_PROFILE), envLine('APP_PROVIDER_PROFILE', nodeEnv.APP_PROVIDER_PROFILE), envLine('APP_USER_PROFILE', nodeEnv.APP_USER_PROFILE), + ...validatorRoutesEnv, ...svUiEnv, '', ].join('\n'); @@ -143,35 +157,10 @@ export function writeLocalnetEnv(config) { return runtimeEnvPath; } -// Writes a generated compose override that runs headless validators (backend on, -// UIs off) by replacing their nginx route template with an empty file. Without -// this, nginx would try to route to those validators' absent UI containers and -// fail to start. Docker Compose merges volume mounts by target path with the last -// file winning, so mounting our empty file over Splice's template neutralizes it. -// Returns the path when an override is needed, or null otherwise (so no file is -// passed to -f when nothing is headless). -export function writeGeneratedOverride(config, headlessValidators) { - const overridePath = path.resolve(config.generatedDir, 'service-overrides.yaml'); - if (headlessValidators.length === 0) { - fs.rmSync(overridePath, { force: true }); - return null; - } - fs.mkdirSync(config.generatedDir, { recursive: true }); - const emptyTemplatePath = path.resolve(config.generatedDir, 'empty-nginx-template'); - fs.writeFileSync(emptyTemplatePath, ''); - const mounts = headlessValidators - .map(key => ` - ${emptyTemplatePath}:${NGINX_ROUTE_TEMPLATE[key]}\n`) - .join(''); - const contents = `# Generated from canton-barebones.config.json — do not edit.\n# These validators run their backend but not their UIs. nginx would otherwise try\n# to route to the missing UI containers and fail to start, so their nginx route\n# templates are replaced with an empty file.\nservices:\n nginx:\n volumes:\n${mounts}`; - fs.writeFileSync(overridePath, contents); - return overridePath; -} - // Builds the docker compose arguments that select Splice LocalNet files and profiles. export function dockerComposeArgs(config, options = {}) { const runtimeEnvPath = writeLocalnetEnv(config); const plan = deriveRuntimePlan(config); - const generatedOverridePath = writeGeneratedOverride(config, plan.headlessValidators); const profiles = options.profiles ?? plan.upProfiles; const args = [ 'compose', @@ -189,18 +178,14 @@ export function dockerComposeArgs(config, options = {}) { // constraints keep it from starving the host and match how Splice runs LocalNet. args.push('-f', path.resolve(config.localnetDir, 'resource-constraints.yaml')); - // The package's static runtime override translates the `sv` UI env vars into - // replica pins and nginx aliases; it ships with the package (not scaffolded), - // and goes before the user's override so user tweaks can still win the merge. + // The package's static runtime override translates the UI env vars into empty + // nginx route templates, replica pins, and nginx aliases. It ships with the + // package (not scaffolded) and sits after Splice's files — so its mounts win + // their merge — but before the user's override, so user tweaks can still win. args.push('-f', resolveFromPackage('templates/runtime-overrides.yaml')); args.push('-f', config.localnetOverridePath); - // The generated override must come last so its empty nginx templates win the merge. - if (generatedOverridePath) { - args.push('-f', generatedOverridePath); - } - args.push(...profileFlags(profiles)); return args; } diff --git a/templates/runtime-overrides.yaml b/templates/runtime-overrides.yaml index 5bebe93..167f0ba 100644 --- a/templates/runtime-overrides.yaml +++ b/templates/runtime-overrides.yaml @@ -1,10 +1,21 @@ # Compose override shipped INSIDE the canton-barebones package — it is always # appended to the compose -f chain and is not scaffolded into your project (put -# your own tweaks in splice-localnet-overrides.yaml instead). It turns the `sv` -# config flags into runtime behavior using only env-var substitution; the values -# come from .generated/localnet.env (see writeLocalnetEnv in src/compose.js). +# your own tweaks in splice-localnet-overrides.yaml instead). It turns the +# UI-related config flags into runtime behavior using only env-var substitution; +# the values come from .generated/localnet.env (see writeLocalnetEnv in +# src/compose.js). # -# Two levers per SV web UI: +# Validator routes (`validators.*.ui`) — one volume mount per validator. Splice's +# nginx config proxies to each validator's UI containers as fixed upstreams and +# nginx dies at startup if one is missing, so a validator's routes may only exist +# while its UI containers run. Each *_NGINX_ROUTES var holds the file to mount as +# that validator's route template: Splice's real routing config when `ui: true`, +# or an empty file (no routes, nothing to resolve) when the validator is headless +# or disabled. Compose deduplicates volume mounts by target with the later file +# winning, so this mount replaces the one Splice's compose.yaml declares for the +# same target. +# +# SV web UIs (`sv.scanUI` / `sv.svUI` / `sv.walletUI`) — two levers per UI: # # - replicas: 1 keeps Splice's default; 0 stops a disabled UI from starting. # (An override entry rather than `docker compose --scale`, which errors for @@ -19,6 +30,9 @@ # list entry always exists and only its value changes. services: nginx: + volumes: + - ${APP_PROVIDER_NGINX_ROUTES}:/etc/nginx/templates/app-provider.conf.template + - ${APP_USER_NGINX_ROUTES}:/etc/nginx/templates/app-user.conf.template networks: default: aliases: diff --git a/templates/splice-localnet-overrides.yaml b/templates/splice-localnet-overrides.yaml index 8d11bee..9727025 100644 --- a/templates/splice-localnet-overrides.yaml +++ b/templates/splice-localnet-overrides.yaml @@ -5,28 +5,12 @@ # static file only sets log rotation. It is scaffolded into your project by `init`, # so you can add your own tweaks here. # -# There is also a SECOND, GENERATED override at .generated/service-overrides.yaml -# that the wrapper rewrites on every run (do not edit it). It runs "headless" -# validators — backend on, UIs off — and it works by exploiting the same merge: -# -# - nginx (the nginx Docker image) renders every /etc/nginx/templates/*.template -# through envsubst into /etc/nginx/conf.d/ at startup. -# - Splice mounts each validator's routing config there, e.g. -# conf/nginx/app-user.conf -> /etc/nginx/templates/app-user.conf.template -# and that config proxies to the validator's UI containers as fixed upstreams. -# If those UI containers are not running, nginx fails to start. -# - Compose deduplicates volume mounts by their target path, and the LAST `-f` -# wins. So the generated override mounts an EMPTY file over the exact same -# target (app-user.conf.template). nginx then renders an empty config, adds no -# routes for that validator, and starts fine — while the validator's backend -# still runs and stays reachable on its direct API ports. -# -# See src/compose.js (deriveRuntimePlan / writeGeneratedOverride) for how it is built. -# -# Finally, a THIRD, STATIC override ships inside the canton-barebones package -# (templates/runtime-overrides.yaml, always applied before this file): it turns -# the `sv` config flags into 0-replica pins and nginx aliases for the SV web UIs, -# driven purely by env vars. See that file's header for the full detail. +# There is also a SECOND, STATIC override shipped inside the canton-barebones +# package (templates/runtime-overrides.yaml, always applied right before this +# file): driven purely by env vars from .generated/localnet.env, it blanks a +# headless validator's nginx route template and turns the `sv` config flags into +# 0-replica pins and nginx aliases for the SV web UIs. See that file's header for +# the full mechanics, and writeLocalnetEnv in src/compose.js for the env values. services: postgres: logging: &default-logging