ci(eql): port the EQL release pipeline to the repository root, inert - #941
Draft
tobyhede wants to merge 2 commits into
Draft
ci(eql): port the EQL release pipeline to the repository root, inert#941tobyhede wants to merge 2 commits into
tobyhede wants to merge 2 commits into
Conversation
The EQL subtree arrived with eleven files under `packages/eql/.github/`, a directory GitHub never reads. Seven of them were workflows, and between them they are the entire publishing pipeline for `@cipherstash/eql`: the npm package, the `eql-bindings` crate, the SQL bundle, the docs bundle and the `postgres-eql` image, all of which ship at one version. None of it executed. This ports the lot and keeps it inert, so it can be reviewed and dry-run before trusted publishing is repointed at this repository. Inertness is a derived switch rather than a flag. `scripts/eql-pipeline-armed.mjs` reads `FROZEN_PUBLISHERS` in `scripts/release-gate.mjs` — the map that already records "this package lives here and is published elsewhere" — and every job that publishes an EQL artefact is gated on its answer. The cutover has to delete that entry (the release gate refuses every release until it does), so deleting it is what arms the pipeline; there is no second flag to forget, and forgetting one would fail silently in the direction that publishes an npm package with no SQL release, no docs and no crate. Three paths had to be rewritten because the subtree root is not the package root here: the npm manifest is two levels down at `packages/eql/packages/eql`, mise config is only found from `packages/eql`, and `release-plz/action` needs explicit `manifest_path` / `config` inputs because a composite action's steps do not inherit the caller's `defaults.run.working-directory`. Three of this repo's guards had to grow rather than be exempted, and two real defects came out of them: the prerelease publish job was missing `node-gyp` before its install, and built with `pnpm --filter` rather than through turbo. Claude-Session: https://claude.ai/code/session_01T26DhFPkLfN3qPFDq2ttKw
|
Seven findings from the review of the ported pipeline, all confirmed before acting. Three were guards that could not fire, which is the failure mode this repository cares most about — a check that reads as protection and is incapable of reporting anything. `rebuild-docs.yml` was the worst of them, and worse than reported: it has never run, not once, against `@cipherstash/stack@1.0.0`, `1.1.0` and `1.1.1`. A ref created with GITHUB_TOKEN starts no workflow run, and changesets creates those tags with that token — the same rule this pipeline already works around for the image build. Porting EQL's `eql-*` tag trigger into it would have added a second dead trigger. The EQL docs rebuild is now a job in `release.yml` that fires the webhook from inside the publishing run; the `@cipherstash/stack@*` half is left alone as a pre-existing bug, recorded in that file's header so the next person does not repeat the mistake. `release-plz.yml`'s "refusing to publish the DEV placeholder" guard grepped for `eql_v3`, which appears 23,723 times in every build of that bundle including a DEV one. It now matches the schema version stamp against the crate version — the only line in the bundle that records which build produced it. `classify` accepted any version containing a hyphen where `prepare-bindings-assets.sh` requires `X.Y.Z-(alpha|beta|rc).N`. Under the loose check, `3.0.6-beta` got a public tag and GitHub release before the npm job died on it. Also: `PRE_GA_LATEST` was still `true` in the EQL publish script, so the first prerelease cut through the new job would have moved the `latest` dist-tag off the GA release. The file's own comment said to flip it once 3.0.0 GA shipped; npm's `latest` is 3.0.5. Two documentation defects of my own (a paragraph duplicated into AGENTS.md, and a SECURITY.md sentence contradicting the permissions split this PR introduces), one stale plan checkbox, an over-deep checkout, and a module-scope `execFileSync` whose failure took 40 unrelated assertions with it. Comment volume across everything this branch added is cut by roughly half. Claude-Session: https://claude.ai/code/session_01T26DhFPkLfN3qPFDq2ttKw
tobyhede
force-pushed
the
toby/cip-3742-eql-release-pipeline
branch
from
August 24, 2026 04:46
463f8b4 to
ab4c4f3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
EQL is the SQL layer that stores and queries encrypted data in Postgres. It was imported into this repository as a subtree, and it brought its whole release pipeline with it — into
packages/eql/.github/, a directory GitHub never reads. Eleven files, seven of them workflows, executing on nothing. That pipeline is what publishes@cipherstash/eqland the four artefacts that must ship at the same version as it: theeql-bindingsRust crate, the SQL bundle customers install, the docs bundle, and thepostgres-eqlDocker image.This moves all of it to the repository root, where GitHub will run it, and keeps it inert — nothing reaches npm, crates.io or GHCR.
@cipherstash/eqlis still published fromcipherstash/encrypt-query-language, and until that is repointed, publishing from here would either fail or ship something nobody can install.Inertness is a derived switch, not a flag somebody flips.
scripts/eql-pipeline-armed.mjsreadsFROZEN_PUBLISHERSinscripts/release-gate.mjs— the existing map that records "this package lives here but is published elsewhere" — and every job that publishes an EQL artefact is gated on its answer. The cutover has to delete that entry, because the release gate blocks every release until it does. So deleting it is what arms the pipeline, and there is no second thing to remember. A forgotten flag would have failed silently in the worst direction: an npm package published with no SQL release, no docs and no crate.Changes
Workflows ported out of
packages/eql/.github/(the directory is now deleted)_build-sql.yml.github/workflows/_build-eql-sql.yml_build-docs.yml.github/workflows/_build-eql-docs.ymlrelease.yml.github/workflows/release.yml— npm trusted publishing binds a package to a repository and a workflow filename, so every npm publish here has to live in that one filerelease-plz.ymlrelease-postgres-eql-image.ymlrebuild-docs.ymllint-release.ymlactionlint.yaml,ISSUE_TEMPLATE/docs-feedback.yml.github/release.ymlworkflows/README.mdmainis unprotected and there is no queue). Its still-true half is inAGENTS.mdEach drop is recorded where the shrinking allowlist used to live, in
scripts/__tests__/eql-suite-ci.test.mjs, which now asserts the directory does not exist and that nothing is tracked under it.Paths that had to change, because the subtree root is not the package root here
packages/eql/packages/eql. Upstream read./packages/eql/package.json; here that resolves to a directory with no manifest and the read throws.working_directory: packages/eql.release-plz/actionneeds explicitmanifest_pathandconfiginputs. A composite action's steps do not inherit the caller'sdefaults.run.working-directory, so aworking-directory:on the step would have done nothing — release-plz would have read the monorepo root, found no workspace namingeql-bindings, and exited 0 having published nothing.Supply chain
uses:in the four new workflow files and in the jobs added torelease.ymlis SHA-pinned with a trailing version comment; upstream used floating tags throughout. The two merged files (lint-release.yml,rebuild-docs.yml) keep the pins the root copies already had.cache: trueon threejdx/mise-actionsteps in publish-adjacent jobs. Nowfalseeverywhere, and all four ported release workflows are inscripts/lint-no-workflow-caching.mjs'sTARGETS.AUDITED_ACTIONSgains seven entries. Two of the three Docker actions turned out to have a GitHub-Actions-cache input defaulting to true (docker/setup-qemu-action'scache-image,docker/setup-buildx-action'scache-binary).docker/build-push-actionhas no such toggle, so it gets a newforbiddenInputsrule oncache-from/cache-to— those name a backend, andtype=ghais the GitHub Actions cache.release-plz.ymllost its mise step entirely. No step in that job runs a mise task, and the publishing-job rule (add_shims_to_path: false) would have left mise's toolchain off PATH anyway — so it was a third-party action inside the crates.io-credential job, installing tools nothing could reach.Guards that had to grow rather than be exempted
workflow-publish-permissions.test.mjs—OIDC_JOBSwas doing double duty as "who may publish?" and "who may write to the repository?". Six new jobs need a writable scope (create a release, dispatch a workflow, move a branch ref) and must not be able to publish. Split intoOIDC_JOBSandREPO_WRITE_JOBS, both equalities, with a third assertion that the first stays a subset of the second.workflow-dispatch-job-conditions.test.mjs— it required every job in a dispatchable workflow to run on a manual dispatch.release.ymlnow has two mutually exclusive paths (production onmain, prerelease on a marker commit elsewhere), so no single synthetic context can run both. The per-job requirement is kept; the exceptions become an equality-checked list with reasons, so a job that quietly stops running on dispatch still fails.eql-suite-ci.test.mjs— its rule "every job that compiles Rust restores the shared cache" collides head-on with "a publishing workflow restores no cache". Supply chain wins and four EQL release jobs pay a cold compile. The exemption is derived from the caching linter's own target list rather than copied, so the two cannot disagree about which jobs those are.Two real defects the guards found in the port, both because upstream's repository had neither dependency: the prerelease publish job was missing
npm install -g node-gypbeforepnpm install(node-pty's install hook needs it on Linux), and it built withpnpm --filter … buildinstead ofpnpm exec turbo run build(so^builddependencies were whatever an earlier step happened to leave behind).Docs —
AGENTS.md,SECURITY.mdand the absorption plan updated: what the pipeline is, what arms it, and the two things the cutover still needs that no workflow can assert ahead of time (aGPG_PRIVATE_KEYsecret, and write access from this repository to theghcr.io/cipherstash/postgres-eqlpackage, which is currently linked to the old repository).Verification
The dry run. Added a temporary
@cipherstash/eqlpatch changeset, ranpnpm run version, confirmed the lockstep hook moved every artefact together, then reverted the tree to clean:Nothing publishes, proven by the gate rather than by observing that it did not happen. In that same bumped state:
Exit 1 fails the
gatejob, which skipsreleaseand — throughneeds:— every EQL job downstream of it.Static checks
actionlintclean over all nine release workflows (locally 1.7.11; CI pins 1.7.7).node scripts/lint-no-workflow-caching.mjsexit 0 across its seven targets.biome checkclean on every changed file.pnpm run test:scripts— 785 passed, 2 failed, 3 files failed to collect. All three are pre-existing local-environment failures, not regressions:lint-no-eql-registry-pinsandcargo-lock-freshnessboth walk the filesystem and pick up 15 stale git worktrees under.claude/worktrees/, andbench-index-expressionsreads a stale@cipherstash/eql@3.0.4entry in the pnpm store. This branch touches noCargo.lockand no package manifest, and the three fail identically on an unmodifiedmain.Not verified, and it cannot be from here. The docs pipeline (
_build-eql-docs.yml) has never executed in this repository — itsdocs:generate:jsontask previously had no caller at all, which is why it carried a written exemption that this PR deletes. Its doxygen dependency is apt-installed by the job rather than pinned by mise. The first armed release is its first real test; expect to iterate there.Related
Refs #885
Review notes
Start with
scripts/eql-pipeline-armed.mjs— the arming switch is the load-bearing idea and everything else is gated on it. Then.github/workflows/release.yml, whereclassifyand the four prerelease jobs are the largest new surface.workflow_dispatchonrelease.ymlis the one behaviour change to the existing release path worth a second look: it did not have one before, soclassifynow gates the FFI and changesets jobs onmode == 'production'to stop a dispatch against a feature branch publishing from it.No changeset: this is repository tooling with no effect on any published package's surface.
https://claude.ai/code/session_01T26DhFPkLfN3qPFDq2ttKw