Skip to content

ci(eql): port the EQL release pipeline to the repository root, inert - #941

Draft
tobyhede wants to merge 2 commits into
mainfrom
toby/cip-3742-eql-release-pipeline
Draft

ci(eql): port the EQL release pipeline to the repository root, inert#941
tobyhede wants to merge 2 commits into
mainfrom
toby/cip-3742-eql-release-pipeline

Conversation

@tobyhede

@tobyhede tobyhede commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Summary

EQL is the SQL layer that stores and queries encrypted data in Postgres. It was imported into this repository as a subtree, and it brought its whole release pipeline with it — into packages/eql/.github/, a directory GitHub never reads. Eleven files, seven of them workflows, executing on nothing. That pipeline is what publishes @cipherstash/eql and the four artefacts that must ship at the same version as it: the eql-bindings Rust crate, the SQL bundle customers install, the docs bundle, and the postgres-eql Docker image.

This moves all of it to the repository root, where GitHub will run it, and keeps it inert — nothing reaches npm, crates.io or GHCR. @cipherstash/eql is still published from cipherstash/encrypt-query-language, and until that is repointed, publishing from here would either fail or ship something nobody can install.

Inertness is a derived switch, not a flag somebody flips. scripts/eql-pipeline-armed.mjs reads FROZEN_PUBLISHERS in scripts/release-gate.mjs — the existing map that records "this package lives here but is published elsewhere" — and every job that publishes an EQL artefact is gated on its answer. The cutover has to delete that entry, because the release gate blocks every release until it does. So deleting it is what arms the pipeline, and there is no second thing to remember. A forgotten flag would have failed silently in the worst direction: an npm package published with no SQL release, no docs and no crate.

Changes

Workflows ported out of packages/eql/.github/ (the directory is now deleted)

was now
_build-sql.yml .github/workflows/_build-eql-sql.yml
_build-docs.yml .github/workflows/_build-eql-docs.yml
release.yml merged into .github/workflows/release.yml — npm trusted publishing binds a package to a repository and a workflow filename, so every npm publish here has to live in that one file
release-plz.yml same filename at the root. It is the one that could not be renamed: crates.io binds to it the same way
release-postgres-eql-image.yml same filename at the root
rebuild-docs.yml merged into the root file of that name (two tag families, one webhook)
lint-release.yml merged into the root file of that name
actionlint.yaml, ISSUE_TEMPLATE/docs-feedback.yml dropped — the root already carries the same content
.github/release.yml dropped — GitHub's release-notes categoriser, which nothing here uses
workflows/README.md dropped — it documented a merge-queue model this repository does not have (main is unprotected and there is no queue). Its still-true half is in AGENTS.md

Each drop is recorded where the shrinking allowlist used to live, in scripts/__tests__/eql-suite-ci.test.mjs, which now asserts the directory does not exist and that nothing is tracked under it.

Paths that had to change, because the subtree root is not the package root here

  • The npm manifest is two levels down at packages/eql/packages/eql. Upstream read ./packages/eql/package.json; here that resolves to a directory with no manifest and the read throws.
  • mise reads config from the current directory and its parents only, so every mise step carries working_directory: packages/eql.
  • release-plz/action needs explicit manifest_path and config inputs. A composite action's steps do not inherit the caller's defaults.run.working-directory, so a working-directory: on the step would have done nothing — release-plz would have read the monorepo root, found no workspace naming eql-bindings, and exited 0 having published nothing.

Supply chain

  • Every remote uses: in the four new workflow files and in the jobs added to release.yml is SHA-pinned with a trailing version comment; upstream used floating tags throughout. The two merged files (lint-release.yml, rebuild-docs.yml) keep the pins the root copies already had.
  • Upstream set cache: true on three jdx/mise-action steps in publish-adjacent jobs. Now false everywhere, and all four ported release workflows are in scripts/lint-no-workflow-caching.mjs's TARGETS.
  • AUDITED_ACTIONS gains seven entries. Two of the three Docker actions turned out to have a GitHub-Actions-cache input defaulting to true (docker/setup-qemu-action's cache-image, docker/setup-buildx-action's cache-binary). docker/build-push-action has no such toggle, so it gets a new forbiddenInputs rule on cache-from / cache-to — those name a backend, and type=gha is the GitHub Actions cache.
  • release-plz.yml lost its mise step entirely. No step in that job runs a mise task, and the publishing-job rule (add_shims_to_path: false) would have left mise's toolchain off PATH anyway — so it was a third-party action inside the crates.io-credential job, installing tools nothing could reach.

Guards that had to grow rather than be exempted

  • workflow-publish-permissions.test.mjsOIDC_JOBS was doing double duty as "who may publish?" and "who may write to the repository?". Six new jobs need a writable scope (create a release, dispatch a workflow, move a branch ref) and must not be able to publish. Split into OIDC_JOBS and REPO_WRITE_JOBS, both equalities, with a third assertion that the first stays a subset of the second.
  • workflow-dispatch-job-conditions.test.mjs — it required every job in a dispatchable workflow to run on a manual dispatch. release.yml now has two mutually exclusive paths (production on main, prerelease on a marker commit elsewhere), so no single synthetic context can run both. The per-job requirement is kept; the exceptions become an equality-checked list with reasons, so a job that quietly stops running on dispatch still fails.
  • eql-suite-ci.test.mjs — its rule "every job that compiles Rust restores the shared cache" collides head-on with "a publishing workflow restores no cache". Supply chain wins and four EQL release jobs pay a cold compile. The exemption is derived from the caching linter's own target list rather than copied, so the two cannot disagree about which jobs those are.

Two real defects the guards found in the port, both because upstream's repository had neither dependency: the prerelease publish job was missing npm install -g node-gyp before pnpm install (node-pty's install hook needs it on Linux), and it built with pnpm --filter … build instead of pnpm exec turbo run build (so ^build dependencies were whatever an earlier step happened to leave behind).

DocsAGENTS.md, SECURITY.md and the absorption plan updated: what the pipeline is, what arms it, and the two things the cutover still needs that no workflow can assert ahead of time (a GPG_PRIVATE_KEY secret, and write access from this repository to the ghcr.io/cipherstash/postgres-eql package, which is currently linked to the old repository).

Verification

The dry run. Added a temporary @cipherstash/eql patch changeset, ran pnpm run version, confirmed the lockstep hook moved every artefact together, then reverted the tree to clean:

npm package  3.0.5 -> 3.0.6
eql-bindings Cargo.toml  3.0.5 -> 3.0.6
SQL bundle stamp (npm package copy)   COMMENT ON SCHEMA eql_v3 IS '3.0.6'
SQL bundle stamp (crate copy)         COMMENT ON SCHEMA eql_v3 IS '3.0.6'
both sql/release-manifest.json        eqlVersion 3.0.6
src/generated/release-manifest.ts     eqlVersion 3.0.6

Nothing publishes, proven by the gate rather than by observing that it did not happen. In that same bumped state:

$ node scripts/release-gate.mjs
  @cipherstash/eql@3.0.6 is not on npm, and cannot be published from this repository.
  stash depends on @cipherstash/eql@3.0.6 [dependencies], which no published version satisfies.
  @cipherstash/stack-prisma depends on @cipherstash/eql@3.0.6 [dependencies], which no published version satisfies.
exit 1

Exit 1 fails the gate job, which skips release and — through needs: — every EQL job downstream of it.

$ node scripts/eql-pipeline-armed.mjs
@cipherstash/eql is a frozen publisher — the EQL release pipeline is INERT.

Static checks

  • actionlint clean over all nine release workflows (locally 1.7.11; CI pins 1.7.7).
  • node scripts/lint-no-workflow-caching.mjs exit 0 across its seven targets.
  • biome check clean on every changed file.
  • pnpm run test:scripts — 785 passed, 2 failed, 3 files failed to collect. All three are pre-existing local-environment failures, not regressions: lint-no-eql-registry-pins and cargo-lock-freshness both walk the filesystem and pick up 15 stale git worktrees under .claude/worktrees/, and bench-index-expressions reads a stale @cipherstash/eql@3.0.4 entry in the pnpm store. This branch touches no Cargo.lock and no package manifest, and the three fail identically on an unmodified main.

Not verified, and it cannot be from here. The docs pipeline (_build-eql-docs.yml) has never executed in this repository — its docs:generate:json task previously had no caller at all, which is why it carried a written exemption that this PR deletes. Its doxygen dependency is apt-installed by the job rather than pinned by mise. The first armed release is its first real test; expect to iterate there.

Related

Refs #885

Review notes

Start with scripts/eql-pipeline-armed.mjs — the arming switch is the load-bearing idea and everything else is gated on it. Then .github/workflows/release.yml, where classify and the four prerelease jobs are the largest new surface.

workflow_dispatch on release.yml is the one behaviour change to the existing release path worth a second look: it did not have one before, so classify now gates the FFI and changesets jobs on mode == 'production' to stop a dispatch against a feature branch publishing from it.

No changeset: this is repository tooling with no effect on any published package's surface.

https://claude.ai/code/session_01T26DhFPkLfN3qPFDq2ttKw

The EQL subtree arrived with eleven files under `packages/eql/.github/`, a
directory GitHub never reads. Seven of them were workflows, and between them
they are the entire publishing pipeline for `@cipherstash/eql`: the npm
package, the `eql-bindings` crate, the SQL bundle, the docs bundle and the
`postgres-eql` image, all of which ship at one version. None of it executed.

This ports the lot and keeps it inert, so it can be reviewed and dry-run
before trusted publishing is repointed at this repository.

Inertness is a derived switch rather than a flag. `scripts/eql-pipeline-armed.mjs`
reads `FROZEN_PUBLISHERS` in `scripts/release-gate.mjs` — the map that already
records "this package lives here and is published elsewhere" — and every job
that publishes an EQL artefact is gated on its answer. The cutover has to delete
that entry (the release gate refuses every release until it does), so deleting
it is what arms the pipeline; there is no second flag to forget, and forgetting
one would fail silently in the direction that publishes an npm package with no
SQL release, no docs and no crate.

Three paths had to be rewritten because the subtree root is not the package
root here: the npm manifest is two levels down at `packages/eql/packages/eql`,
mise config is only found from `packages/eql`, and `release-plz/action` needs
explicit `manifest_path` / `config` inputs because a composite action's steps
do not inherit the caller's `defaults.run.working-directory`.

Three of this repo's guards had to grow rather than be exempted, and two real
defects came out of them: the prerelease publish job was missing `node-gyp`
before its install, and built with `pnpm --filter` rather than through turbo.

Claude-Session: https://claude.ai/code/session_01T26DhFPkLfN3qPFDq2ttKw
@changeset-bot

changeset-bot Bot commented Aug 24, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: ab4c4f3

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Seven findings from the review of the ported pipeline, all confirmed before
acting. Three were guards that could not fire, which is the failure mode this
repository cares most about — a check that reads as protection and is incapable
of reporting anything.

`rebuild-docs.yml` was the worst of them, and worse than reported: it has never
run, not once, against `@cipherstash/stack@1.0.0`, `1.1.0` and `1.1.1`. A ref
created with GITHUB_TOKEN starts no workflow run, and changesets creates those
tags with that token — the same rule this pipeline already works around for the
image build. Porting EQL's `eql-*` tag trigger into it would have added a second
dead trigger. The EQL docs rebuild is now a job in `release.yml` that fires the
webhook from inside the publishing run; the `@cipherstash/stack@*` half is left
alone as a pre-existing bug, recorded in that file's header so the next person
does not repeat the mistake.

`release-plz.yml`'s "refusing to publish the DEV placeholder" guard grepped for
`eql_v3`, which appears 23,723 times in every build of that bundle including a
DEV one. It now matches the schema version stamp against the crate version —
the only line in the bundle that records which build produced it.

`classify` accepted any version containing a hyphen where
`prepare-bindings-assets.sh` requires `X.Y.Z-(alpha|beta|rc).N`. Under the loose
check, `3.0.6-beta` got a public tag and GitHub release before the npm job died
on it.

Also: `PRE_GA_LATEST` was still `true` in the EQL publish script, so the first
prerelease cut through the new job would have moved the `latest` dist-tag off
the GA release. The file's own comment said to flip it once 3.0.0 GA shipped;
npm's `latest` is 3.0.5.

Two documentation defects of my own (a paragraph duplicated into AGENTS.md, and
a SECURITY.md sentence contradicting the permissions split this PR introduces),
one stale plan checkbox, an over-deep checkout, and a module-scope `execFileSync`
whose failure took 40 unrelated assertions with it.

Comment volume across everything this branch added is cut by roughly half.

Claude-Session: https://claude.ai/code/session_01T26DhFPkLfN3qPFDq2ttKw
@tobyhede
tobyhede force-pushed the toby/cip-3742-eql-release-pipeline branch from 463f8b4 to ab4c4f3 Compare August 24, 2026 04:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant