Skip to content

fix: verify all image candidates before release promotion - #237

Merged
DavidHLP merged 2 commits into
mainfrom
fix/dav65-release-gates
Oct 5, 2026
Merged

DavidHLP merged 2 commits into
mainfrom
fix/dav65-release-gates

Conversation

@DavidHLP

@DavidHLP DavidHLP commented Oct 5, 2026 •

Copy link
Copy Markdown
Owner

Problem

Seven backend candidates contain Jackson 2.21.4 with five fixed HIGH vulnerabilities. Existing publication assigns release tags before scans and signed evidence finish.

Changes

  • Upgrade Jackson BOM to 2.21.7; align root and seven authoritative owner release versions at 1.0.1. Preserve existing v1.0.0 tags.
  • Docker Verify loads and scans all nine local images with pinned Trivy 0.74.0; no registry push and no weakened HIGH/CRITICAL threshold. Verify and Publish bypass only runtime-stage cache so Alpine security upgrades execute again, retaining builder caches.
  • Publish digest-only candidates. After all nine builds, scans, signatures and SPDX/SLSA attestations succeed, validate complete evidence and verified DSSE subjects/source/predicates before release-tag promotion.
  • Restrict writes to the serialized Docker Publish-on-main workflow. Reject old-tag collisions, preflight output conflicts, read back all tags and emit one complete release manifest. No registry CAS or protection from external registry writers is claimed.
  • Preserve immutable deployment/rollback consumer policy; document recovery using the failed promote job and original candidate artifacts.

Verification

  • Passed: actual Maven dependency tree, core/databind and Jackson modules 2.21.7, annotations 2.21.
  • Passed: supply-chain contract and final promoter regressions, including real Cosign SLSA field-dropping behavior, rejected signature verification, wrong source/subject/predicate, malformed reports, missing artifacts, authorization/network errors, collision, no-write preflight failures, all-tag readback, dry-run, idempotency and partial failure.
  • Independent static review: all identified findings fixed; no remaining blocking findings.
  • Passed: fixed-input local Java unit reactor (./mvnw -B -Punit test), 3,271 tests, zero failures/errors, 16 skipped. First attempt failed on mismatched owner release versions; the final rerun includes the fix.
  • First local pass had an auth scanner timeout and the pre-fix App dependency mismatch. CI run 37355576077 removed all five Jackson findings but blocked on cached libexpat 2.8.4-r0 (all nine images) and pcre2 10.48-r0 (frontends). Actual build logs confirm the upgrade layer was cached; commit 21b1865 fixes the common cause without suppressions.
  • Passed at current head 21b1865: all nine local no-push builds/scans with fixed POM hashes, uncached runtime stages, one newly downloaded private Trivy DB, zero blocking vulnerabilities and matching image IDs. Read-only image inventories confirm libexpat 2.8.5-r0 in all nine images and pcre2 10.49-r0 in both frontends. CI run 37358023872 passed (23 success, 1 skipped), including all nine no-push image gates; downloaded raw reports match checkout merge SHA 43ffcda and contain zero blocking findings.
  • Graph update attempted (AST-only); timed out after extraction. Not a completed index refresh.

Delivery

Merged at bc50511. Main CI 37361248796 and Docker Publish 37361248376 succeeded. All nine candidates passed real scans, signatures and SBOM/provenance verification before complete release promotion. Downloaded release evidence/source hashes verified; registry readback confirms nine sha-bc50511 and seven v1.0.1 tags match candidate digests, while sixteen historical sha-c3194bc/v1.0.0 digests remain unchanged. No production deployment.

Refs DAV-65

Upgrade Jackson to 2.21.7 and align owner releases at 1.0.1. Preserve HIGH/CRITICAL scans and existing immutable deployment policy.

Publish digest-only candidates, verify all nine signed evidence sets, then promote tags through one serialized workflow. Reject historical tag collisions and incomplete release manifests.
CI image scans exposed fixed libexpat and pcre2 HIGH CVEs in cached Alpine upgrade layers. Bypass only final-stage cache in Verify and Publish so apk upgrade executes again; retain builder caches and scan thresholds.
@DavidHLP
DavidHLP marked this pull request as ready for review October 5, 2026 19:04
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-05T19:14:29.107461Z 21b1865 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@DavidHLP
DavidHLP merged commit bc50511 into main Oct 5, 2026
24 checks passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 21b1865626

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +285 to +287
tags = [f"sha-{source_sha[:7]}"]
if candidate["version"]:
tags.append(f"v{candidate['version']}")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Do not retag unchanged service versions

On the next main push where any Maven service retains its current service.version.*, this always plans the same v<version> tag. The workflow passes metadata-action labels into every build, and those labels include commit-specific org.opencontainers.image.revision and time-specific org.opencontainers.image.created values (metadata-action documentation), so even an otherwise unchanged service normally receives a new digest. The check at lines 302–304 then rejects the existing version tag because it points to the previous digest. Since all seven versioned services are built on every run, every later release would require bumping all seven versions rather than only the changed service; skip promotion of an already-owned unchanged version tag or otherwise prevent unchanged services from blocking the complete release.

Useful? React with 👍 / 👎.

ISSUER = "https://token.actions.githubusercontent.com"
DIGEST = re.compile(r"^sha256:[0-9a-f]{64}$")
SHA = re.compile(r"^[0-9a-f]{40}$")
VERSION = re.compile(r"^[0-9]+\.[0-9]+\.[0-9]+(?:[-+][0-9A-Za-z.-]+)?$")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Sanitize SemVer build metadata before creating Docker tags

When a service uses a valid SemVer value with build metadata, such as 1.0.2+hotfix, this regex accepts it and the promoter later constructs v1.0.2+hotfix verbatim. Docker tag references only allow the pattern [\w][\w.-]{0,127}, so buildx imagetools inspect fails with an invalid reference before any promotion (container reference grammar). Either reject + versions at this boundary or sanitize the version into a valid, collision-safe Docker tag as metadata-action previously did.

Useful? React with 👍 / 👎.

docker = os.environ.get("DOCKER_BIN", "docker")
planned = []
for candidate in candidates:
tags = [f"sha-{source_sha[:7]}"]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use the full commit SHA for immutable release tags

When two main commits share the same first seven hexadecimal characters, both releases plan the identical sha-xxxxxxx tag. The first release permanently assigns that tag, and the second then hits the different-digest rejection at lines 302–304, blocking the complete release even though its full source SHA is distinct and valid. Because these tags are now deliberately immutable, derive the tag from the full SHA (or another collision-resistant identifier) rather than a seven-character prefix.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant