Skip to content

gitproxy: serve git fetches from cache-server on an instance - #165

Merged
linyiru merged 11 commits into
mainfrom
feat/git-proxy
Nov 5, 2026
Merged

linyiru merged 11 commits into
mainfrom
feat/git-proxy

Conversation

@linyiru

@linyiru linyiru commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Second part of the git checkout cache. Adds firerunner git-proxy. Nothing starts it yet: the tier option, the CA for github.com and the AMI come next.

What it does

A loopback proxy that git alone is pointed at, through http.https://github.com/.proxy. It carries CONNECT github.com:443 and refuses anything else, terminates TLS, and handles three kinds of request:

  • Ref listing: forwarded. A 200 records that GitHub accepted these credentials for this repository.
  • A fetch gitcache accepts: answered from the cache when it can be, otherwise forwarded, and the response stored.
  • Everything else, including pushes: forwarded unchanged.

It uses the same Twirp API as actions/cache, through the instance's nginx, with git/<hash> as the key. Isolation is cache-server's, and tests run against the real server: another repository cannot read an entry, and the default branch cannot read a pull request's.

Failure

The rule is that the cache must never fail a checkout GitHub would have served, and not every client retries: actions/checkout does, a workflow's own git fetch does not. So an entry is downloaded to a temporary file and checked before its first byte goes out. That costs git the chance to start early, and it means any failure until then goes to GitHub.

failure behaviour
lookup fails or is stuck forwarded to GitHub
entry missing, stuck, truncated, damaged or over the limit forwarded to GitHub; the client never sees it
entry is whole and valid but for the wrong commit git refuses it, and the retry goes to GitHub, as a key is served once per proxy
GitHub breaks part way the client sees the break; nothing is stored
ERR line, sideband 3, truncated, bad checksum or non-pack response the client gets what GitHub sent; nothing is stored
over the size limit, no temporary file, store fails the client is unaffected; nothing is stored

GitHub answers 200 before it knows how a pack will go, so responseCheck reads the stream, sideband by sideband, and verifies the pack's own checksum. The fourth case is the only one a client can see, and only a job writing to its own ref can cause it.

A hit needs credentials GitHub just accepted

A hit skips GitHub's own check, so the same credentials must have listed refs for the same repository within 10 minutes. git always lists refs first.

This is correctness for an honest job, not a boundary. A job can read the leaf key and, with sudo, replace the process. What a job cannot do is widen what cache-server gives it.

Known limit: the proxy cannot tell a job's own repository from another it has access to. A fetch of private repository B from a job in repository A is stored under A. Reaching it from a later job in A needs the commit's ID, and it is the same exposure as that job saving B to actions/cache. I would rather document this than add a config input.

Verified

  • Real git through the real proxy over TLS, to git http-backend: miss then hit with no upstream fetch; a damaged pack and a cache that is down being invisible; another commit's valid response refused by git, with the retry recovering.
  • Fault injection for every row above, under -race.
  • scripts/mutate.sh on about 35 guards. Surviving mutants led to tests for the host check, the content type, the request target and a pack's magic, and to removing two redundant conditions.
  • An independent review found the design above was needed, ran the proxy against real github.com, and reported that five kinds of fetch stored and replayed correctly. That was before the pack checksum existed, so it is not evidence for the checksum on real packs; git http-backend is.

Not verified: EC2, a private repository with a real token, and nginx in the path. How long the first checkout waits for the job's identity also needs measuring: the 15 s lookup limit is a judgement, and if identity takes longer, a first step waits 15 s and still misses, which decides whether this pays.

For the wiring PR: the temporary directory shares the job's disk, and a dead proxy fails git, as git has no fallback.

firerunner git-proxy is the loopback proxy git is pointed at, through
http.https://github.com/.proxy. It answers a fetch gitcache accepts from
cache-server, through the same API as actions/cache, so a git entry is
scoped, evicted and stored as any other, and stores what it had to fetch.

The cache must never fail a checkout GitHub would have served. A cache
error falls through to GitHub. A hit that breaks part way aborts the
connection, so git retries, and a key is served from the cache once per
proxy, which also gets a job past a poisoned entry. A response is stored
only if it is whole, as GitHub reports a failure inside a 200.

A hit skips GitHub's own check, so it needs GitHub to have accepted the
same credentials for the same repository moments earlier.

Nothing starts this yet. It needs a tier option, a CA for github.com and
the AMI, which come next.

Signed-off-by: Yi-Ru Lin <deduce@gmail.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

An opaque request target can bypass the GitHub-only restriction, causing the proxy to forward Authorization headers to an unintended host.

3 open issues
What changed in this PR

Adds firerunner git-proxy, the instance-side component of the git checkout cache. Automatic startup, certificate provisioning, and AMI integration remain deferred.

Changes:

  • Adds a loopback TLS proxy that caches eligible fetches and forwards other git traffic.
  • Integrates with cache-server and checks response framing before storing entries.
  • Adds fault-injection, repository-isolation, CLI-validation, and real-git integration tests.
File Description
internal/​gitproxy/​proxy.go Implements proxying, credential gating, cache hits, and background saves.
internal/​gitproxy/​proxy_test.go Tests forwarding, cache failures, retries, and response validation.
internal/​gitproxy/​git_test.go Exercises real git fetches through the TLS proxy.
internal/​gitproxy/​check.go Checks streamed fetch responses for complete framing and errors.
internal/​gitproxy/​cache.go Implements cache-server lookup, download, and upload operations.
internal/​gitproxy/​cache_test.go Tests cache-server integration and repository/ref isolation.
cmd/​firerunner/​main.go Registers and documents the new subcommand.
cmd/​firerunner/​gitproxy.go Configures certificates, loopback listening, and shutdown.
cmd/​firerunner/​gitproxy_test.go Tests loopback restrictions and invalid CLI arguments.

Review effort: Balanced

Comment thread internal/gitproxy/proxy.go Resolved
Comment thread internal/gitproxy/proxy.go Resolved
Comment thread internal/gitproxy/proxy_test.go Resolved
A request for https:@example.com/path has the RequestURI @example.com/path,
so pasting it after the host sent the job's Authorization header to
example.com. Build the URL from a fixed scheme and host, and refuse a
target that is not a plain path.

Signed-off-by: Yi-Ru Lin <deduce@gmail.com>
An entry that fails on its first read has sent nothing, so aborting
failed a fetch GitHub could still have served.

Signed-off-by: Yi-Ru Lin <deduce@gmail.com>
The store's goroutine removes its file after Save returns, so the test
could look before it did.

Signed-off-by: Yi-Ru Lin <deduce@gmail.com>
…aborts

Signed-off-by: Yi-Ru Lin <deduce@gmail.com>
Aborting a broken hit relied on the client retrying. actions/checkout
does; a workflow's own git fetch does not, so a cache fault failed a step
GitHub would have served. Download the entry and check it first, so any
failure goes to GitHub. That also bounds a stream that stalls, and closes
a race between the lookup's timer and the first read.

The check now verifies the pack's checksum, which catches an entry
damaged in store or in transit, and refuses a packfile section with no
pack, an ERR line inside one and an unknown sideband.

Signed-off-by: Yi-Ru Lin <deduce@gmail.com>
git matches its proxy setting that way, so a URL written GitHub.com was
sent here and refused, failing a fetch that works without the proxy.

Signed-off-by: Yi-Ru Lin <deduce@gmail.com>
A 502 said nothing about its cause, and a failed blob request logged a
URL holding its signature.

Signed-off-by: Yi-Ru Lin <deduce@gmail.com>
…used

Signed-off-by: Yi-Ru Lin <deduce@gmail.com>
Two tests could pass without observing what they claim, one of them
flaky, and three comments still described the hit path before it spooled.

Signed-off-by: Yi-Ru Lin <deduce@gmail.com>
@linyiru

linyiru commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

babysit-pr · head b99c7ef
Fixed: #165 (comment) — URL built from parts, 4b8b994
Fixed: #165 (comment) — entries checked before the first byte, 09e1956
Fixed: #165 (comment) — tests wait for cleanup, 0047671
Local review (subagent): 8ef50dd..57217c1 — adopted 3: a flaky temporary file check, a test that could pass unobserved, stale comments; b99c7ef is that fix
CI: running on this head

@linyiru
linyiru merged commit d9f0567 into main Nov 5, 2026
7 checks passed
@linyiru
linyiru deleted the feat/git-proxy branch November 5, 2026 21:10
@linyiru

linyiru commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

babysit-pr · head b99c7ef
Local review (subagent): 57217c1..b99c7ef — none
CI: green

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants