feat(cli): bdrive verify proves this folder matches the hub (BEA-161) - #192
Open
ssowonny wants to merge 1 commit into
Open
feat(cli): bdrive verify proves this folder matches the hub (BEA-161)#192ssowonny wants to merge 1 commit into
ssowonny wants to merge 1 commit into
Conversation
"Your folder is the same everywhere" was a belief with no receipt. `bdrive status` counts pending ops and unscanned changes but never reads a byte of content, so a file whose bytes changed while its size and mtime stayed put was invisible to every check we shipped. `bdrive verify` hashes every synced file and compares it against journal.Replay(AllOps), reporting drifted / never-pushed / missing-locally / not-yet-scanned, and with --remote also missing-on-hub. Exit 0 when every category is empty, 1 otherwise, so it composes as a pre-flight check. The logic lives in internal/syncer/verify.go next to Drift and Explain — the two read-only siblings it completes — because neverSync, chunkThreshold and loadFilter are unexported there, and because a multi-device test cannot drive a func main package. cmd/bdrive/verify.go is the thin cobra shell. Two things that would have made it silently wrong: - --remote must probe BOTH blobs/<sha> and manifests/<sha>. Files over 4 MiB are pushed as chunks plus a manifest keyed by the file's own sha, so a check asking only blobs/ would call every large file missing from the hub. Size only orders the probe — it can never be a filter, because browser uploads always write blobs/<sha> at any size, pushChunked falls back to a whole blob when the manifest key is refused, and pre-delta-sync history is whole blobs regardless. - missing-locally applies filter.Skip + neverSync, the same guard materialize uses. The rules are symmetric in scan and materialize, so a path the local filter excludes is legitimately absent — without this, every project narrowed by `bdrive scope --only` would report its whole out-of-scope set as missing. Pure read throughout: no Session, no volume flock, no ops, no journal writes, no materialize, and no network without --remote. LoadProject rather than ResolveMount and remote.Open rather than openSession, so a read never enrolls the device. An unreachable hub degrades to a printed warning and the local verdict still decides. The command says its own caveat out loud: the journals it replays are this device's local copies, so it proves "this folder matches what I last pulled". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
bdrive verifyhashes every synced file and tells you what drifted, what never pushed, and what's missing.bdrive statusstructurally cannot: bytes changed, size and mtime put back.--remoteasks the hub whether it still holds the content — one check per blob, bothblobs/<sha>andmanifests/<sha>, or every file over 4 MiB reads as missing.The hole this fills
Three read-only inspectors already existed. The fourth — the one that answers the question people actually ask — was missing:
Drift()—internal/syncer/drift.goExplain()—internal/syncer/explain.gobdrive statusVerify()— new--remoteWhat it looks like
Clean:
A file edited behind the daemon's back (exit status 1):
Hub gone,
--remote. The remote leg degrades, the local verdict still decides:The two things that would have made it silently wrong
--remotemust probe both key shapes. Files overchunkThreshold(4 MiB) are pushed as content-defined chunks plus a manifest keyed by the file's own sha256 —manifests/<sha>, neverblobs/<sha>. A check asking onlyblobs/calls every large file missing from the hub. And the size can only order the probe, never filter it, because a large file legitimately lives underblobs/three ways: the browser upload path always writesblobs/<sha>at any size,pushChunkedfalls back to a whole-blobPutwhen the manifest key is refused, and anything pushed before delta sync existed is a whole blob regardless. So: probe both, ordered by size, short-circuit on the first hit.TestVerifyRemoteChunkedFileasserts the big file is reachable only undermanifests/, so it can't pass trivially.Checking the manifest alone is enough for a chunked file — "a manifest exists ⟹ its chunks exist" is enforced hub-side at ingest, not an honest-client convention — so there's no chunk-by-chunk walk.
missing-locallyapplies the local filter..bdriveignorerules are symmetric in scan and materialize, so a path the local filter excludes is legitimately absent from disk. Withoutfilter.Skip(rel) || neverSync(rel)— the same guardmaterializeuses — every project narrowed bybdrive scope --onlywould scream its entire out-of-scope set as missing.TestVerifyIgnoredPathNotMissingcarries a negative control: remove the rule and the very same folder does report it.Deviations from the reviewed plan
None of substance. The plan's optional
HasBlobhint was taken (it costs one field and one line): whenmissing-locallyis really just "not downloaded yet", the output says so and points atbdrive syncinstead of reading as damage.Shape
internal/syncer/verify.goholds the comparison, next toDriftandExplain;cmd/bdrive/verify.gois the thin cobra shell. That split is not cosmetic —neverSync,chunkThresholdandloadFilterare unexported insyncer, and the multi-device test the spec requires cannot drive afunc mainpackage.The pure-read contract is copied from
drift.goverbatim and holds: noSession, nostore.Lock(), noSaveSync/SaveCache, noAppendOps, no materialize, and no network at all without--remote. The shell usesconfig.LoadProjectrather thanmustProjectandremote.Openrather thanopenSession, because both of those reachResolveMount, which enrolls the device — a read must not do that.store.Openis stat-guarded so a read never creates a volume for a project that has never synced.Exit status uses the same mechanism
bdrive grepuses for its no-match exit (errVerifyProblems+SilenceErrors/SilenceUsage), so a findings exit is status 1 with the findings and no cobra usage block. Every path printed frommissing-locallyandmissing-on-hubcomes out of a peer's journal, so it goes throughsafeFieldlike every other journal string we print.Architecture changes
architecture/cli-sync.md: two new classes ininternal/syncer—VerifyandVerifyReport— joiningDrift/Explain/SyncedFilesas read-only inspectors, with new edges toSyncedFiles(the walk),Filter(its own fresh instance, formissing-locally),Store(AllOps/DeviceOps/LoadSync/HasBlob) andBackend(Existsper blob,--remoteonly).verifyjoins theCommandsroster. Nothing was removed.✅ added · ❌ removed (strikethrough) · unmarked = unchanged
flowchart TB Verify["<div style='text-align:left'><b>Verify</b><br/>+Verify(ctx, folder, include, st, device, be) VerifyReport<br/>-existsEither(ctx, be, blob, size) bool</div>"] VerifyReport["<div style='text-align:left'><b>VerifyReport</b><br/>+Files int / +Bytes int64 / +Elapsed Duration<br/>+Drifted / NeverPushed / MissingLocally paths<br/>+NotYetScanned / MissingOnHub paths<br/>+NotFetched int / +RemoteErr error<br/>+Problems() int</div>"] Drift["<div style='text-align:left'><b>Drift</b><br/>+Drift(folder, include, accepted, cache) added, modified, removed</div>"] Explain["<div style='text-align:left'><b>Explain</b><br/>+Explain(folder, include, accepted) two lists<br/>+NotSyncedFiles(entries) int</div>"] SyncedFiles["<div style='text-align:left'><b>SyncedFiles</b><br/>+SyncedFiles(folder, include, accepted) paths</div>"] Filter["Filter"] Store["<div style='text-align:left'><b>Store</b><br/>+AppendOps / DeviceOps / AllOps<br/>+LoadSync / SaveSync<br/>+PutBlob / OpenBlob / HasBlob</div>"] Backend["Backend"] Commands["<div style='text-align:left'><b>Commands</b><br/>init login logout<br/>sync stop scope grep stale <span style='background:#22c55e55;padding:0 4px;border-radius:3px'>✅ verify</span> forget status log<br/>restore url share export import<br/>web daemon hooks read-log<br/>resume autostart</div>"] Note["verify.go — the third read-only sibling,<br/>and the only one that HASHES.<br/>Same pure-read contract as Drift/Explain:<br/>no Session, no flock, no ops, no journal write,<br/>no network unless a Backend is passed.<br/>--remote probes BOTH blobs/<sha> and manifests/<sha>;<br/>size only orders the probe.<br/>RemoteErr stops the remote leg, never the local verdict."] Verify -- "<span style='background:#22c55e55;padding:0 5px;border-radius:3px'>✅ hashes what syncs</span>" --> SyncedFiles Verify -- "<span style='background:#22c55e55;padding:0 5px;border-radius:3px'>✅ own fresh instance, for missing-locally</span>" --> Filter Verify -- "<span style='background:#22c55e55;padding:0 5px;border-radius:3px'>✅ AllOps / DeviceOps / LoadSync / HasBlob</span>" --> Store Verify -- "<span style='background:#22c55e55;padding:0 5px;border-radius:3px'>✅ Exists per blob, --remote only</span>" --> Backend Verify -. "<span style='background:#22c55e55;padding:0 5px;border-radius:3px'>✅ findings and exit status</span>" .-> VerifyReport Commands -- "<span style='background:#22c55e55;padding:0 5px;border-radius:3px'>✅ bdrive verify</span>" --> Verify Drift --> Filter Explain --> Filter SyncedFiles --> Filter Verify -.- Note classDef added fill:#22c55e22,stroke:#22c55e,stroke-width:2px classDef removed fill:#ef444422,stroke:#ef4444,stroke-width:2px,stroke-dasharray:4 3 classDef noteBox fill:#88888822,stroke:#888888,stroke-dasharray:2 2 class Verify added class VerifyReport added class Note noteBox linkStyle 0 stroke:#22c55e,stroke-width:2px linkStyle 1 stroke:#22c55e,stroke-width:2px linkStyle 2 stroke:#22c55e,stroke-width:2px linkStyle 3 stroke:#22c55e,stroke-width:2px linkStyle 4 stroke:#22c55e,stroke-width:2px linkStyle 5 stroke:#22c55e,stroke-width:2pxAcceptance
go build ./...go vet ./...go test ./...internal/syncer/verify_test.gointernal/webappCLI e2eTestCLIVerifyE2E, passnpm run e2eThe multi-device tests drive the real converge/diverge paths through
newDevice/sharedRemote/cycle: clean after converge; a restored-mtime edit reported as drifted whileDrift()on the same folder reports zero; a deleted file as missing-locally; an ignored path not reported; an offline cycle's ops as never-pushed; a fresh file as not-yet-scanned; a >4 MiB file present on the hub undermanifests/; a blob removed from the remote as missing-on-hub.Docs
README.mdcommand table andweb/docs/src/content/docs/reference/cli.md(table row plus a### bdrive verifysection covering the five states, the both-key-shapes rule and the staleness caveat). NoINSTALL_FOR_AGENTS.mdchange — this touches neither init, login, nor hooks.Deliberately not here
No hub UI (no device panel, no sync-state page), no
--pullthat refreshes journals before comparing, no JSON output, no repair, no--fast, no chunk-level walk. The CLI is the whole win in v1.The open one:
--remoteproves the hub still holds everything this device knows about — it cannot prove the hub holds nothing extra. Whether that second half is worth a--pullflag depends on whether anyone actually hits it.Closes BEA-161.
Build session
(only works on this machine)