Skip to content

db/snaptype: tolerate files deleted during dir scan - #22853

Queued
AskAlexSharov wants to merge 3 commits into
mainfrom
alex/parsedir_skip_vanished_37
Queued

db/snaptype: tolerate files deleted during dir scan#22853
AskAlexSharov wants to merge 3 commits into
mainfrom
alex/parsedir_skip_vanished_37

Conversation

@AskAlexSharov

@AskAlexSharov AskAlexSharov commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

ParseDir fails the whole directory scan when a file is deleted between ReadDir and the per-entry DirEntry.Info stat. Merged-over segments are unlinked concurrently with directory scans, so OpenFolder after retire can abort:

[EROR] [snapshots] retire blocks err="open: OpenFolder: lstat .../snapshots/v1.1-047370-047371-transactions.seg: no such file or directory"

Skip entries whose stat returns ErrNotExist (OpenFolder already treats vanished files as retired), and wrap remaining stat errors with the directory for context.

Seen aborting the QA sync-from-scratch (gnosis archive) run on release/3.6: https://github.com/erigontech/erigon/actions/runs/30347797880/job/90237994145

Merged-over segments are unlinked concurrently with directory scans,
so DirEntry.Info can hit a vanished file between ReadDir and lstat.
ParseDir now skips such entries instead of failing the whole
OpenFolder, and wraps remaining stat errors with the dir for context.

Seen as QA sync-from-scratch abort on gnosis:
[snapshots] retire blocks err="open: OpenFolder: lstat .../v1.1-047370-047371-transactions.seg: no such file or directory"

Claude-Session: https://claude.ai/code/session_01UYCsHj9HYTJnqUCy8a7W91

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens snapshot directory scanning in db/snaptype against concurrent file deletion during ReadDirDirEntry.Info() stat, which can happen when merged-over snapshot segments are unlinked while another goroutine is scanning the directory. It makes ParseDir skip entries that vanish mid-scan instead of failing the whole scan, and adds a targeted unit test for the race.

Changes:

  • Refactor ParseDir to delegate entry processing into a helper (parseDirEntries) for testability.
  • Skip entries whose DirEntry.Info() fails with fs.ErrNotExist, tolerating concurrent unlink.
  • Wrap remaining stat errors with the directory path for better diagnostics, and add a test covering the “vanished entry” case.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
db/snaptype/files.go Skip ErrNotExist from per-entry stat during scans; wrap other stat errors with directory context; extract parseDirEntries helper.
db/snaptype/files_test.go Add unit test ensuring a vanished entry during scan is skipped (and does not abort the scan).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@AskAlexSharov
AskAlexSharov added this pull request to the merge queue Jul 29, 2026
Any commits made after this event will not be merged.
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 29, 2026
@AskAlexSharov
AskAlexSharov added this pull request to the merge queue Jul 29, 2026
Any commits made after this event will not be merged.
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 29, 2026
@AskAlexSharov
AskAlexSharov enabled auto-merge July 30, 2026 02:43
@AskAlexSharov
AskAlexSharov added this pull request to the merge queue Jul 30, 2026
Any commits made after this event will not be merged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants