Handle stale filesystem metadata when creating output directories

[dusdnKR]

Summary

This pull request makes output directory handling tolerant of filesystems that report directory metadata with a delay, where os.path.isdir() can transiently return False for a directory that does exist.

With the current if not os.path.isdir(x): os.makedirs(x) pattern, this can result in FileExistsError. The same stale directory view can later cause the to_csv() calls to fail with pandas' Cannot save file into a non-existent directory, terminating the entire run.

This addresses and closes #107.

Changes

• Adds ensureDir() to fsqcUtils.py, which retries os.makedirs(path, exist_ok=True) a few times before giving up. exist_ok=True alone is not sufficient here because CPython can re-raise FileExistsError when its own isdir() check also sees a stale False.
• Replaces the directory-creation logic at all 18 sites in fsqcMain.py and the additional site in checkMotion.py.
• Calls ensureDir() immediately before the two to_csv() writes where the affected runs were observed to fail.
• Adds fsqc/utils/tests/test_ensure_dir.py with regression tests for this behavior.

Only FileExistsError is retried — other errors (e.g. PermissionError) propagate immediately — and a genuine FileExistsError (a non-directory file blocking the path) is still raised once the retries are exhausted, without removing that file.

Notes

ensureDir() creates missing parent directories, whereas eight of the replaced sites previously used os.mkdir(). In practice, this means that an output path such as a/b/c now also works when a/b does not exist yet.

The new test file was added with git add -f because the repository's .gitignore contains a bare tests pattern that matches fsqc/utils/tests/ at any depth. The existing tests in that directory are tracked for the same reason. If preferred, I can also remove the new test file and keep the change limited to the implementation.

[Copilot]

🟡 Changes recommended

ensureDir() should validate retries/delay inputs (e.g., negative retries currently results in silently doing nothing), to avoid surprising behavior in a new shared utility.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR hardens fsqc’s output-directory creation and subsequent writes against filesystems that can temporarily present stale directory metadata (e.g., SMB/NFS), preventing intermittent FileExistsError / “non-existent directory” write failures as described in #107.

Changes:

• Added ensureDir() utility that retries os.makedirs(..., exist_ok=True) on FileExistsError to tolerate transient stale directory visibility.
• Replaced multiple if not isdir(): makedirs()/mkdir() patterns with ensureDir() across the main workflow and motion-check module.
• Added pytest regression tests covering directory creation, existing directories, simulated transient FileExistsError, and persistent path-blocking files.

File summaries

File	Description
fsqc/utils/tests/test_ensure_dir.py	Adds regression tests for tolerant directory creation under transient metadata staleness.
fsqc/fsqcUtils.py	Introduces ensureDir() helper with retry-on-FileExistsError behavior.
fsqc/fsqcMain.py	Routes output directory creation through ensureDir() and re-checks dirs immediately before to_csv() writes.
fsqc/checkMotion.py	Uses ensureDir() for mask output directory creation to avoid stale-metadata failures.

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.