Skip to content

fix(artifact): write and detect artifacts through wide paths on Windows (#1171) - #2461

Open
DeusData wants to merge 4 commits into
mainfrom
fix/issue-1171
Open

DeusData wants to merge 4 commits into
mainfrom
fix/issue-1171

Conversation

@DeusData

@DeusData DeusData commented Oct 1, 2026

Copy link
Copy Markdown
Owner

With persistence=true, indexing a repository under a CJK path on Windows
made the worker exit nonzero and no .codebase-memory/graph.db.zst was
written, while an ASCII junction to the same directory worked.

Root cause: write_file_atomic() -- used for artifact.json, graph.db.zst
and the import temp db -- opened its temp file with the raw CRT fopen()
and published it with MoveFileExA(). Both go through the ANSI code page,
so a repo path containing characters outside it cannot be opened. The
same file had two more ANSI calls on the same flow: cbm_artifact_exists()
used stat() on the repo path, and the import published the cache db with
CRT rename(), which is ANSI and also fails when the destination exists.

Fix: use the repo's UTF-8 helpers -- cbm_fopen() for the temp file,
cbm_rename_replace() (MoveFileExW, write-through, replace-existing) for
both publishes, and a cbm_fopen() probe for the non-empty existence
check. rename_temp now reports errno on every platform (cbm_rename_replace
translates the Win32 error).

Test: artifact::artifact_roundtrip_non_ascii_paths exports from a CJK
repo directory, checks cbm_artifact_exists(), and imports twice into a
CJK cache directory (the second import replaces the existing db).

Follow-up commit (fix(artifact): create .gitattributes through the UTF-8 path layer):

Follow-up to the #1171 fix. Exporting an artifact from a repository under
a CJK path on Windows wrote graph.db.zst and artifact.json, but the
.codebase-memory/.gitattributes file carrying the merge=ours protection
was silently missing.

Root cause: ensure_gitattributes() created the file with the raw POSIX
open(O_WRONLY | O_CREAT | O_EXCL). On Windows that is the ANSI CRT, so a
repo path with characters outside the active code page fails with ENOENT.
The failure was only logged, and the warning itself was malformed: it
passed printf-style arguments to the key/value logger, which printed the
literal "msg=artifact.gitattributes.open_path=%s_err=%s =".

Fix: create the file with cbm_fopen(path, "wbx") (_wfopen on Windows).
The "x" mode keeps the O_CREAT | O_EXCL create-only-if-absent semantics,
so an existing (possibly user-edited) .gitattributes is never rewritten.
Binary mode writes the same LF bytes on every platform. The warning now
uses the logger's key/value form (path=..., err=...).

Test: artifact::artifact_roundtrip_non_ascii_paths now reads
.gitattributes back from the CJK repo through cbm_fopen and asserts the
merge=ours line. It also replaces the file with user content, exports
again, and asserts the content is unchanged. On the Windows VM the
assertion failed 3 of 3 without the fix and passes 3 of 3 with it. The
no-overwrite check fails if the mode loses its "x".

Fixes #1171

…ws (#1171)

With persistence=true, indexing a repository under a CJK path on Windows
made the worker exit nonzero and no .codebase-memory/graph.db.zst was
written, while an ASCII junction to the same directory worked.

Root cause: write_file_atomic() -- used for artifact.json, graph.db.zst
and the import temp db -- opened its temp file with the raw CRT fopen()
and published it with MoveFileExA(). Both go through the ANSI code page,
so a repo path containing characters outside it cannot be opened. The
same file had two more ANSI calls on the same flow: cbm_artifact_exists()
used stat() on the repo path, and the import published the cache db with
CRT rename(), which is ANSI and also fails when the destination exists.

Fix: use the repo's UTF-8 helpers -- cbm_fopen() for the temp file,
cbm_rename_replace() (MoveFileExW, write-through, replace-existing) for
both publishes, and a cbm_fopen() probe for the non-empty existence
check. rename_temp now reports errno on every platform (cbm_rename_replace
translates the Win32 error).

Test: artifact::artifact_roundtrip_non_ascii_paths exports from a CJK
repo directory, checks cbm_artifact_exists(), and imports twice into a
CJK cache directory (the second import replaces the existing db).

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Follow-up to the #1171 fix. Exporting an artifact from a repository under
a CJK path on Windows wrote graph.db.zst and artifact.json, but the
.codebase-memory/.gitattributes file carrying the merge=ours protection
was silently missing.

Root cause: ensure_gitattributes() created the file with the raw POSIX
open(O_WRONLY | O_CREAT | O_EXCL). On Windows that is the ANSI CRT, so a
repo path with characters outside the active code page fails with ENOENT.
The failure was only logged, and the warning itself was malformed: it
passed printf-style arguments to the key/value logger, which printed the
literal "msg=artifact.gitattributes.open_path=%s_err=%s <path>=<error>".

Fix: create the file with cbm_fopen(path, "wbx") (_wfopen on Windows).
The "x" mode keeps the O_CREAT | O_EXCL create-only-if-absent semantics,
so an existing (possibly user-edited) .gitattributes is never rewritten.
Binary mode writes the same LF bytes on every platform. The warning now
uses the logger's key/value form (path=..., err=...).

Test: artifact::artifact_roundtrip_non_ascii_paths now reads
.gitattributes back from the CJK repo through cbm_fopen and asserts the
merge=ours line. It also replaces the file with user content, exports
again, and asserts the content is unchanged. On the Windows VM the
assertion failed 3 of 3 without the fix and passes 3 of 3 with it. The
no-overwrite check fails if the mode loses its "x".

Refs #1171

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[v0.9.0][Windows] persistence=true crashes the indexing worker for repositories under CJK paths

1 participant