Repository navigation
Conversation
…ws (#1171) With persistence=true, indexing a repository under a CJK path on Windows made the worker exit nonzero and no .codebase-memory/graph.db.zst was written, while an ASCII junction to the same directory worked. Root cause: write_file_atomic() -- used for artifact.json, graph.db.zst and the import temp db -- opened its temp file with the raw CRT fopen() and published it with MoveFileExA(). Both go through the ANSI code page, so a repo path containing characters outside it cannot be opened. The same file had two more ANSI calls on the same flow: cbm_artifact_exists() used stat() on the repo path, and the import published the cache db with CRT rename(), which is ANSI and also fails when the destination exists. Fix: use the repo's UTF-8 helpers -- cbm_fopen() for the temp file, cbm_rename_replace() (MoveFileExW, write-through, replace-existing) for both publishes, and a cbm_fopen() probe for the non-empty existence check. rename_temp now reports errno on every platform (cbm_rename_replace translates the Win32 error). Test: artifact::artifact_roundtrip_non_ascii_paths exports from a CJK repo directory, checks cbm_artifact_exists(), and imports twice into a CJK cache directory (the second import replaces the existing db). Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Follow-up to the #1171 fix. Exporting an artifact from a repository under a CJK path on Windows wrote graph.db.zst and artifact.json, but the .codebase-memory/.gitattributes file carrying the merge=ours protection was silently missing. Root cause: ensure_gitattributes() created the file with the raw POSIX open(O_WRONLY | O_CREAT | O_EXCL). On Windows that is the ANSI CRT, so a repo path with characters outside the active code page fails with ENOENT. The failure was only logged, and the warning itself was malformed: it passed printf-style arguments to the key/value logger, which printed the literal "msg=artifact.gitattributes.open_path=%s_err=%s <path>=<error>". Fix: create the file with cbm_fopen(path, "wbx") (_wfopen on Windows). The "x" mode keeps the O_CREAT | O_EXCL create-only-if-absent semantics, so an existing (possibly user-edited) .gitattributes is never rewritten. Binary mode writes the same LF bytes on every platform. The warning now uses the logger's key/value form (path=..., err=...). Test: artifact::artifact_roundtrip_non_ascii_paths now reads .gitattributes back from the CJK repo through cbm_fopen and asserts the merge=ours line. It also replaces the file with user content, exports again, and asserts the content is unchanged. On the Windows VM the assertion failed 3 of 3 without the fix and passes 3 of 3 with it. The no-overwrite check fails if the mode loses its "x". Refs #1171 Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2 tasks done
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With persistence=true, indexing a repository under a CJK path on Windows
made the worker exit nonzero and no .codebase-memory/graph.db.zst was
written, while an ASCII junction to the same directory worked.
Root cause: write_file_atomic() -- used for artifact.json, graph.db.zst
and the import temp db -- opened its temp file with the raw CRT fopen()
and published it with MoveFileExA(). Both go through the ANSI code page,
so a repo path containing characters outside it cannot be opened. The
same file had two more ANSI calls on the same flow: cbm_artifact_exists()
used stat() on the repo path, and the import published the cache db with
CRT rename(), which is ANSI and also fails when the destination exists.
Fix: use the repo's UTF-8 helpers -- cbm_fopen() for the temp file,
cbm_rename_replace() (MoveFileExW, write-through, replace-existing) for
both publishes, and a cbm_fopen() probe for the non-empty existence
check. rename_temp now reports errno on every platform (cbm_rename_replace
translates the Win32 error).
Test: artifact::artifact_roundtrip_non_ascii_paths exports from a CJK
repo directory, checks cbm_artifact_exists(), and imports twice into a
CJK cache directory (the second import replaces the existing db).
Follow-up commit (fix(artifact): create .gitattributes through the UTF-8 path layer):
Follow-up to the #1171 fix. Exporting an artifact from a repository under
a CJK path on Windows wrote graph.db.zst and artifact.json, but the
.codebase-memory/.gitattributes file carrying the merge=ours protection
was silently missing.
Root cause: ensure_gitattributes() created the file with the raw POSIX
open(O_WRONLY | O_CREAT | O_EXCL). On Windows that is the ANSI CRT, so a
repo path with characters outside the active code page fails with ENOENT.
The failure was only logged, and the warning itself was malformed: it
passed printf-style arguments to the key/value logger, which printed the
literal "msg=artifact.gitattributes.open_path=%s_err=%s =".
Fix: create the file with cbm_fopen(path, "wbx") (_wfopen on Windows).
The "x" mode keeps the O_CREAT | O_EXCL create-only-if-absent semantics,
so an existing (possibly user-edited) .gitattributes is never rewritten.
Binary mode writes the same LF bytes on every platform. The warning now
uses the logger's key/value form (path=..., err=...).
Test: artifact::artifact_roundtrip_non_ascii_paths now reads
.gitattributes back from the CJK repo through cbm_fopen and asserts the
merge=ours line. It also replaces the file with user content, exports
again, and asserts the content is unchanged. On the Windows VM the
assertion failed 3 of 3 without the fix and passes 3 of 3 with it. The
no-overwrite check fails if the mode loses its "x".
Fixes #1171