Fix races during IDS metadata construction - #145
Open
chupakobra6 wants to merge 1 commit into
Open
Conversation
Freeze metadata per instance instead of removing the class setter, and publish the node type map only after construction. Add deterministic concurrent construction and immutability regressions, independent IDS filling checks, and document the limited thread-safety scope. Encode strings in the existing hash test so the suite also runs with xxhash 4.
chupakobra6
marked this pull request as ready for review
September 11, 2026 13:07
chupakobra6
requested review from
maarten-ic,
olivhoenen and
prasad-sawantdesai
as code owners
September 11, 2026 13:07
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Concurrent metadata cache misses can delete
IDSMetadata.__setattr__twice and raiseAttributeError. The class mutation also temporarily permits writes to metadata belonging to existing IDSs.Freeze each metadata node when its own initialization finishes, keeping the class setter installed throughout construction. This preserves the constructor's assignments and documented attribute definitions. Build the node type map locally before publishing it, so concurrent first use cannot observe missing numeric array types. The metadata cache and read-only attribute errors are preserved.
Fixes #130.
The guarantee is limited to removing these metadata-construction races. It does not provide general thread safety, concurrent access to a shared IDS, or thread safety for IMAS-Core
put,get,serialize, ordeserialize. These limits are documented alongside the metadata API and in the changelog.Tests pause construction at a child node with events while a second IDS is created through
IDSFactory, covering different IDS types and concurrent misses for the same cache key. They also check existing metadata during construction, immutability after both builds, failed construction, and overlapping type-map initialization. Waits are bounded and paused workers are released infinallyblocks. Four independently populatedcore_profilesIDSs are validated and compared with sequential results for DD 3.39.0 and 4.0.0.Validation at
6994382ab643df2085fb9116fc020c9c1c9be0d2:develop(4390fae), the four race regressions fail with twoAttributeErrors, a missing read-only exception, and a type-mapKeyError.git diff --check, distribution build, local Sphinx build, and Read the Docs passed. The local wheel contains the exact committed implementation and tests.A local CPython 3.12 microbenchmark measured uncached
core_profilesmetadata construction at 5.30 ms before and 7.07 ms after this change; cache hits were approximately 0.035 µs in both cases. These are medians of 15 alternating samples with 20 uncached builds per sample, using DD 3.39.0 and already-parsed XML/path caches. Per-instance freezing adds initialization work; it does not affect the cached construction path.One line in the existing string-array hash test now encodes expected input as UTF-8. Its previous raw-string input fails with xxhash 4 on unmodified
develop; production hashing already encodes strings, and the expected digest is unchanged.Upstream GitHub Actions runs require maintainer approval for this external contribution. The successful fork runs linked above tested the exact PR commit with the repository's unchanged workflows.