Skip to content

fix(otel-thread-ctx): don't derive CtxWrap from node::ObjectWrap - #388

Open
szegedi wants to merge 1 commit into
mainfrom
szegedi/ctxwrap-teardown-fix
Open

fix(otel-thread-ctx): don't derive CtxWrap from node::ObjectWrap#388
szegedi wants to merge 1 commit into
mainfrom
szegedi/ctxwrap-teardown-fix

Conversation

@szegedi

@szegedi szegedi commented Aug 7, 2026

Copy link
Copy Markdown

CtxWrap has the same defect #385 fixed in the wall profiler's PersistentContextPtr.

The abort

node::ObjectWrap registers a per-instance environment cleanup hook in its constructor and calls RemoveEnvironmentCleanupHook from its destructor, which CHECKs that an Environment is current. A CtxWrap is owned by a weak V8 handle, so V8 picks when it dies — and weak callbacks run during isolate teardown with no context entered:

Assertion failed: (env) != nullptr
 2: node::RemoveEnvironmentCleanupHook(...)
 3: otel_thread_ctx_nodejs::CtxWrap::~CtxWrap()

This one is not subtle. Create a few thousand ThreadContexts and exit normally:

N before
1, 10, 100 exit 0
1000, 3000 exit 134

Every run, on a plain release build — no ASAN required, unlike the PCP case. Nothing below ~1000 reproduces it: V8 has to still have instances left to collect at teardown. ThreadContext is public API, so any Linux consumer creating contexts at volume aborts at process exit.

Why not just skip the removal

Worth being explicit, because it's the tempting fix and it's wrong. The CHECK guards something real:

inline Environment* Environment::GetCurrent(v8::Isolate* isolate) {
  if (!isolate->InContext()) [[unlikely]] return nullptr;

Null means "no context entered", not "no environment" — the Environment may well be alive. Leaving a hook behind whose arg is a freed pointer turns the abort into a use-after-free when CleanupQueue::Drain later calls cb.fn_(cb.arg_). So the fix is to never register the per-instance hook, not to bypass its removal.

Replacing what the hook did

Dropping the base loses the one thing it provided: deletion at teardown even when V8 never collects the object. PersistentContextPtr could lean on ~WallProfiler walking its live list — CtxWrap has no such owner and owns a malloc'd record_, so a naive port would trade the abort for a leak.

So this adds the equivalent: a thread-local list of live CtxWraps drained by one per-isolate cleanup hook instead of one per instance. It's registered from Wrap() — inside a JS constructor call, where a context is entered, so AddEnvironmentCleanupHook's own CHECK is satisfied honestly — and never removed, since it fires exactly once at teardown while the Environment is alive. Removal timing we control, rather than V8.

Thread-local rather than locked, for the same reason the wall profiler's active-profiler pointer is: Node pins each isolate to a thread, and CtxWraps are only constructed and destroyed on their own isolate's thread. otel_thread_ctx_nodejs_v1 in the same file is thread-local on the same grounds.

offsetof / -Winvalid-offsetof

Losing the base also makes CtxWrap standard-layout — no base subobject, no virtuals, all data members in one access section — so offsetof on it is now unconditionally valid, and the two -Winvalid-offsetof suppressions the inheriting version needed are gone. A static_assert(std::is_standard_layout<CtxWrap>::value) keeps it that way, since the reader contract depends on offsetof(record_) being well-defined. Verified with both clang and gcc 12: builds clean, zero offsetof warnings.

Reader contract

With no base class, record_ is CtxWrap's first member, so threadlocal.native_wrap_fields_offset goes 24 → 0, and is now computed with offsetof rather than sizeof() of a foreign type we don't control. That's a breaking reader-contract change, done now because no readers exist yet. SCHEMA_VERSION is left at nodejs_v1_dev; say the word if you'd rather mark it.

internal-field.hh

Node 26 requires an EmbedderDataTypeTag on both the get and the set of an internal field — the thing that broke my first attempt at #385. The pair now lives in one header so they can't drift when only one is exercised on the version you happen to build against. wall.cc keeps its own copies for now to avoid conflicting with #387; folding those in is a follow-up.

Verification

result
new test vs pre-fix binding exited with code=null signal=SIGABRT
new test vs fixed binding passing
Node 26 npm test exit 0, 164 passing, 0 aborts
Node 24 test:js-asan exit 0, 164 passing, 0 leaks, 0 aborts
Node 20 test:js-asan exit 0, 100 passing, 0 leaks, 0 aborts
probe at N=1000 / 3000 / 10000 all exit 0

The zero leak counts are the load-bearing check: they confirm the drain hook really does replace what ObjectWrap was doing, rather than just moving the failure.

The regression test forks, since the failure is a SIGABRT that would otherwise take the whole mocha run down.

Follow-up

The same defect is present in the upstream custom-labels copy (js/addon.cpp) on both otel-thread-ctx-wip and otel-thread-ctx-invalidate — I reproduced it there with the same probe and thresholds. This PR is intended as the reference implementation to port.

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown

Overall package size

Self size: 2.46 MB
Deduped: 3.17 MB
No deduping: 3.17 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | pprof-format | 2.3.0 | 503.97 kB | 503.97 kB | | source-map | 0.8.0 | 185.66 kB | 185.66 kB | | node-gyp-build | 4.8.4 | 13.86 kB | 13.86 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

@datadog-official

datadog-official Bot commented Aug 7, 2026

Copy link
Copy Markdown

Pipelines  Tests

⚠️ Warnings

🚦 3 Pipeline jobs failed

Build | build / darwin-arm64-test-22   View in Datadog   GitHub Actions

Build | build / win32-test-22   View in Datadog   GitHub Actions

Build | build-successful   View in Datadog   GitHub Actions

ℹ️ Info

🔄 Datadog auto-retried 1 job - 1 passed on retry View in Datadog

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 54e10ab | Docs | Datadog PR Page | Give us feedback!

@szegedi
szegedi force-pushed the szegedi/ctxwrap-teardown-fix branch from 4685c17 to bcb660b Compare August 7, 2026 11:06
@szegedi szegedi added the semver-patch Bug or security fixes, mainly label Aug 7, 2026
@szegedi
szegedi force-pushed the szegedi/ctxwrap-teardown-fix branch from bcb660b to 83939e7 Compare August 7, 2026 12:22
CtxWrap has the same defect #385 fixed in the wall profiler's
PersistentContextPtr. node::ObjectWrap registers a per-instance
environment cleanup hook in its constructor and calls
RemoveEnvironmentCleanupHook from its destructor, which CHECKs that an
Environment is current. A CtxWrap is owned by a weak V8 handle, so V8
picks the moment it dies, and weak callbacks run during isolate teardown
with no context entered:

    Assertion failed: (env) != nullptr
     2: node::RemoveEnvironmentCleanupHook(...)
     3: otel_thread_ctx_nodejs::CtxWrap::~CtxWrap()

This one is not subtle: create a few thousand ThreadContexts and exit
normally and it aborts every time, on a plain release build. No ASAN
needed, unlike the PCP case. Nothing below ~1000 instances reproduces it
— V8 has to still have some left to collect at teardown.

Note the CHECK guards something real, so it must not be worked around by
skipping the removal. Environment::GetCurrent(isolate) returns null on
`!isolate->InContext()` alone, so the Environment may well still be
alive; leaving a hook behind whose arg is a freed pointer would turn the
abort into a use-after-free when CleanupQueue::Drain later calls it. The
fix is to not register the per-instance hook at all.

Dropping the base loses what that hook provided: deletion at teardown
even when V8 never collects the object. PCP could rely on ~WallProfiler
walking its live list; CtxWrap has no such owner and owns a malloc'd
record, so without a replacement this would trade an abort for a leak.
Add the equivalent: a thread-local list of live CtxWraps drained by a
single per-isolate cleanup hook, registered from Wrap() — inside a JS
constructor call, where a context is entered, so AddEnvironmentCleanupHook
is satisfied honestly — and never removed, since it fires once at
teardown while the Environment is alive. One hook per isolate instead of
one per instance, with removal timing we control rather than V8.

With no base class, `record_` becomes CtxWrap's first member, so the
published threadlocal.native_wrap_fields_offset goes from 24 to 0 and is
now computed with offsetof rather than sizeof() of a foreign type. That
is a reader-contract change, made now because no readers exist yet.

Losing the base also makes CtxWrap standard-layout — no base subobject,
no virtuals, all data members in one access section — so offsetof on it
is now unconditionally valid and the two -Winvalid-offsetof suppressions
the inheriting version needed are gone. A static_assert on
is_standard_layout keeps it that way, since the reader contract depends
on offsetof(record_) being well-defined.

The two internal-field accessors move to a new internal-field.hh: Node 26
requires an EmbedderDataTypeTag on both the get and the set, and having
the pair in one place stops them drifting when only one is exercised on
the version you build against. wall.cc keeps its own copies for now to
avoid conflicting with in-flight work there; folding those in is a
follow-up.

Verified on Node 20, 24 and 26, with both clang and gcc. New regression
test fails with signal=SIGABRT against the pre-fix binding and passes
after; ASAN exit 0 with zero leaks on 20 and 24, which is the check that
the drain hook really does replace what ObjectWrap was doing.
@szegedi
szegedi force-pushed the szegedi/ctxwrap-teardown-fix branch from 83939e7 to 54e10ab Compare August 7, 2026 12:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

semver-patch Bug or security fixes, mainly

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant