Skip to content

[Backport][0.7 to 0.6] | | | fix(thread): complete context-switch clobbers and reserve Win64 shadow space (#1720) (#1725) (#1726) (#1727) - #1728

Merged
lihuiba merged 1 commit into
release/0.6from
backport-pr-982b7117943df6505c45f780a688229c99dace04-0.6
Nov 4, 2026
Merged

lihuiba merged 1 commit into
release/0.6from
backport-pr-982b7117943df6505c45f780a688229c99dace04-0.6

Conversation

@photonlibos

Copy link
Copy Markdown
Collaborator

[Backport][0.8 to 0.7] | | fix(thread): complete context-switch clobbers and reserve Win64 shadow space (#1720) (#1725) (#1726) (#1727)

switch_context() / switch_context_defer() hide a function call inside an
asm statement, across which other photon threads run and may change any
register or memory. The clobber lists only named GPRs (plus xmm6-15 on
Win64), so the compiler was free to keep values in vector registers, in
memory it had cached, or in the red zone across a switch.

This is what makes GCC 10/11/12 CI jobs crash (SEGFAULT in test-thread,
test-socket, test-pool, test-executor-std, test-exportfs) on part of the
nodes: with -march=native on a CPU model unknown to these versions (most
likely Emerald Rapids, 0xCF), the driver falls back to -march=cooperlake
-mtune=generic, under which GCC keeps the thread pointer behind
return rq.current->error_number in %xmm0 across the switch in
thread_yield_to().

  • x86_64 / aarch64: declare all vector registers (and AVX-512 opmask /
    SVE predicate registers when enabled) as clobbered, plus "memory";
    plus "cc" on aarch64.
  • x86_64 SysV / Apple arm64: skip the 128-byte red zone before the call,
    as GCC's manual describes for asm containing calls or pushes.
  • x86_64: call the defer function with %rsp aligned to 16 bytes instead
    of jumping into it on whatever the switch-point %rsp happens to be.
  • Win64: reserve the 32-byte shadow space for the defer function and the
    thread entry; the latter otherwise overlaps the head of struct thread.
  • tools/photongdb.py: account for the skipped red zone when recovering
    %rsp of a switched-out thread.
  • fix(thread): clarify why volatile registers must be listed as clobbers

The asm statement is not treated as a call, so the ABI's caller-saved
registers are not implied clobbered; nor does the C++ call of the inline
switch_context() help, as it is inlined, or subject to -fipa-ra.

  • fix(thread): remove speculative red-zone padding

Restore the original saved-stack layout and photongdb offset. Keep vector/memory/cc clobbers, defer alignment and Win64 shadow space; no current Photon red-zone failure has been demonstrated.

  • fix(thread): preserve original defer jump trampolines

  • fix(thread): move Win64 shadow-space comment to _photon_thread_stub

  • fix(thread): resolve Win64 thread stub conflict for release/0.9

Take the 32-byte shadow-space reservation from #1720 and drop the
Win64 call to _asan_start, as main did in #937. ASan builds are
Linux-only (PHOTON_BUILD_WITH_ASAN fails the configure elsewhere),
so on Win64 _asan_start is always an empty function.


Co-authored-by: Coldwings coldwings@me.com

  • fix(thread): resolve the Win64 thread stub conflict for release/0.7

Take the 32-byte shadow-space reservation from #1720. release/0.7 never
had the Win64 call _asan_start that the upstream change replaced, so
there is nothing to drop here.


Co-authored-by: Coldwings coldwings@me.com
Co-authored-by: lihuiba huiba.lhb@alibaba-inc.com
Generated by Backport Auto PR, by cherry-pick related commits.

Please review and decide whether to merge or close this backport PR.

…ers and reserve Win64 shadow space (#1720) (#1725) (#1726) (#1727)

* [Backport][main to 0.9] | fix(thread): complete context-switch clobbers and reserve Win64 shadow space (#1720) (#1725) (#1726)

* fix(thread): complete context-switch clobbers and reserve Win64 shadow space (#1720)

* fix(thread): complete clobbers of context switch, align defer stack, skip red zone

switch_context() / switch_context_defer() hide a function call inside an
asm statement, across which other photon threads run and may change any
register or memory. The clobber lists only named GPRs (plus xmm6-15 on
Win64), so the compiler was free to keep values in vector registers, in
memory it had cached, or in the red zone across a switch.

This is what makes GCC 10/11/12 CI jobs crash (SEGFAULT in test-thread,
test-socket, test-pool, test-executor-std, test-exportfs) on part of the
nodes: with -march=native on a CPU model unknown to these versions (most
likely Emerald Rapids, 0xCF), the driver falls back to -march=cooperlake
-mtune=generic, under which GCC keeps the thread pointer behind
`return rq.current->error_number` in %xmm0 across the switch in
thread_yield_to().

- x86_64 / aarch64: declare all vector registers (and AVX-512 opmask /
  SVE predicate registers when enabled) as clobbered, plus "memory";
  plus "cc" on aarch64.
- x86_64 SysV / Apple arm64: skip the 128-byte red zone before the call,
  as GCC's manual describes for asm containing calls or pushes.
- x86_64: call the defer function with %rsp aligned to 16 bytes instead
  of jumping into it on whatever the switch-point %rsp happens to be.
- Win64: reserve the 32-byte shadow space for the defer function and the
  thread entry; the latter otherwise overlaps the head of `struct thread`.
- tools/photongdb.py: account for the skipped red zone when recovering
  %rsp of a switched-out thread.

* fix(thread): clarify why volatile registers must be listed as clobbers

The asm statement is not treated as a call, so the ABI's caller-saved
registers are not implied clobbered; nor does the C++ call of the inline
switch_context() help, as it is inlined, or subject to -fipa-ra.

* fix(thread): remove speculative red-zone padding

Restore the original saved-stack layout and photongdb offset. Keep vector/memory/cc clobbers, defer alignment and Win64 shadow space; no current Photon red-zone failure has been demonstrated.

* fix(thread): preserve original defer jump trampolines

* fix(thread): move Win64 shadow-space comment to _photon_thread_stub

* fix(thread): resolve Win64 thread stub conflict for release/0.9

Take the 32-byte shadow-space reservation from #1720 and drop the
Win64 call to _asan_start, as main did in #937. ASan builds are
Linux-only (PHOTON_BUILD_WITH_ASAN fails the configure elsewhere),
so on Win64 _asan_start is always an empty function.

---------

Co-authored-by: Coldwings <coldwings@me.com>

* fix(thread): resolve the Win64 thread stub conflict for release/0.7

Take the 32-byte shadow-space reservation from #1720. release/0.7 never
had the Win64 `call _asan_start` that the upstream change replaced, so
there is nothing to drop here.

---------

Co-authored-by: Coldwings <coldwings@me.com>
Co-authored-by: lihuiba <huiba.lhb@alibaba-inc.com>
@photonlibos photonlibos added the need-backport A PR that should be back-ported to prior release branches (release/*) label Nov 4, 2026
@lihuiba
lihuiba merged commit fa8301d into release/0.6 Nov 4, 2026
7 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

need-backport A PR that should be back-ported to prior release branches (release/*)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants