listIosDevices can leave its temp directory behind, and the test that guards it asserts on the shared tmpdir

[janicduplessis]

Two problems, found because the second one failed the unit suite on a developer machine while CI stayed green.

1. The temp directory leaks from real runs

listIosDevices writes devicectl output into a stim-devicectl-* directory under tmpdir() and removes it afterwards. After an iOS caches e2e run, one survived with its payload intact:

drwx------  3 janicduplessis  staff   96 Sep 11 08:17 /var/folders/.../T/stim-devicectl-YRNy7x
-rw-r--r--  1 janicduplessis  staff  487 Sep 11 08:17 devices.json

So the removal is skipped on some path — most likely when the process is interrupted or the call throws between writing and cleaning up. Each leak is small, but they accumulate silently in tmpdir() and are invisible to stim gc, which does not survey them.

2. The guard asserts on global state, so anything else on the machine can fail it

packages/stim-cli/src/__tests__/engine-ios-device.test.ts:281:

expect(readdirSync(tmpdir()).some((e) => e.startsWith('stim-devicectl-') && existsSync(join(tmpdir(), e)))).toBe(
  false,
);

and at :692 the same idea as a deep-equality against []. These read the shared system temp directory, not a directory the test owns. Any leaked directory from any earlier or concurrent Stim process fails them, regardless of whether the code under test behaved correctly.

Observed as:

FAIL  engine-ios-device.test.ts > listIosDevices runs devicectl into a temp file, parses it, and removes the directory
AssertionError: expected true to be false

FAIL  engine-ios-device.test.ts > listIosDevices against a real devicectl > the argv is accepted and every entry it returns is well formed
AssertionError: expected [ 'stim-devicectl-YRNy7x' ] to deeply equal []

Removing the single leaked directory made the file pass with nothing else changed — 131/131 test files green afterwards.

This is why CI never sees it: a fresh runner has an empty tmpdir(). It is reproducible for anyone who has run Stim on their machine, and it presents as a failure in code the developer did not touch, which is the expensive kind of false signal.

Suggested fix

For 2, have the test scope its assertion to directories it created — record the paths the stubbed executor was handed and assert those are gone — rather than scanning a directory it does not own.

For 1, remove the directory in a finally so an interrupted or throwing call cannot leave it, and consider having gc report stim-devicectl-* residue so existing leaks are reclaimable.

Both predate 1.1.0.

[janicduplessis]

Claimed: @janic/issue-714-owned-temp-assertions

[janicduplessis]

Triage against current main: the shared-temp assertions are fixed by #725, and both listIosDevices and iosDeviceProcess already remove their invocation-owned directory in finally, including executor exceptions. The remaining report is interrupted-process residue, which still needs a controlled reproduction; adding another finally would not fix it. Keeping this open for that narrower investigation, without adding a broad temp-directory sweep.

[janicduplessis]

Both halves are resolved, and one of them was misdiagnosed in this issue.

Problem 2 (the false failure) is fixed. No readdirSync(tmpdir()) scan remains anywhere in the package. The guard now asserts only on what the test itself created — outPath and dirname(outPath), captured from the stubbed executor — and it additionally creates its own stim-devicectl-* sentinel and asserts that one survives, so the test also proves the code does not sweep directories it does not own. That is a stronger guard than the original and it cannot be failed by residue from another process. 48/48 in engine-ios-device.test.ts, exit 0.

Problem 1 was diagnosed wrong here. This issue guessed the removal is "skipped on some path — most likely when the call throws". It is not: both call sites (listIosDevices, iosDeviceProcess) have removed the directory in a finally since 02cfcb2 / 43fd760 on 2026-09-01, ten days before the leaked directory was observed on 09-11. A throw was already covered when that leak happened, so the throw path cannot be the cause. The only way the directory survives is the process being killed outright — SIGKILL, or SIGINT landing before finally runs — which no finally can cover.

Not adding a gc category for it. The residue is ~487 bytes per event and only a killed process produces one. Three days and many iOS runs after the original observation — the full cache QA matrix, an iPad run, and three concurrent agent loops — the machine has zero stim-devicectl-* directories anywhere under /var/folders. A sweep category to reclaim a few hundred bytes from a rare kill-timed event is not worth the code or the concurrency care it would need (it would have to age-threshold every candidate to avoid deleting a live run's directory).

Reopen if residue ever actually accumulates; the evidence right now says it does not.