Skip to content

Re-enable 18 quarantined AgentHubCore tests (headless failures + hangs) #380

Description

@jamesrochabrun

Summary

The AgentHubCore test suite now runs headlessly (./scripts/test.sh, and CI via .github/workflows/test.yml). The first time it ran outside Xcode's Cmd+U it surfaced 18 tests that fail or hang headlessly. To keep the gate green they are temporarily quarantined with .disabled("headless-quarantine: …") and documented in TestQuarantine.md.

None of these are fixed — each is a real bug or a flaky/non-deterministic test. This issue tracks fixing the root causes and removing the .disabled(...) traits. They group into 4 root causes; A is a genuine product concurrency bug and the highest priority.

To work one: remove its .disabled(...) trait and run
cd app/modules/AgentHubCore && xcodebuild test -scheme AgentHubCore-Tests -destination 'platform=macOS' -test-timeouts-enabled YES -only-testing:<Target>/<SuiteType>/<method>.


A — Git subprocess deadlock on non-git paths (product bug · HANG) 🔴

GitDiffService.runGitCommand (AgentHubGitDiff/Services/GitDiffService.swift) spawns /usr/bin/git and reads its pipes with Pipe() + readToEnd() + readGroup.wait(). Under concurrent spawning (parallel tests, and the real app), pipe write-end FDs are inherited by sibling git children, so readToEnd() never sees EOF until those unrelated children exit — the call blocks until the 30s gitCommandTimeout fires. Raw git rev-parse --show-toplevel on a non-git dir returns in 0.00s, so this is purely the Swift Process/pipe handling, not git. This likely causes real app responsiveness issues, not just test hangs.

Fix direction: set close-on-exec on the pipe FDs (or use posix_spawn_file_actions); and/or fast-fail findGitRoot when libgit2 has already reported "no repository" instead of falling back to the CLI.

  • DiffAvailabilityServiceTests → "Non-git path is unavailable"

B — Reactive / async-delivery timing (test fragility, product-adjacent)

These poll waitUntil { … } / await condition() for state delivered via Combine .values async sequences or real Date.now/sleep-based throttles. Values sent between awaits get dropped, or the timing window is too tight under headless parallel load. The CLISessionsViewModel subscription is setupSubscriptions() (CLISessionsViewModel.swift, for await … repositoriesPublisher.values).

Fix direction: make delivery deterministic (await the publisher directly / inject a clock / pump the run loop) instead of wall-clock polling.

C — Temp-dir symlink path normalization

macOS temp dirs (/var/folders/…, /tmp) are symlinks to /private/…. Fixtures pass the unresolved path; some product comparisons resolve symlinks and some don't, so prefix/equality checks mismatch. WorktreeModuleResolver.normalizedDirectoryPath and CLISessionsViewModel.isProjectPath do raw prefix matching without resolvingSymlinksInPath().

Fix direction: resolve symlinks consistently (canonicalize in the product path helpers, or resolve fixture temp paths). Decide whether product code should canonicalize.

  • GitDiffServiceTests → "branch changes are scoped to the selected worktree"
  • SpotlightProjectFileSearchServiceTests → "Ranks Spotlight paths and filters directories and paths outside the project"
  • WorktreeSettingsInventoryTests → "Delete worktree for nested session removes the worktree root" (may overlap with B)

D — Deterministic product-vs-test drift / latent bugs (decide which side is right)

  • CodexTimestampParserTests → "Returns nil for invalid timestamps" — product bug: the strict byte-parser rejects 2026-02-31, but the ISO8601DateFormatter fallback (CodexTimestampParser.swift) leniently rolls it to Mar 3. Reject in the fallback, or relax the test.
  • EmbeddedTerminalLaunchBuilderTests → "Claude launch passes raw prompt, AgentHub MCP config, and installs worktree skill" — asserts shellCommand.contains("/bin/sh") but the MCP-config JSON escapes slashes as \/bin\/sh. Test too literal, or builder should encode with .withoutEscapingSlashes.
  • EmbeddedTerminalLaunchBuilderTests → "Blank Claude launch still passes AgentHub MCP config and installs worktree skill" — same JSON slash-escaping issue.
  • InlineEditStyleReconcilerTests → "Forwards request fields to the programmatic service" — system prompt no longer contains "preserve the semantic change"; prompt text drifted vs the test.
  • SpotlightProjectFileSearchServiceTests → "Escapes quoted query literals before building the metadata query" — the backslash-escape branch in SpotlightQueryBuilder.escapeQuotedString appears unreachable because \ is consumed as a path separator by filename-component extraction first.

Context: headless test infrastructure added in this branch (scripts/test.sh, AgentHubCore-Tests scheme, CI test.yml). Full root-cause notes live in TestQuarantine.md.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions