Skip to content

perf(temporal): reuse the prediction HTTP client - #708

Open
frgfm wants to merge 1 commit into
mainfrom
codex/perf-temporal-client
Open

frgfm wants to merge 1 commit into
mainfrom
codex/perf-temporal-client

Conversation

@frgfm

@frgfm frgfm commented Oct 7, 2026 •

Copy link
Copy Markdown
Member

Reuse one HTTP client per API worker. Close it after the validation worker stops.

flowchart LR
    A[Create once] --> B[Reuse for predictions] --> C[Close at shutdown]
Loading
Warm median per prediction main This PR Less time
HTTP 9.93 ms 1.47 ms 85%
HTTP, server closes every connection 8.32 ms 2.11 ms 75%
HTTP, simulated 50 ms inference 60.29 ms 51.76 ms 14%

Actual predict(), local TCP, Python 3.11.15/httpx 0.28.1; median of 5 fresh-process runs × 100 calls after warmup. Inference is simulated; first-call speedup is unproven.

Benchmark sources and raw results.

Scope: +10 net production lines (+29 total), one new test. Independent of #705, #706 and #707; merge checks pass.

Checks: 25 focused tests, real TCP recovery/shutdown probes, lint, typing, pre-commit and full CI pass.

@codecov

codecov Bot commented Oct 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.89%. Comparing base (25f3d2e) to head (48eef0e).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #708      +/-   ##
==========================================
+ Coverage   93.87%   93.89%   +0.01%     
==========================================
  Files          59       59              
  Lines        3218     3226       +8     
==========================================
+ Hits         3021     3029       +8     
  Misses        197      197              
Flag Coverage Δ
backend 94.00% <100.00%> (+0.01%) ⬆️
client 91.30% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant