Skip to content

perf_hooks: implement qrde analysis support in Histogram - #65806

Open
jasnell wants to merge 1 commit into
nodejs:mainfrom
jasnell:jasnell/perf-hooks-qrde
Open

perf_hooks: implement qrde analysis support in Histogram#65806
jasnell wants to merge 1 commit into
nodejs:mainfrom
jasnell:jasnell/perf-hooks-qrde

Conversation

@jasnell

@jasnell jasnell commented Sep 5, 2026

Copy link
Copy Markdown
Member

Continue expanding the built-in statistical analysis capabilities of perf_hooks Histogram.

Implements quantile-respectful density estimate (qrde) calculation on Histogram. Helps with tail-focused latency analysis without retaining raw samples. Think of it as a way of generating and exporting a compact analysis of the collected metrics without having to export the entire bucket. Very useful for generating plots of latency data.

Baking this directly into Node.js, based on a 1 million sample, 1k bin workload, this impl is roughly 150-325x faster than performing the equivalent in Rscript.

It's an async API because, while calculation should typically be quite fast for reasonably sized sample sets, the cost can compound as the number of buckets in the Histogram increases.

Example application that uses it: https://gist.github.com/jasnell/a5283d73b4ef6bb6e4e549bf15dc8f70

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/performance

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. labels Sep 5, 2026

@mcollina mcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RSLGTM

have you verified the algorithm against a reference implementation?

Is this such a complex computation that requires using the thread pool?

@codecov

codecov Bot commented Sep 5, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.06029% with 43 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.16%. Comparing base (57860ef) to head (26cb664).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/histogram.cc 89.78% 19 Missing and 23 partials ⚠️
lib/internal/histogram.js 98.52% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #65806      +/-   ##
==========================================
+ Coverage   90.14%   90.16%   +0.02%     
==========================================
  Files         769      769              
  Lines      262968   263424     +456     
  Branches    50052    50140      +88     
==========================================
+ Hits       237049   237515     +466     
+ Misses      16924    16894      -30     
- Partials     8995     9015      +20     
Files with missing lines Coverage Δ
src/histogram.h 73.91% <100.00%> (+1.18%) ⬆️
lib/internal/histogram.js 96.40% <98.52%> (+0.17%) ⬆️
src/histogram.cc 86.09% <89.78%> (+1.25%) ⬆️

... and 29 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jasnell

jasnell commented Sep 5, 2026

Copy link
Copy Markdown
Member Author

have you verified the algorithm against a reference implementation?

Yes! Verified everything against the equivalent Rscript implementation. There's even test for that with one of the fixtures.

Is this such a complex computation that requires using the thread pool?

It can be, yeah but it really depends on the size of data set. I was able to get the implementation pretty highly optimized but the calculation can range from tens to hundreds of milliseconds in common cases and I had one test case hit at ~1 second. While still way faster than the equivalent R implementation, given that these are better suited for post-collection analysis, making it async felt like the best option. That said, I was able to make major performance improvements in this over the original iteration and with the typical cases being in the 10s-100s ms range, we might be able to get away with making it fully sync. I'm still evaluating and running some more benchmarks today.

Implements quantile-respectful density estimate calcuation
on Histogram. Helps with tail-focused latency analysis
without retaining raw samples.

Baking this directly into Node.js, based on a 1 million
sample, 1k bin workload, this impl is roughly 150-325x
faster than performing the equivalent in Rscript.

Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode
@jasnell
jasnell force-pushed the jasnell/perf-hooks-qrde branch from 26cb664 to 0a744b8 Compare September 5, 2026 14:42
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants