chore(appsec): classify Error Tracking audit sites by darccio · Pull Request #5466 · DataDog/dd-trace-go

darccio · 2026-09-28T20:53:55Z

Status

Classify 17 of the 19 AppSec audit sites as ineligible for Error Tracking reporting:

caller API use while AppSec is disabled or without required request instrumentation
user-provided blocked-response template paths
native-library and host compatibility failures
expected per-request WAF event capacity limits
Remote Config failures returned to the backend through apply statuses

Two actionable internal startup sites remain intentionally unclassified:

duplicate internal WAF action-handler registration
unexpected AppSec startup failure

Both can execute before the telemetry reporter is ready. Adoption remains gated on the startup-ordering work tracked with #5250. This PR remains a draft until that ordering is safe; it does not hide these two sites with exclusion directives.

This branch starts from main and is not stacked on another migration PR.

Validation

go test ./appsec ./internal/appsec/... ./instrumentation/appsec/... -count=1
go vet ./appsec ./internal/appsec/... ./instrumentation/appsec/...
make lint/errlog
git diff --check

chatgpt-codex-connector · 2026-09-28T20:59:33Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-29T07:24:53.188651Z	`10c4f9f`	Manual request
🔒 Security Review	✅ Completed 2026-09-29T07:25:15.146526Z	`10c4f9f`	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

datadog-prod-us1-6 · 2026-09-28T21:00:11Z

✅ All CI checks and tests passed.

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 23.53%
• Overall Coverage: 71.20% (+12.84%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 10c4f9f | Docs | View more details | Give us feedback!}

pr-commenter · 2026-09-28T21:38:13Z

Benchmarks

Benchmark execution time: 2026-09-28 22:20:23

Comparing candidate commit 10c4f9f in PR branch session/errtrack-appsec with baseline commit e80f19e in branch main.

📊 Benchmarking dashboard

Found 0 performance improvements and 0 performance regressions! Performance is the same for 340 metrics, 1 unstable metrics, 1 flaky benchmarks without significant changes.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

🟩 = significantly better candidate vs. baseline
🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:BenchmarkContextWithSpan/detach-no-snapshot

unstable execution_time [-1.738ns; +1.570ns] or [-6.023%; +5.441%]

Known flaky benchmarks

These benchmarks are marked as flaky and will not trigger a failure. Modify FLAKY_BENCHMARKS_REGEX to control which benchmarks are marked as flaky.

Known flaky benchmarks without significant changes:

scenario:BenchmarkOTLPTraceWriterFlush

darccio · 2026-09-29T06:06:47Z

@codex review

chatgpt-codex-connector · 2026-09-29T06:08:59Z

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: 10c4f9fa92

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-09-29T06:09:51Z

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 10c4f9fa92

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 10c4f9fa92

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-09-29T06:43:07Z

 		}
 		// 2. If there is an error and the loading is ok: log as an informative error where appsec can be used
-		log.Error("appsec: non-critical error while loading libddwaf: %s", err.Error())
+		log.Error("appsec: non-critical error while loading libddwaf: %s", err.Error()) //errtrack:ignore host native-library compatibility failure


Add the required AppSec README update

This change introduces persistent Error Tracking eligibility classifications throughout internal/appsec, but does not update internal/appsec/README.md; the scoped repository instructions explicitly require that README to be updated whenever AppSec changes are introduced and direct reviewers to flag omissions. Document the classification convention and rationale so future audits do not depend solely on scattered inline markers.

AGENTS.md reference: internal/appsec/AGENTS.md:L3-L7

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-09-29T06:43:07Z


 	if !op.limiter.Allow() {
-		log.Error("appsec: too many WAF events, stopping further reporting")
+		log.Error("appsec: too many WAF events, stopping further reporting") //errtrack:ignore expected per-request capacity limit


Describe this as a global rate limit

This exclusion records the wrong rationale: op.limiter is the single limiter owned by the WAF feature, constructed from cfg.TraceRateLimit, and installed into every request context, so exhaustion here is global across requests. Only the separate maxWAFEventsPerRequest branch below is a per-request capacity limit; describe this site as expected global rate limiting on the request path so the audit record accurately explains why it is excluded.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-09-29T06:43:07Z

 	// to all config statuses since we can't know which config is the faulty one
 	if err := a.SwapRootOperation(); err != nil {
-		log.Error("appsec: remote config: could not apply the new security rules: %s", err.Error())
+		log.Error("appsec: remote config: could not apply the new security rules: %s", err.Error()) //errtrack:ignore failure is returned through Remote Config apply status


Do not exclude failures that produce no apply status

This failure is not always returned through a Remote Config apply status. onRCRulesUpdate creates status entries only for ASM rule/data products, but the global callback also receives products such as ASM_FEATURES and explicitly ignored non-ASM products; it still calls SwapRootOperation afterward. When a poll contains only those ignored products, statuses is empty, so this error is merely logged and the loop cannot associate it with any apply status, making the new blanket exclusion hide a swallowed poll-path SDK failure from the Error Tracking audit.

Useful? React with 👍 / 👎.

darccio · 2026-09-29T07:22:26Z

@codex review

chatgpt-codex-connector · 2026-09-29T07:24:51Z

Codex Review: Didn't find any major issues. 👍

Reviewed commit: 10c4f9fa92

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-09-29T07:25:13Z

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 10c4f9fa92

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

chore(appsec): start Error Tracking eligibility review

6738578

darccio marked this pull request as ready for review September 28, 2026 20:59

darccio requested a review from a team as a code owner September 28, 2026 20:59

chore(appsec): classify ineligible Error Tracking sites

10c4f9f

darccio changed the title ~~chore(appsec): adopt explicit Error Tracking reporting~~ chore(appsec): classify Error Tracking audit sites Sep 28, 2026

darccio marked this pull request as draft September 28, 2026 21:49

darccio marked this pull request as ready for review September 29, 2026 06:37

chatgpt-codex-connector Bot reviewed Sep 29, 2026

View reviewed changes

Conversation

darccio commented Sep 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Status

Validation

Uh oh!

chatgpt-codex-connector Bot commented Sep 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

datadog-prod-us1-6 Bot commented Sep 28, 2026 • edited by datadog-prod-us1-4 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter Bot commented Sep 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Explanation

More details about the CI and significant changes

Unstable benchmarks

scenario:BenchmarkContextWithSpan/detach-no-snapshot

Known flaky benchmarks

Uh oh!

darccio commented Sep 29, 2026

Uh oh!

chatgpt-codex-connector Bot commented Sep 29, 2026

Uh oh!

chatgpt-codex-connector Bot commented Sep 29, 2026

🛡️ Codex Security Review · Automatically triggered

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Sep 29, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Sep 29, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Sep 29, 2026

Choose a reason for hiding this comment

Uh oh!

darccio commented Sep 29, 2026

Uh oh!

chatgpt-codex-connector Bot commented Sep 29, 2026

Uh oh!

chatgpt-codex-connector Bot commented Sep 29, 2026

🛡️ Codex Security Review · Automatically triggered

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

darccio commented Sep 28, 2026 •

edited

Loading

chatgpt-codex-connector Bot commented Sep 28, 2026 •

edited

Loading

datadog-prod-us1-6 Bot commented Sep 28, 2026 •

edited by datadog-prod-us1-4 Bot

Loading

pr-commenter Bot commented Sep 28, 2026 •

edited

Loading