Skip to content

FIX: count attacks without converters as no_converters in analytics - #3170

Open
Utkarsh Bahuguna (u7k4rs6) wants to merge 1 commit into
microsoft:mainfrom
u7k4rs6:fix/analytics-no-converters
Open

Utkarsh Bahuguna (u7k4rs6) wants to merge 1 commit into
microsoft:mainfrom
u7k4rs6:fix/analytics-no-converters

Conversation

@u7k4rs6

Copy link
Copy Markdown
Contributor

Fixes #3088.

Identifiers leave out an empty converter list, so a real attack with no converters had no list and no edges, and its converter array came out NULL, which is "missing". The converter source now falls back to [] when the attack identifier is recorded (the joined attack hash is there), which is "no_converters". Results with no attack identifier stay missing, and a retained list or edges still win first.

Both arrays (with and without the embedded list) get the fallback, so grouping, filters, facets and drill-downs all agree.

Test runs real PromptSendingAttacks, one plain and one with Base64 on request and response, plus a result with no identifier, for both directions:

  • group by converter_type gives no_converters / base64converter / missing, one each
  • filtering on no_converters returns the plain attack
  • get_attack_results_async(has_converters=False) agrees

It fails on main. tests/unit/memory, analytics, backend and models: 6476 passed.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: analytics no_converters bucket never matches real attacks

1 participant