feat(eap): Implement gen_ai attribute transformations by constantinius · Pull Request #6201 · getsentry/relay

constantinius · 2026-07-09T14:41:52Z

Summary

Implements the two gen_ai attribute transformations introduced in sentry-conventions PR #465.

Old SDKs send AI messages and responses using deprecated attributes (gen_ai.request.messages, gen_ai.response.text, gen_ai.response.tool_calls) whose value shapes differ from the canonical replacements (gen_ai.input.messages, gen_ai.output.messages). Simple key renaming is not enough — the values need reshaping to match the new parts-based message schema.

The transformation runs inside normalize_attribute_names so it applies to every path that normalizes attributes — both SpanV1 and SpanV2. Attributes with the new "transform" deprecation status produce WriteBehavior::CurrentName so the generic renaming logic leaves them alone.

SDK input formats are deserialized into typed Rust structs with serde aliases for field name variations across SDKs (OpenAI, Google GenAI, Anthropic, LiteLLM, Vercel AI, etc.), then mapped via From impls into canonical OTel output structs. When parsing fails, deprecated keys are kept in place and a gen_ai.transform.failed counter metric is emitted.

Contributes to TET-2587

linear-code · 2026-07-09T14:41:56Z

TET-2587

Dav1dde · 2026-07-13T15:22:12Z

@constantinius talked with the team, we'll look into dealing with the transaction normalization case in #6216 - Since you already removed the code for it, is this normalization required to run on transaction spans?

So to get you unblocked I would propose we merge this change for the span streaming pipeline and we as a team take over the transaction change, if it is necessary.

Also please remove the redundant information from the PR description, changed files and tests I can tell by the diff, even the functional changes I can see in code, the important information in the description and later commit is the motivation and the effect not the functional changes, which are already present in code.

loewenheim

This is a good start. Please feel free to ask if you need help with the requested changes.

Dav1dde · 2026-07-14T08:37:38Z

This shouldn't be necessary anymore, master has a more up to date version.

This sentry-conventions PR contains the transformations. We should point to that version, once its merged.

loewenheim

This is a lot better. I have two comments, otherwise I'm good with merging this. If you like I can also take care of these.

loewenheim · 2026-07-16T14:29:56Z

+/// A message in the old `gen_ai.request.messages` format.
+#[derive(Deserialize)]
+struct OldMessage {
+    #[serde(default)]


This isn't necessary for Option.

Suggested change

#[serde(default)]

loewenheim · 2026-07-16T14:37:30Z

+                    // and let the rest carry the original fields.
+                    return NewMessage {
+                        role: None,
+                        parts: vec![],


parts doesn't have skip_serializing_if. That means that in this case it will be serialized as parts: []. Not sure if that is intentional.

I've split up the messages struct to make this explicit. Hope this improves it. But parts shouldn't be empty anyways.

Implement the two attribute transformations from sentry-conventions PR #465: `gen_ai_request_messages_to_input_messages` and `gen_ai_response_to_output_messages`. These reshape attribute values beyond simple key renaming. `gen_ai.request.messages` with `content` fields is converted to `gen_ai.input.messages` with `parts` arrays. `gen_ai.response.text` (plain string, JSON string array, or objects with content) and `gen_ai.response.tool_calls` are combined into a single `gen_ai.output.messages` assistant message with typed parts. The transformation is generic over `AttributesLike` so it works on both `Attributes` (SpanV2) and `SpanData` (SpanV1) without duplication. It runs inside both `normalize_ai` (EAP pipeline) and `enrich_ai_span_data` (transaction/legacy pipeline). Attributes with the new `"transform"` deprecation status produce `WriteBehavior::CurrentName` so `normalize_attribute_names` leaves them alone — the dedicated transformation code handles the full move-and-reshape. Deprecated keys are always cleaned up, even when the canonical key already exists or the value cannot be parsed. Contributes to TET-2587

- Remove redundant #[serde(default)] on Option<OldContent> - Use TransformedMessage struct with skip_serializing_if for request message output instead of raw serde_json::Value - Split OutputMessage (response only) from TransformedMessage (request) - Messages without content field pass through unchanged without spurious empty parts array - Remove inline integration test module

Fix collapsible_if lint in gen_ai_transform content_to_parts. Update normalize_mobile_measurements snapshot for new app.vitals.stall.duration attribute from conventions bump.

Run the gen_ai attribute transformation inside normalize_attribute_names rather than from each AI normalization function separately. This ensures the transformation runs on every path that normalizes attribute names, without needing explicit call sites in normalize_ai and enrich_ai_span_data.

Dav1dde

The original code had a very loose conversion going with just using serde_json::Value for anything.

The suggestion was to type the effective schema which is actually sent and convert it into the schema of what will actually be produced. If any part fails to deserialize we know this is not something the product can understand and we can just fall back to keeping the original string and skipping the transform.
The PR now only addresses the original concern on a single level, keeping a lot of the loose conversion stuff still around.

Overall this allows the product to actually expect a certain schema Relay can enforce, but also is something we can build on and add more and more variants as we discover them with proper documentation.
It also means we won't be producing schema invalid messages according to otel.

Ideally in the end there is no more serde_json::Value except where the schema is expected to match 1:1 (parts) or must be extensible (not can be). For these cases we can also use &'a RawValue. serde_json::json! shouldn't be necessary at all.

I suggest starting with a single message you want to convert, add an integration test, type and implement the schema. Then go to the next message variant, write a new integration test, adjust the schema/types and so on.

Dav1dde · 2026-08-04T08:57:00Z


 mod ai;
 mod attribute_like;
+pub(crate) mod gen_ai_transform;


Why did you make this pub(crate)?

Dav1dde · 2026-08-04T08:59:02Z

+TEST_CONFIG = {
+    "outcomes": {
+        "emit_outcomes": True,
+    }
+}


Curious why you enabled outcomes, doesn't look like you're actually consuming them.

Dav1dde · 2026-08-04T09:02:36Z

+
+use super::attribute_like::{AttributeLike, AttributesLike};
+
+// --- Input models (what SDKs send) ---


Please do a human pass over these comments, if you need more organization for readability, we can also make more modules, though that doesn't seem to be necessary.

Dav1dde · 2026-08-04T09:03:30Z

+//!
+//! The functions are generic over [`AttributesLike`] so they work on both
+//! [`Attributes`](relay_event_schema::protocol::Attributes) (SpanV2) and
+//! [`SpanData`](relay_event_schema::protocol::SpanData) (SpanV1).


Seems redundant.

Dav1dde · 2026-08-04T09:06:36Z

+// --- Transformation logic ---
+
+/// Applies gen_ai attribute transformations.
+pub(crate) fn transform_gen_ai<T: AttributesLike>(attributes: &mut T) {


If you use other modules as a reference, you see that we don't use pub(crate) and if you follow the visibility rules of Rust, it's also not necessary.

Dav1dde · 2026-08-04T09:08:40Z

+struct OldMessage {
+    content: Option<OldContent>,
+    #[serde(flatten)]
+    rest: serde_json::Map<String, serde_json::Value>,


Why is this necessary, do you now have a schema which exactly describes what other fields are allowed?

Dav1dde · 2026-08-04T09:09:28Z

+#[serde(untagged)]
+enum OldContent {
+    String(String),
+    Parts(Vec<serde_json::Map<String, serde_json::Value>>),


Again here, curious why you fall back to Map<>, if the idea was to type out the Schema, this will also make the conversion logic easier.

Remove _meta from span 0 (invoke_agent, no deprecated attrs) and HTTP client spans (no deprecated attrs). Add _meta to gen_ai and tool spans that now produce deprecation remarks. Update deprecated attribute values to match current conventions behavior.

sentry · 2026-08-04T15:33:36Z

+enum SdkContentItem {
+    Tagged(SdkPart),
+    Untagged(SdkContentObject),
+    Generic(GenericPart),
+}


Bug: Deserializing a content item with an unknown type and a text field incorrectly matches SdkContentObject::Text, causing other fields in the object to be silently discarded.
_{Severity: HIGH}

Suggested Fix

Reorder the SdkContentItem enum variants to prioritize a generic catch-all before the more specific SdkContentObject. Alternatively, add #[serde(deny_unknown_fields)] to specific objects like SdkTextObject to prevent incorrect matches. Adding test coverage for this scenario is also recommended.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not valid. Location: relay-event-normalization/src/eap/gen_ai_transform.rs#L39-L43 Potential issue: The `SdkContentItem` enum uses `#[serde(untagged)]`, which deserializes variants in order. When an object with an unknown `type` and a `text` field (e.g., `{"type": "future_type", "text": "some text"}`) is processed, it fails to match any `SdkPart` variant. It then falls back to `SdkContentObject`, where it successfully matches `SdkContentObject::Text` because a `text` field is present. This causes all other fields, including the original `type`, to be silently discarded, leading to data loss and breaking forward compatibility with new SDK content types.

sentry · 2026-08-04T15:33:36Z

+    }
+}
+


Bug: The gen_ai.response.text attribute is removed even if it parses to zero parts (e.g., from "[]"), but the new gen_ai.output.messages attribute is not created, causing silent data loss.
_{Severity: MEDIUM}

Suggested Fix

The gen_ai.response.text attribute should only be removed if extract_text_parts successfully produces one or more parts and the new gen_ai.output.messages attribute is created. If no parts are produced, the original attribute should be preserved, similar to how unparseable tool_calls are handled.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not valid. Location: relay-event-normalization/src/eap/gen_ai_transform.rs#L551-L553 Potential issue: In `transform_response_to_output_messages`, the `gen_ai.response.text` attribute is unconditionally removed after being read. If the attribute's value is a JSON string that parses into zero content parts (e.g., an empty array `"[]"`), the function exits early without creating the new `gen_ai.output.messages` attribute. This results in the silent deletion of the original attribute without creating its replacement, causing data loss.

Remove _meta from span 0 (invoke_agent, no deprecated attrs) and HTTP client spans (no deprecated attrs). Add _meta to gen_ai and tool spans that now produce deprecation remarks. Update deprecated attribute values to match current conventions behavior. # Conflicts: # relay-event-normalization/src/eap/gen_ai_transform.rs

…able attributes When parsing fails, keep deprecated keys in place instead of moving them. Emit a gen_ai.transform.failed counter tagged with the transformation name (request_messages, response_text, tool_calls).

sentry · 2026-08-09T08:52:44Z

+    gen_ai_transform::transform_gen_ai(attributes);
+
    normalize_attribute_names_inner(
        attributes.as_object_mut(),
        relay_conventions::attribute_info_with_fragment,


Bug: Deprecated gen_ai attributes sent under an alias are ignored by both transformation and normalization logic, causing the data to be lost.
_{Severity: HIGH}

Suggested Fix

Modify the transform_gen_ai function to correctly handle aliases. Instead of directly looking up canonical attribute keys, iterate through the attributes and use a utility that resolves aliases (like attribute_info) to identify if an attribute corresponds to a target gen_ai field. Once identified, perform the transformation and remove the original aliased attribute.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not valid. Location: relay-event-normalization/src/eap/mod.rs#L585-L589 Potential issue: When a deprecated `gen_ai` attribute is sent by an SDK under one of its aliases, the attribute is neither transformed nor renamed. The `transform_gen_ai` function only looks for canonical names and misses the alias. Subsequently, the `normalize_attribute_names_inner` function is designed to skip attributes with a `Transform` status by using `WriteBehavior::CurrentName`, which prevents renaming. As a result, the data sent under the alias is effectively lost and not processed into the canonical format expected by downstream systems.

sentry · 2026-08-09T08:52:44Z

+    fn from(tc: SdkToolCall) -> Self {
+        OutputPart::ToolCall(ToolCallPart {
+            id: tc.id,
+            name: tc.name.unwrap_or_default(),


Bug: A missing name in a tool call is incorrectly converted to an empty string "" during serialization, instead of being omitted from the output.
_{Severity: MEDIUM}

Suggested Fix

Change the name field in the ToolCallPart struct to be an Option<String> and annotate it with #[serde(skip_serializing_if = "Option::is_none")]. Then, update the From<SdkToolCall> implementation to assign tc.name directly without calling unwrap_or_default(). This will ensure the name field is omitted from the JSON output when it is not present in the input.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not valid. Location: relay-event-normalization/src/eap/gen_ai_transform.rs#L404 Potential issue: If a tool call in `gen_ai.response.tool_calls` is received without a `name` or `toolName` field, it is deserialized into an `SdkToolCall` struct where the `name` field is `None`. During conversion to the `OutputPart` type, `tc.name.unwrap_or_default()` is called, which transforms `None` into an empty string `""`. Because the `name` field in the final `ToolCallPart` struct is a non-optional `String`, it is always serialized, resulting in `"name":""` in the output JSON instead of the field being omitted. This misrepresents the absence of a name.

cursor

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 210abcc. Configure here.}

cursor · 2026-08-09T08:56:37Z

+            parts: vec![OutputPart::Text(TextPart {
+                content: raw.to_owned(),
+            })],
+        });


Plain text counted as transform failure

Medium Severity

extract_response_text emits gen_ai.transform.failed whenever gen_ai.response.text is not valid JSON, then still successfully converts it to an assistant text part. Plain-text values are a common valid SDK shape, so this path is not a failure and will inflate the failure counter for normal traffic.

^{Reviewed by Cursor Bugbot for commit 210abcc. Configure here.}

cursor · 2026-08-09T08:56:37Z

+    role: Option<String>,
+    #[serde(default)]
+    content: Option<String>,
+}


Loose object match drops response data

Medium Severity

SdkResponseMessage makes both role and content optional with defaults, so in the untagged SdkResponseText enum almost any JSON object matches MessageObject. Objects without a string content field become an empty assistant message, and the original gen_ai.response.text value is removed.

Additional Locations (1)

relay-event-normalization/src/eap/gen_ai_transform.rs#L617-L619

^{Reviewed by Cursor Bugbot for commit 210abcc. Configure here.}

constantinius requested a review from a team July 10, 2026 09:51

constantinius marked this pull request as ready for review July 10, 2026 09:51

constantinius requested a review from a team as a code owner July 10, 2026 09:51

cursor Bot reviewed Jul 10, 2026

View reviewed changes

Comment thread relay-event-normalization/src/eap/gen_ai_transform.rs

Comment thread relay-event-normalization/src/eap/gen_ai_transform.rs

sentry Bot reviewed Jul 10, 2026

View reviewed changes

Comment thread relay-event-normalization/src/eap/gen_ai_transform.rs Outdated

constantinius changed the title ~~feat(eap): implement gen_ai attribute transformations~~ feat(eap): Implement gen_ai attribute transformations Jul 10, 2026

Dav1dde requested changes Jul 10, 2026

View reviewed changes

Comment thread relay-server/src/processing/legacy_spans/store.rs Outdated

Comment thread relay-event-normalization/src/eap/ai.rs Outdated

Comment thread relay-server/src/processing/transactions/spans.rs Outdated

cursor Bot reviewed Jul 10, 2026

View reviewed changes

Comment thread relay-event-normalization/src/eap/ai.rs Outdated

sentry Bot reviewed Jul 10, 2026

View reviewed changes

Comment thread relay-event-normalization/src/eap/gen_ai_transform.rs

Comment thread relay-event-normalization/src/eap/gen_ai_transform.rs Outdated

cursor Bot reviewed Jul 10, 2026

View reviewed changes

Comment thread relay-event-normalization/src/eap/gen_ai_transform.rs

sentry Bot reviewed Jul 10, 2026

View reviewed changes

Comment thread relay-event-normalization/src/eap/ai.rs

sentry Bot reviewed Jul 13, 2026

View reviewed changes

Comment thread relay-event-schema/src/protocol/span.rs Outdated

Comment thread relay-event-normalization/src/eap/gen_ai_transform.rs Outdated

cursor Bot reviewed Jul 13, 2026

View reviewed changes

Comment thread relay-event-normalization/src/eap/ai.rs Outdated

constantinius requested a review from Dav1dde July 13, 2026 12:10

Dav1dde mentioned this pull request Jul 13, 2026

Basic abstraction over Spans to support shared normalizations #6216

Closed

loewenheim requested changes Jul 14, 2026

View reviewed changes

Comment thread relay-conventions/build/attributes.rs Outdated

Comment thread relay-event-normalization/src/eap/gen_ai_transform.rs Outdated

Comment thread tests/integration/test_ai.py

Comment thread relay-event-normalization/src/eap/gen_ai_transform.rs Outdated

Dav1dde reviewed Jul 14, 2026

View reviewed changes

Dav1dde mentioned this pull request Jul 15, 2026

ref(attributes): Introduce AttributeLike for shared normalizations #6235

Merged

constantinius force-pushed the constantinius/feat/eap/gen-ai-attribute-transformations branch 2 times, most recently from 5b1564d to 8f8a074 Compare July 16, 2026 08:39

cursor Bot reviewed Jul 16, 2026

View reviewed changes

Comment thread relay-event-normalization/src/eap/gen_ai_transform.rs

constantinius requested review from Dav1dde and loewenheim July 16, 2026 12:06

loewenheim reviewed Jul 16, 2026

View reviewed changes

sentry Bot reviewed Aug 3, 2026

View reviewed changes

Comment thread relay-event-normalization/src/eap/gen_ai_transform.rs Outdated

constantinius requested a review from loewenheim August 3, 2026 11:30

sentry Bot reviewed Aug 3, 2026

View reviewed changes

Comment thread relay-event-normalization/src/eap/gen_ai_transform.rs Outdated

sentry-warden Bot reviewed Aug 3, 2026

View reviewed changes

Comment thread relay-event-normalization/src/eap/ai.rs Outdated

constantinius force-pushed the constantinius/feat/eap/gen-ai-attribute-transformations branch from ad98c57 to 063a813 Compare August 3, 2026 11:59

sentry Bot reviewed Aug 3, 2026

View reviewed changes

Comment thread relay-event-normalization/src/eap/gen_ai_transform.rs Outdated

constantinius mentioned this pull request Aug 3, 2026

chore(conventions): Bump sentry-conventions to include transform status #6281

Merged

constantinius added 3 commits August 4, 2026 09:39

fix: clippy collapsible_if and update mobile measurements snapshot

0214391

Fix collapsible_if lint in gen_ai_transform content_to_parts. Update normalize_mobile_measurements snapshot for new app.vitals.stall.duration attribute from conventions bump.

constantinius force-pushed the constantinius/feat/eap/gen-ai-attribute-transformations branch from 06f26cd to 0214391 Compare August 4, 2026 07:40

sentry Bot reviewed Aug 4, 2026

View reviewed changes

Comment thread relay-event-normalization/src/eap/gen_ai_transform.rs Outdated

constantinius force-pushed the constantinius/feat/eap/gen-ai-attribute-transformations branch from da5df35 to 298d646 Compare August 4, 2026 08:46

cursor Bot reviewed Aug 4, 2026

View reviewed changes

Comment thread CHANGELOG.md Outdated

constantinius force-pushed the constantinius/feat/eap/gen-ai-attribute-transformations branch from 298d646 to 48fa42d Compare August 4, 2026 08:55

fix: remove duplicate changelog entry from 26.7.2

e89603c

Dav1dde requested changes Aug 4, 2026

View reviewed changes

cursor Bot reviewed Aug 4, 2026

View reviewed changes

Comment thread relay-event-normalization/src/eap/gen_ai_transform.rs

sentry-warden Bot reviewed Aug 4, 2026

View reviewed changes

Comment thread relay-event-normalization/src/eap/gen_ai_transform.rs Outdated

constantinius force-pushed the constantinius/feat/eap/gen-ai-attribute-transformations branch from d76f1bf to 193a8ca Compare August 4, 2026 15:30

constantinius marked this pull request as draft August 4, 2026 15:30

sentry Bot reviewed Aug 4, 2026

View reviewed changes

cursor Bot reviewed Aug 4, 2026

View reviewed changes

Comment thread relay-event-normalization/src/eap/gen_ai_transform.rs

Comment thread relay-event-normalization/src/eap/gen_ai_transform.rs

sentry-warden Bot reviewed Aug 4, 2026

View reviewed changes

Comment thread relay-event-normalization/src/eap/gen_ai_transform.rs Outdated

constantinius force-pushed the constantinius/feat/eap/gen-ai-attribute-transformations branch from 113c5d5 to 369d0bc Compare August 7, 2026 16:15

constantinius force-pushed the constantinius/feat/eap/gen-ai-attribute-transformations branch from 369d0bc to 311a713 Compare August 7, 2026 16:20

constantinius requested a review from Dav1dde August 9, 2026 08:48

constantinius marked this pull request as ready for review August 9, 2026 08:48

sentry Bot reviewed Aug 9, 2026

View reviewed changes

cursor Bot reviewed Aug 9, 2026

View reviewed changes


		use super::attribute_like::{AttributeLike, AttributesLike};

		// --- Input models (what SDKs send) ---

Uh oh!

Conversation

constantinius commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

linear-code Bot commented Jul 9, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Dav1dde commented Jul 13, 2026

Uh oh!

loewenheim left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

loewenheim left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Dav1dde left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

sentry Bot Aug 4, 2026

Choose a reason for hiding this comment

Uh oh!

sentry Bot Aug 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

constantinius commented Jul 9, 2026 •

edited

Loading