Allow controlling JSON formatting for gen_ai input/output message telemetry

[flaviocdc]

Summary

OpenTelemetryChatClient and related MEAI OpenTelemetry paths currently serialize gen_ai.input.messages / gen_ai.output.messages as indented JSON, and there does not appear to be a public knob to emit compact JSON instead.

For large prompts, responses, tool calls, tool results, or multimodal content, the extra whitespace increases span attribute size and can make exporter/backend size limits easier to hit.

Current behavior

The chat path uses OtelMessageSerializer.DefaultOptions, which is created by copying OtelContext.Default.Options:

private static JsonSerializerOptions CreateDefaultOptions()
{
    JsonSerializerOptions options = new(OtelContext.Default.Options)
    {
        Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping
    };

    options.TypeInfoResolverChain.Add(AIJsonUtilities.DefaultOptions.TypeInfoResolver!);
    options.MakeReadOnly();

    return options;
}

OtelContext is source-generated with WriteIndented = true:

[JsonSourceGenerationOptions(
    PropertyNamingPolicy = JsonKnownNamingPolicy.SnakeCaseLower,
    WriteIndented = true,
    DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull)]
internal sealed partial class OtelContext : JsonSerializerContext;

OtelMessageSerializer.SerializeChatMessages(...) then serializes the final OTel payload with those options/type metadata:

return JsonSerializer.Serialize(output, DefaultOptions.GetTypeInfo(typeof(IList<object>)));

OpenTelemetryChatClient.JsonSerializerOptions is passed as customContentSerializerOptions, but that only influences the fallback serialization path for unknown/custom AIContent instances:

customContentSerializerOptions?.TryGetTypeInfo(content.GetType(), out JsonTypeInfo? ctsi) is true ? ctsi :
DefaultOptions.TryGetTypeInfo(content.GetType(), out JsonTypeInfo? dtsi) ? dtsi :
null;

It does not affect the outer gen_ai.input.messages / gen_ai.output.messages JSON formatting, nor the built-in OTel message-part shapes.

The realtime path has the same effective behavior: it serializes message payloads through OtelContext.Default.IEnumerableRealtimeOtelMessage, which is generated from the same OtelContext with WriteIndented = true.

Why this matters

The gen_ai.input.messages and gen_ai.output.messages attributes can be large, especially when EnableSensitiveData is enabled or when OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true is set.

This matters for exporters/backends with size limits. For example, open-telemetry/opentelemetry-dotnet-contrib#873 discusses Geneva exporter string/event limits:

• Geneva exporter currently has a string field limit of 16 KiB / 16,383 UTF-16 characters.
• Maintainers note there is also an approximately 65,360-byte total event limit from underlying ETW / Linux user_events constraints.
• The 16 KiB per-string limit is described as artificial/protective, but the overall event-size limit is not easy to lift.
• Later discussion asks for configurable string-size limits because exceeding the total event size can lead to full event loss.

Given those constraints, always emitting pretty-printed JSON for potentially large GenAI message attributes consumes limited telemetry payload budget without adding semantic value for production pipelines. Compact JSON would reduce the risk of truncation or dropped events/spans and reduce storage/export cost.

Ask

Please consider adding a supported way to control this formatting. Possible approaches:

1. Add an option on OpenTelemetryChatClient / OpenTelemetryRealtimeClient to choose compact vs indented OTel message JSON.
2. Add a more general OpenTelemetry*Client serialization/formatting option for gen_ai.input.messages and gen_ai.output.messages.
3. If feasible, make existing JsonSerializerOptions influence final OTel message serialization, not only fallback serialization for custom/unknown AIContent.

The default could remain unchanged for compatibility. The key ask is to make compact JSON possible without post-processing Activity tags.

Current workaround

The only practical workaround today is an OpenTelemetry processor that rewrites or removes the message attributes before export, for example by parsing and reserializing gen_ai.input.messages / gen_ai.output.messages compactly in a BaseProcessor<Activity>.

[ibrahimErbilen]

Thanks for the detailed write-up. I was looking at this and wanted to share some thoughts — not 100% sure this is the right direction, but maybe it helps move the discussion forward.

The core problem, as I understand it: OtelContext is source-generated with WriteIndented = true, and OtelMessageSerializer.DefaultOptions inherits from it, so there's currently no way to get compact output without a post-processing BaseProcessor<Activity> workaround.

One possible approach would be to introduce a second source-generated context alongside OtelContext — something like OtelContextCompact — with WriteIndented = false, and derive a CompactOptions from it in OtelMessageSerializer similarly to how DefaultOptions is created today. Then SerializeChatMessages could accept an optional bool writeIndented parameter (defaulting to true to preserve existing behavior), and pass through either DefaultOptions or CompactOptions at the final JsonSerializer.Serialize call.

On the public API side, a property like WriteIndentedMessageJson on OpenTelemetryChatClientOptions (and the realtime equivalent) could expose this to callers without breaking anything.

I'm not sure whether this is the cleanest shape for the API, or whether it would be better to fold this into the existing JsonSerializerOptions parameter somehow. Also not certain about the realtime path — it goes through OtelContext.Default.IEnumerableRealtimeOtelMessage directly, so it might need a parallel fix.

For anyone hitting this in the meantime, the processor-based workaround from the issue description does work — parse and reserialize the tag in OnEnd:

public override void OnEnd(Activity activity)
{
    foreach (var key in new[] { "gen_ai.input.messages", "gen_ai.output.messages" })
    {
        if (activity.GetTagItem(key) is not string tag) continue;
        try
        {
            using var doc = JsonDocument.Parse(tag);
            activity.SetTag(key, JsonSerializer.Serialize(doc,
                new JsonSerializerOptions { WriteIndented = false }));
        }
        catch { }
    }
}

Happy to be corrected if I'm missing something about why a separate context wouldn't work here.