[Microsoft.Extensions.AI] AIFunctionFactory does not tolerate a JsonElement of ValueKind String as a JSON-stringified argument (gap in #6572)

[flaviocdc]

Package: Microsoft.Extensions.AI.Abstractions 10.4.1 (code unchanged on main at tag v10.4.1)

Summary

#6572 ("AIFunctionFactory: tolerate JSON string function parameters", fixing #6544) made
AIFunctionFactory's parameter marshaller tolerant of a parameter whose value is a JSON string
wrapping the real argument (e.g. "{\"a\":1}" for an object parameter). That tolerance
(IsPotentiallyJson + MarshallViaJsonRoundtrip) is only reached when the argument value is a
raw .NET string.

When the same JSON-stringified object arrives as a JsonElement whose ValueKind == String,
the marshaller takes the earlier JsonElement arm and calls
JsonSerializer.Deserialize(element, typeInfo) directly, which throws before the tolerant path runs.

This is the exact shape produced when a model double-encodes a tool-call argument (emits
"input":"{...}" instead of "input":{...}) and the arguments JSON is parsed into JsonElements
(as FunctionInvokingChatClient-backed pipelines do). The failure is intermittent from the model's
side — the model usually emits the correct nested object and only occasionally stringifies it — but
whenever that shape occurs, the deserialization failure below is deterministic. A raw string with the
identical content is handled fine; a JsonElement string is not.

https://github.com/dotnet/extensions/blob/v10.4.1/src/Libraries/Microsoft.Extensions.AI.Abstractions/Functions/AIFunctionFactory.cs#L889

JsonElement element => JsonSerializer.Deserialize(element, typeInfo),   // throws for ValueKind == String
JsonDocument doc    => JsonSerializer.Deserialize(doc, typeInfo),
JsonNode node       => JsonSerializer.Deserialize(node, typeInfo),
_ => MarshallViaJsonRoundtrip(value),                                   // <- IsPotentiallyJson tolerance lives here

Repro (net9.0, Microsoft.Extensions.AI.Abstractions 10.4.1)

using System.Text.Json;
using Microsoft.Extensions.AI;

public sealed class Input { public string A { get; init; } = ""; public string B { get; init; } = ""; }

var func = AIFunctionFactory.Create((Input input) => "ok", new AIFunctionFactoryOptions { Name = "f" });
string inner = "{\"a\":\"x\",\"b\":\"y\"}";

// A: JsonElement of ValueKind String  -> THROWS
await Invoke(JsonSerializer.SerializeToElement(inner));
// B: raw .NET string (same content)   -> OK  (tolerated by #6572)
await Invoke(inner);
// C: JsonElement of ValueKind Object  -> OK
await Invoke(JsonSerializer.Deserialize<JsonElement>(inner));

async Task Invoke(object value)
{
    try { Console.WriteLine("OK: " + await func.InvokeAsync(new AIFunctionArguments { ["input"] = value })); }
    catch (Exception ex) { Console.WriteLine(ex.GetType().Name + ": " + ex.Message); }
}

Output:

JsonException: The JSON value could not be converted to Input. Path: $ | LineNumber: 0 | BytePositionInLine: 17.
OK: ok
OK: ok

Expected

A JsonElement of ValueKind == String whose content decodes to a valid argument should be
tolerated the same way a raw string is (parity with #6572), rather than throwing JsonException.

Actual

System.Text.Json.JsonException: The JSON value could not be converted to <T>. Path: $ thrown from
ObjectDefaultConverter<T>.OnTryRead via the GetParameterMarshaller delegate. In a
FunctionInvokingChatClient pipeline this surfaces as a failed tool invocation.

Why this affects real first-party clients (not a synthetic-only case)

The failing shape — an argument stored as a JsonElement of ValueKind == String — is exactly what
first-party IChatClient bridges produce, because they parse tool-call arguments into an
IDictionary<string, object?> / Dictionary<string, object?>, and System.Text.Json boxes every
value in such a dictionary as a JsonElement. So when a model double-encodes a tool argument, that
value arrives as a JsonElement{ValueKind: String} and hits the throwing arm above — the #6572
tolerance (raw string only) never runs.

Two public first-party bridges do exactly this today:

• Microsoft.Extensions.AI.OpenAI — OpenAIResponsesChatClient maps a FunctionCallResponseItem
  via OpenAIClientExtensions.ParseCallContent(...), which calls
  FunctionCallContent.CreateFromParsedArguments(json, ..., json => JsonSerializer.Deserialize(json, OpenAIJsonContext.Default.IDictionaryStringObject)).
  Target type IDictionary<string, object> ⇒ values are JsonElement.
  https://github.com/dotnet/extensions/blob/main/src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAIResponsesChatClient.cs
  (the case FunctionCallResponseItem: arm)

• Official Anthropic .NET SDK — AnthropicClientExtensions maps a ToolUseBlock via
  FunctionCallContent.CreateFromParsedArguments(element.GetRawText(), ..., json => JsonSerializer.Deserialize(json, ...GetTypeInfo(typeof(Dictionary<string, object?>)))).
  Target type Dictionary<string, object?> ⇒ values are JsonElement.
  https://github.com/anthropics/anthropic-sdk-csharp/blob/main/src/Anthropic/AnthropicClientExtensions.cs
  (the case ToolUseBlock toolUse: arm in ContentBlockValueToAIContent)

In both, a double-encoded argument reaches AIFunctionFactory's marshaller as a
JsonElement{ValueKind: String} and throws. Because both first-party bridges share this shape, a fix
in GetParameterMarshaller repairs every provider at once. Note the divergence with Semantic Kernel
(the original #6572 reporter): SK's KernelArguments carry raw .NET string values — the one shape
#6572 already tolerates — so the same logical defect was fixed for SK but not for the
JsonElement-string shape the OpenAI/Anthropic bridges produce.

Suggested fix

In the JsonElement arm of GetParameterMarshaller, when element.ValueKind == JsonValueKind.String
and the target type is not string, attempt to re-parse element.GetString() as JSON before
deserializing — mirroring the existing IsPotentiallyJson / MarshallViaJsonRoundtrip handling used
for raw strings.

[jeffhandley]

As @eiriktsarpalis pointed out in #7630 (comment), even if we did tolerate the outer parameter, that would leave open the possibility of a nested part of the object being encoded, and we certainly don't want to get into recursive forgiveness.

Rather than trying to tolerate malformed arguments, we should look for opportunities to improve the error handling here so that the LLM can self-correct.

@flaviocdc After the exception is thrown, it's unclear to me what happens to the end-to-end scenario thereafter. Does that error make it back to the model? Is the model given a chance to retry the tool call? Is it given enough information to understand why the tool call failed? Is it able to self-correct?

If the tool call fails and the model isn't given the opportunity or information needed to retry with corrected formatting, I'd be supportive of addressing that gap rather than forgiveness on the parameter encoding.

[flaviocdc]

Thanks @eiriktsarpalis @jeffhandley — that reasoning makes sense to me, and I agree the marshaller isn't the right place to be forgiving of malformed arguments. The nested double-encoding gap and the risk to types that legitimately serialize as JSON strings (Guid, DateTimeOffset, custom converters) are convincing, and "we can't anticipate every malformation an LLM might produce" is fair.

One thing I looked at on the error-handling angle: today the exception is caught, turned into a FunctionResultContent, and the loop continues back to the model — but with the default IncludeDetailedErrors = false the model just receives "Error: Function failed.", and in the trace I looked at it didn't try to self-correct, it just moved on. So the retry path exists, but the signal the model gets is very thin.

I don't want to prescribe a fix here — you both have much better context on the intended design. Are there better ways you'd want to see this handled?

[flaviocdc]

Here's how the LLM chat thread looks like:

┌─ TURN 1 · chat claude-sonnet-4-6 (span 5904d86a…, 15.1s)
│
│  ▸ USER (orchestrator)  →  model
│      Reply-All / Incoming / Conversation IDs, Sender=Customer,
│      customer email: "Share to Teams dialog never prompts for sign-in"
│
│  ◂ ASSISTANT  ←  model   (finish_reason: tool_call)   [3 tool calls]
│      1. tool_call EngagementHandling
│           arguments: { "input": { "goal": …, "toolChoiceReasoning": … } }   ✅ nested object
│      2. tool_call ReplyAllWithFullThread
│           arguments: { "messageId": …, "introComment": … }                  ✅
│      3. tool_call schedule_email_followup   (id toolu_01PSgaRRPH55Cj6DpyrAB9vK)
│           arguments: { "input": "{\"agentReply\":\"<p>Dear Isabella…\", … }" }
│                                └──────────── ⚠ DOUBLE-ENCODED: value is a STRING, not an object
│
├─ TOOL EXECUTION (FunctionInvokingChatClient)
│      • EngagementHandling            → ok
│      • ReplyAllWithFullThread        → ok
│      • execute_tool schedule_email_followup   (span de4d9011…, 3.6ms)
│            ✗ System.Text.Json.JsonException:
│              "could not be converted to EmailFollowupToolInput. Path: $ | BytePositionInLine: 2979"
│            ↳ FICC CATCHES it (no rethrow) → builds FunctionResultContent
│            ↳ log: "schedule_email_followup invocation failed." (Error)
│            ↳ status Error — but the run CONTINUES
│
│  ▸ TOOL RESULT  →  model   (injected by FICC)
│      { tool_call_response, id: toolu_01PSgaRRPH55Cj6DpyrAB9vK,
│        response: "Error: Function failed." }   ← generic; no "malformed args / retry" hint
│                                                  (IncludeDetailedErrors = false)
│
└─ TURN 2 · chat claude-sonnet-4-6 (span a4c73192…)   [the model's reaction]
     ◂ ASSISTANT  ←  model   (finish_reason: tool_call)
         tool_call Recording
           arguments: { "input": { "actions": […], "decision": "InProgress → Resolved", … } }  ✅
         ✗ NO retry of schedule_email_followup — follow-up silently dropped
     … → Recording sub-steps → job ends  Status=Ok