Structured output

\n**A table from partial JSON**: ```razor @implements IAsyncDisposable ``` ```csharp private readonly BitAiChatSession session = new(AiDemoContentTransports.Structured()); private readonly string[] suggestions = ["Compare our options"]; public ValueTask DisposeAsync() => session.DisposeAsync(); ``` `AiDemoContentTransports.cs`: ```csharp // AiScriptedTransport and AiScript are the demo's helpers - see the Basic chat scenario. public static IBitAiTransport Structured() => new AiScriptedTransport((request, token) => AiScript.Stream(token, async yielder => { await yielder(new BitAiMessageStartedEvent()); const string intro = "Four candidates, ranked by how much work each one would be.\n\n"; await foreach (var evt in AiScript.Write(intro, 4, 12, token)) await yielder(evt); const string json = """ [ {"option":"Server-side rendering","effort":"Low","risk":"Low","notes":"Fastest to ship"}, {"option":"WebAssembly","effort":"Medium","risk":"Low","notes":"Best offline story"}, {"option":"Hybrid per page","effort":"Medium","risk":"Medium","notes":"Two mental models"}, {"option":"Rewrite the front end","effort":"High","risk":"High","notes":"Not this quarter"} ] """; // Fourteen characters at a time: invalid JSON until the very last slice. foreach (var slice in AiScript.Slice(json, 14)) { await AiScript.Pause(70, token); await yielder(new BitAiStructuredDataDeltaEvent(slice) { SchemaName = "options" }); } await yielder(new BitAiMessageCompletedEvent()); })); ```

The transport streams the value as BitAiStructuredDataDeltaEvents, and for most of a generation the payload is syntactically invalid JSON. BitAiPartialJson repairs the truncated document on every delta: open containers are closed, an incomplete trailing key is dropped, and a partially written string value is kept.

Nothing is invented: every value in the repaired document was present in the input, which is why it is safe to bind straight to a typed model - part.As(typeInfo), with the JsonTypeInfo of the app's own JsonSerializerContext, which keeps it safe to trim. With no renderer registered for its schema, an array of objects is drawn as a table, an object as a list of labelled values, and anything else as a tree.

The SchemaName of the deltas (options here) names the shape, which is what picks a renderer registered for it (renderers.RegisterStructured(name, ...), as the Forms scenario does). A transport whose provider returns the whole value at once sends one BitAiStructuredDataEvent(value) instead.

\n**A response format in the request**: ```razor @implements IAsyncDisposable ``` ```csharp // The shape asked for: a JSON Schema, written by hand here, and the name the part is given. private static readonly BitAiResponseFormat OptionsFormat = BitAiResponseFormat.FromSchema(AiScript.Json(""" { "type": "array", "items": { "type": "object", "required": ["option", "effort", "risk"], "properties": { "option": { "type": "string" }, "effort": { "type": "string", "enum": ["Low", "Medium", "High"] }, "risk": { "type": "string", "enum": ["Low", "Medium", "High"] }, "notes": { "type": "string" } } } } """), "options"); // The request options go with every request of the session. private readonly BitAiChatSession session = new(AiDemoContentTransports.StructuredAsRequested(), options: new BitAiChatSessionOptions { RequestOptions = new BitAiRequestOptions { ResponseFormat = OptionsFormat }, }); private bool askForJson = true; private readonly string[] suggestions = ["Compare our options"]; // The options are an immutable record the session reads at the start of every request: a new one is the change. private void ApplyResponseFormat() => session.Options = session.Options with { RequestOptions = new BitAiRequestOptions { ResponseFormat = askForJson ? OptionsFormat : null }, }; public ValueTask DisposeAsync() => session.DisposeAsync(); ``` `AiDemoContentTransports.cs`: ```csharp // AiScriptedTransport and AiScript are the demo's helpers - see the Basic chat scenario. public static IBitAiTransport StructuredAsRequested() => new AiScriptedTransport((request, token) => AiScript.Stream(token, async yielder => { // What the request asks for. A real transport hands it to its provider, which holds the model to it. var format = request.Options?.ResponseFormat ?? BitAiResponseFormat.Text; await yielder(new BitAiMessageStartedEvent()); if (format.Kind is BitAiResponseFormatKind.Text) { const string text = """ No response format was asked for, so here is the comparison as text. | Option | Effort | Risk | Notes | | --- | --- | --- | --- | | Server-side rendering | Low | Low | Fastest to ship | ... """; await foreach (var evt in AiScript.Write(text, 5, 10, token)) await yielder(evt); await yielder(AiScript.Usage(request, text)); await yielder(new BitAiMessageCompletedEvent()); return; } var intro = $"Asked for {(format.Kind is BitAiResponseFormatKind.JsonSchema ? "JSON matching a schema" : "JSON")}, so here are the four candidates as data.\n\n"; await foreach (var evt in AiScript.Write(intro, 4, 12, token)) await yielder(evt); const string json = """ [ {"option":"Server-side rendering","effort":"Low","risk":"Low","notes":"Fastest to ship"}, ... ] """; // Named after the requested schema, which is what picks a renderer registered for it. foreach (var slice in AiScript.Slice(json, 14)) { await AiScript.Pause(70, token); await yielder(new BitAiStructuredDataDeltaEvent(slice) { SchemaName = format.Name }); } await yielder(AiScript.Usage(request, json)); await yielder(new BitAiMessageCompletedEvent()); })); ```

The model is constrained by the request, not by the UI. A session's RequestOptions (a BitAiRequestOptions, sent with every request) carry a ResponseFormat: BitAiResponseFormat.Text (free text, the default), BitAiResponseFormat.Json (any JSON), or BitAiResponseFormat.FromSchema(schema, name) (JSON matching a JSON Schema, the name some providers require) - its Kind, a BitAiResponseFormatKind, says which. The transport hands the format to its provider; this one, which plays the provider, answers with structured data named after the requested schema, and with a Markdown table when no format is asked for.

The toggle replaces the session's options (session.Options = session.Options with { ... }): they are an immutable record the session reads at the start of every request, so the next question goes out with the new format. The schema here is written by hand; BitAiTool.CreateSchema(MyJsonContext.Default.ListOption) derives one from the app's own type, the same one the value is read back into.