Skip to content
BlazorUI

AI scenarios

Streaming

Bit.BlazorUI.AI Scripted transport

A long Markdown answer arriving a few characters at a time: headings, a table, a fenced code block, a quote - readable at every step, and never redrawn more often than the eye can follow.

Live demo

Send anything: the reply is a full Markdown document, written progressively, and the events it arrives as are listed beside the chat.

\n**A Markdown answer, streamed**: ```razor @implements IAsyncDisposable
``` ```csharp // A session of its own, over the transport this scenario streams from. private readonly BitAiChatSession session = new(AiDemoTransports.Markdown()); private readonly string[] suggestions = ["Compare the render modes"]; public ValueTask DisposeAsync() => session.DisposeAsync(); ``` `AiDemoTransports.cs`: ```csharp // AiScriptedTransport and AiScript are the demo's helpers - see the Basic chat scenario. public static IBitAiTransport Markdown() => new AiScriptedTransport((request, token) => AiScript.Stream(token, async yielder => { await yielder(new BitAiMessageStartedEvent()); await yielder(new BitAiMetadataEvent(new BitAiGenerationMetadata { ModelId = "demo-model-1", ContextWindow = 128_000 })); const string answer = """ Blazor renders in two places, and the difference matters. ## Where the code runs | Mode | Runs on | First paint | Best for | | --- | --- | :---: | --- | | Server | The server, over a circuit | Fast | Data-heavy internal apps | | WebAssembly | The browser | Slower | Offline and low-latency work | ... """; // A few characters every 10 ms, the way a model streams. await foreach (var evt in AiScript.Write(answer, chunk: 5, delayMs: 10, token)) await yielder(evt); await yielder(AiScript.Usage(request, answer)); await yielder(new BitAiMessageCompletedEvent()); })); ``` `AiEventLog.razor`: ```razor @implements IAsyncDisposable @* Every event the session applies, in order: a run of the same event is one line with a count. *@ @if (lines.Length == 0) {

Nothing yet. Send a message.

} else {
    @foreach (var line in lines) {
  • @line.Type@(line.Count > 1 ? $" × {line.Count}" : null)
    • }
} @code { private readonly List log = []; private Line[] lines = []; private BitAiRenderScheduler? renders; private BitAiChatSession? subscribed; [Parameter, EditorRequired] public BitAiChatSession Session { get; set; } = default!; // EventApplied is raised on the thread that applied the event, for every token: the log is drawn through a // scheduler, at most once every 100 ms, and only this component renders for it. protected override void OnInitialized() => renders = new BitAiRenderScheduler(() => InvokeAsync(Refresh), TimeSpan.FromMilliseconds(100)); protected override void OnParametersSet() { if (ReferenceEquals(subscribed, Session)) return; if (subscribed is not null) subscribed.Processor.EventApplied -= OnEventApplied; subscribed = Session; subscribed.Processor.EventApplied += OnEventApplied; } private void OnEventApplied(BitAiEvent evt) { lock (log) { // Every reply starts a log of its own. if (evt is BitAiMessageStartedEvent) log.Clear(); if (log.Count > 0 && log[^1].Type == evt.Type) log[^1] = log[^1] with { Count = log[^1].Count + 1 }; else log.Add(new Line(evt.Type, 1)); } renders?.Request(); } private void Refresh() { lock (log) lines = [.. log]; StateHasChanged(); } public async ValueTask DisposeAsync() { if (subscribed is not null) subscribed.Processor.EventApplied -= OnEventApplied; if (renders is not null) await renders.DisposeAsync(); } private sealed record Line(string Type, int Count); } ``` `Text pacing`: ```csharp // Every session paces text by default (BitAiTextPacingOptions.Default): a chunk every 16 ms, sized to drain what is // buffered over 150 ms, between 1 and 48 characters. One session can be given other settings... var session = new BitAiChatSession(transport, options: new BitAiChatSessionOptions { Pacing = BitAiTextPacingOptions.Default with { TargetDrain = TimeSpan.FromMilliseconds(300), MaxCharsPerTick = 24 }, }); // ...or none, to render the deltas exactly as the transport produces them. var unpaced = new BitAiChatSession(transport, options: new BitAiChatSessionOptions { Pacing = null }); // Every session the factory creates, in Program.cs. builder.Services.AddBitBlazorUIAIServices(ai => ai.ConfigureSessions(options => options with { Pacing = BitAiTextPacingOptions.Default with { TargetDrain = TimeSpan.FromMilliseconds(300) }, })); // A stream consumed outside a session - relayed by a server endpoint, say - is paced with the same operator. await foreach (var evt in transport.StreamAsync(request, cancellationToken).Paced(cancellationToken: cancellationToken)) { // Hand evt on. } ``` `The reducer`: ```csharp // The reducer a session applies every event with, on its own: recorded events replayed into a conversation, to test // a renderer or to show a reply a server pushed. var conversation = new BitAiConversation(); var processor = new BitAiConversationStreamProcessor(conversation); processor.BeginGeneration(); foreach (var evt in recordedEvents) processor.Apply(evt); processor.EndGeneration(); // What the framework has no event for, a transport sends as a custom event, its name namespaced... yield return new BitAiCustomEvent("acme:quota", 42); // ...which the reducer applies nothing for, but raises through EventApplied like every other event. session.Processor.EventApplied += evt => { if (evt is BitAiCustomEvent { Name: "acme:quota", Data: int remaining }) { quotaLeft = remaining; _ = InvokeAsync(StateHasChanged); } }; ```

How it works

The transport answers with a stream of events, listed beside the chat as the session applies them: BitAiMessageStartedEvent; a BitAiMetadataEvent naming the model and its context window, merged into the reply's metadata (the model name under the reply comes from it); a BitAiTextDeltaEvent per chunk of the answer; a BitAiUsageUpdatedEvent with the tokens it used; and BitAiMessageCompletedEvent.

Each text delta is appended to the reply's text part (BitAiTextPart, which stays IsStreaming until the reply completes). The Markdown is parsed again on every delta, but only its tail is: everything before the last confirmed block boundary is kept as it was, so a long answer costs the same per token at its end as at its start.

What is still open is drawn so it never flickers: a fenced code block is left unhighlighted until its closing fence arrives, and an unmatched ** is hidden rather than shown as a stray asterisk until its partner comes.

Only the part the text goes into re-renders as it streams, and at most once per 50 ms (BitAiUiOptions.RenderInterval) however many deltas arrive in between; the rest of the transcript is not drawn again for them, and the chat around it renders as the turn starts and ends. Streaming faster than that changes nothing a reader can see, and on Blazor Server every render is a diff sent down the circuit, so it costs real bandwidth.

Text pacing

Providers deliver text in bursts: nothing for a moment, then a paragraph. Drawn as it comes, that reads as stutter. So a session paces it: BitAiChatSessionOptions.Pacing is on by default (BitAiTextPacingOptions.Default), buffering the text deltas and releasing them at a steady rate - a chunk every Interval (16 ms), sized to drain what is buffered over TargetDrain (150 ms), between MinCharsPerTick (1) and MaxCharsPerTick (48) characters. Every other event passes through in order, behind the text that came before it; a terminal error is let through at once. Pacing adds no delay when the model is slower than the pacer, which is the common case, and never holds the end of a reply back by more than one drain window.

The text deltas counted beside the chat are the paced ones. Set Pacing to null to render the deltas exactly as the transport produces them, change it for every session with ai.ConfigureSessions, or pace a stream consumed outside a session with the same operator, BitAiEventStreamExtensions.Paced (stream.Paced(options, cancellationToken)) - the Text pacing tab shows all three.

Parts, and the events that address them

A message is an ordered list of parts, not a string: text, reasoning, code, images, files, tool calls, artifacts, structured data, each an IBitAiMessagePart with a PartId and a Kind that picks its view. A part that contributes words to the message's plain text - what copy and export use, and what a transport reads off the history with IBitAiMessage.GetText() - is an IBitAiTextualPart; one that grows while the reply streams is an IBitAiStreamablePart, whose IsStreaming lets its view skip expensive work (highlighting a half-written code block) until it is complete.

The events that fill a part are BitAiPartScopedEvents. A delta - text, reasoning, structured data - with no PartId, as this transport sends them, goes into the newest open part of its kind, created if there is none; a transport that interleaves several parts of one kind gives each delta the id of its part. BitAiPartStartedEvent(kind) opens a part explicitly, for what a delta cannot say: a code part's language and file name (BitAiMessagePartKinds.Code with the language and fileName Attributes), a Markdown part, a structured part's schema. BitAiPartCompletedEvent closes one, so it is drawn finished - a code part highlighted - before the whole reply is.

The other events each have a scenario of their own: reasoning (Reasoning), images and files (Attachments), citations and status lines (Citations), artifacts (Artifacts), structured data (Structured output), generated components (Generative UI), tool calls (Tool calling), agent runs (Multi-step agent) and errors (Errors and retry).

The reducer, and custom events

A session applies every event through its Processor, a BitAiConversationStreamProcessor: the reducer that turns events into parts, tool calls, artifacts and agent runs on the conversation, and the only code that knows the event vocabulary. Its EventApplied is raised for every event once it has been applied, on the thread that applied it - the log beside the chat is a component listening to it (AiEventLog.razor), drawn through a BitAiRenderScheduler, the same coalescer the AI components render with, so it renders at most once every 100 ms and the chat is not drawn again for it.

The reducer is usable on its own: new BitAiConversationStreamProcessor(conversation), then BeginGeneration(), Apply(evt) for each event and EndGeneration() replay recorded events into a conversation - to test a renderer, or to show a reply a server pushed rather than one a session asked for.

Information the framework has no event for travels as a BitAiCustomEvent(name, data), its name namespaced (acme:quota) so it never collides with another. The reducer applies nothing for it, but EventApplied still raises it, so the app handles it there - a quota shown in the page, or a custom part added to the reply (Processor.Current) and drawn by a renderer registered for its kind (see BitAiMessagePartView). The The reducer tab shows both.

Components used

Feedback

Found a mistake, a gap, or something that could be clearer? Every page and every component is one click from its source.