Skip to content
BlazorUI

AI scenarios

Streaming from a server

Bit.BlazorUI.AI Microsoft.Extensions.AI Server-sent events WebSocket

The model runs on the server, behind an IChatClient. The browser streams its reply over server-sent events, a WebSocket or SignalR, and neither side maps a thing by hand.

Live demo

Send anything: the demo's server streams the reply from a chat client that calls a function on the server - its clock - before it answers. Switch the transport: each keeps its own conversation.

\n**An IChatClient on the server, BitAiChat in the browser**: ```razor
``` ```csharp // Nothing: BitAiChat creates its session from the transport registered in the browser (Program.cs). ``` `Program.cs`: ```csharp // The Blazor app (WebAssembly, Server or Hybrid): dotnet add package Bit.BlazorUI.AI // Program.cs - the reply streams from the server's endpoint. One transport, either of these: builder.Services.AddBitBlazorUIAIServices(ai => { // Server-sent events: a POST per reply, its events read from the response. ai.UseSseTransport(sp => sp.GetRequiredService(), new() { Endpoint = "api/ai/chat" }); // Or a WebSocket per reply, which also carries the decisions on the tool calls the server holds. // ai.UseWebSocketTransport(sp => new Uri(new Uri(sp.GetRequiredService().BaseUri), "api/ai/ws")); }); ``` `Server/Program.cs`: ```csharp // The server: dotnet add package Bit.BlazorUI.AI.Server // dotnet add package Bit.BlazorUI.AI.ChatClient using Microsoft.Extensions.AI; // Program.cs - the provider's chat client, and the endpoints streaming its replies. builder.Services.AddChatClient(sp => new OpenAI.Chat.ChatClient("gpt-5-mini", apiKey).AsIChatClient()) .UseFunctionInvocation(); var app = builder.Build(); // Pinned by the server: a browser's request only fills in what these leave unset. var chatOptions = new ChatOptions { ModelId = "gpt-5-mini", MaxOutputTokens = 2048, Tools = [AIFunctionFactory.Create(() => DateTimeOffset.UtcNow.ToString("HH:mm:ss 'UTC'"), "get_server_time", "Reads the server's clock.")], }; app.MapBitAiSse("/api/ai/chat", (request, http, cancellationToken) => http.RequestServices.GetRequiredService().StreamBitAiEventsAsync(request, chatOptions, cancellationToken)) .RequireAuthorization(); // The same handler over a WebSocket. Authenticated by cookie, it lists the origins it serves. app.MapBitAiWebSocket("/api/ai/ws", (request, http, cancellationToken) => http.RequestServices.GetRequiredService().StreamBitAiEventsAsync(request, chatOptions, cancellationToken), new() { WebSockets = new() { AllowedOrigins = { "https://my.app" } } }) .RequireAuthorization(); ``` `AiHub.cs`: ```csharp // The server, as above. The browser adds: dotnet add package Bit.BlazorUI.AI.SignalR using Microsoft.Extensions.AI; // The same reply over SignalR: map the hub (app.MapHub("/hubs/ai")) and, in the browser, // ai.UseSignalRTransport(sp => sp.GetRequiredService()) with a connection to it. public sealed class AiHub(IChatClient chatClient) : BitAiHub { protected override IAsyncEnumerable StreamAsync(BitAiRequest request, CancellationToken cancellationToken) => chatClient.StreamBitAiEventsAsync(request, cancellationToken: cancellationToken); } ```

How it works

The reply crosses the network in the package's wire format: each BitAiEvent is one JSON object told apart by its type ("text.delta", "tool_call.started", ...), and the request is the transcript as the same message snapshots a saved conversation holds. BitAiWireSerializer reads and writes both, through metadata generated at compile time, so it is safe to trim and to compile ahead of time on both ends. A reader skips what it does not know - a property, or a whole event of a newer type - so a server can be updated before its clients.

  1. The browser's session hands each request to a BitAiSseTransport, which posts it to the server and reads the events from the text/event-stream response as they arrive.
  2. MapBitAiSse maps that endpoint. It reads the request back into live messages and streams whatever its handler produces - here StreamBitAiEventsAsync of the server's IChatClient, which turns the chat client's updates into events: text, reasoning, the function call and its result, the usage, the model id and the finish reason.
  3. The function here runs on the server, in the chat client's FunctionInvokingChatClient, and the browser shows its call and its result. A tool registered in the browser (BitAiRequest.Tools) is declared to the model without being invocable on the server, so its call comes back to the session, which runs it and asks again with the result.

The request comes from the browser, so it never overrides the server: the chat options the server passes win, and the request only fills in what they leave unset. Pin the model and the limits on the server. A system prompt the browser sends is dropped, and so is a tool choice forcing a tool it did not declare: the instructions and the server's functions are the server's alone. The endpoints read a JSON body only, so a page of another site cannot post to them as a user signed in by cookie without passing the app's CORS policy. A failure on the server, or an event the wire format cannot write, ends the stream with a terminal error event and a generic message - the exception is logged, never sent. The model behind this page is scripted, so it costs nothing to call; an app registers its provider's chat client instead, and nothing else changes.

Which package goes where

Each end installs only what it runs, so nothing of the server reaches the browser and nothing of the UI reaches the server:

  • The Blazor app installs Bit.BlazorUI.AI, which brings Bit.BlazorUI.AI.Core with the SSE and WebSocket transports. Bit.BlazorUI.AI.SignalR adds the SignalR transport, and Bit.BlazorUI.AI.ChatClient an IChatClient called right where the components run (Blazor Server, Hybrid).
  • The server installs Bit.BlazorUI.AI.Server - the SSE and WebSocket endpoints, the hub, the approval broker - and Bit.BlazorUI.AI.ChatClient to stream an IChatClient's reply. Neither has a component, a stylesheet or a script.

Over a WebSocket

MapBitAiWebSocket maps the same handler on a WebSocket endpoint, and UseWebSocketTransport (or a BitAiWebSocketTransport) opens one socket per reply to it: the request goes up as the first message, the events come down one message each, and the server closes the socket after the last one. Stopping the reply closes the socket, which cancels the handler. The address has to be absolute - the transport takes the app's http or https base address and speaks ws or wss.

What it adds is the way back: a decision on a tool call the server holds goes up the very socket the call came down on, so it reaches the server instance holding the call without sticky sessions, and only that client can answer it. A browser sends its cookies with a WebSocket from any page and CORS does not apply, so an endpoint authenticated by cookie lists the origins it serves (BitAiWebSocketEndpointOptions.WebSockets).

Over SignalR

The same reply streams over a SignalR connection the app already has: derive a hub from BitAiHub and produce the reply in its StreamAsync, then register UseSignalRTransport in the browser with the connection to it. The AiHub.cs tab shows the hub. The request and the events cross the hub as JSON values the package writes itself, so the hub protocol's settings change nothing.

Approvals the server holds

All three transports implement IBitAiApprovalTransport: when the server streams a BitAiToolApprovalRequestedEvent for a call it runs, the user's decision is sent back - to the endpoint of MapBitAiApprovals, up the WebSocket, or to the hub - and handed to the stream waiting for it through a BitAiApprovalBroker (await approvals.WaitAsync(toolCallId, cancellationToken)), in the same reply. The endpoints and the hub tell the broker about the call before they send its event, so a decision that arrives before the reply gets to WaitAsync is kept for it. The WebSocket and the hub take a decision only on a call their own connection was asked about, and only once.

Components used

Feedback

Found a mistake, a gap, or something that could be clearer? Every page and every component is one click from its source.