LlmClient.Stream — a model's answer as it is written

`LlmClient.Stream(prompt)` gives you a model's answer in the pieces it was produced in, so a reader sees it being written instead of waiting for it. It can only be read inside a `stream<T>` function, which is what passes those pieces on to whoever is watching.

Summary#

A model writes its answer a piece at a time. LlmClient.Complete(prompt) hides that — it hands you the finished text, and until it does, the reader has nothing. LlmClient.Stream(prompt) gives you the pieces, so a reader watches the answer being written.

Signature#

stream<string> LlmClient.Stream(string prompt)

Read only inside a stream<T> function, with a foreach that passes each piece on.

Description#

The whole shape is: read the model's pieces, pass each one on. A component then binds it with an ordinary live var and renders the answer as it appears (yield — a function that produces results over time).

It can only be read inside a stream<T> function, and anywhere else is a compile error that names the wrapper to write. That is not a style rule. An ordinary foreach reads its whole source before the first turn of the loop, because that list is what the platform writes down if the function pauses in the middle — and a model's answer cannot be written down half-finished. So the loop that reads one has to be the kind that never pauses, and a stream<T> function is exactly that kind.

The practical version: if you want a model's answer as it arrives, the thing that reads it is a stream<T> function. If you only want the finished text, use LlmClient.Complete(prompt).

A piece is whatever the model produced in one go — usually a few characters, sometimes a word or a fragment of one. Do not treat a piece as a word, a sentence or a token: concatenate them and you have the answer, exactly, and that is all that is promised. The pieces are not word-aligned and they carry their own spacing — a real reply came back as [The] then [ wire is live], with the space leading the second piece — so they join with no separator.

For a rendered reply, put string.Concat between the stream and the atom (Markdown — rendering markdown text): Markdown(string.Concat(answer), streaming: !answer.Done). One atom over the whole answer, not one per piece — a reply is a single document, and an atom per piece renders a paragraph break at every delta and splits any construct that spans two of them.

To STORE the finished answer, use on settled — run something once, when a stream finishes — on settled(answer) { … } runs once, when the stream stops, whichever way it ended. Reach for it rather than the shape it looks like you want: on change { if (answer.Done) … } fires again on every later render, because Done stays true once it is true, and in a real app that stored one reply fourteen times.

What it does not do:

No tools, no system prompt, no usage numbers. This is the raw completion, streamed — the same reduction LlmClient.Complete is of one call. A richer surface belongs with agents.
It needs a model provider. A host that has wired none fails the call rather than streaming nothing: an empty answer and a missing provider must not look the same.
Stopping the loop stops the generation. A reader who leaves, or a run the platform drops, stops the pieces being pulled — and the model stops producing, so you are not charged for the rest of an answer nobody wanted. You are still charged for the part it had already written. The provider bills for what it produced whether or not anyone was still reading, and the platform records the same — leaving early makes an answer cheaper, never free.

Choosing the model#

An app declares its model once (app.DefaultModel), and every call uses it. A call that needs a different one can say so:

foreach (var piece in LlmClient.Stream(question, "claude-opus-4-8")) { yield return piece; }

⚠ Naming a model can only NARROW what the app allows. The name is matched against the models the app admits; if it is not one of them the call is refused, with a message naming what was asked for and what the app allows. It is never quietly served by the default — a call site cannot know the app's policy, which is exactly why relaxing it is not the call site's to do.

Examples#

An assistant's reply, from the model to the page:

stream<string> Ask(string question) {
  foreach (var piece in LlmClient.Stream(question)) {
    yield return piece;
  }
}

component Reply(string Question) {
  live var answer = Ask(Question);

  render {
    Stack {
      Markdown(string.Concat(answer), streaming: !answer.Done);
      if (answer.Failed) { Text(answer.Error); }
      else if (!answer.Done) { Text("…"); }
    }
  }
}