A language model doesn’t behave the way most frontend code expects. A normal request returns the same result every time, and the whole response arrives at once. A model response comes back a few words at a time over several seconds, and it’s different each time you ask. The user can also stop it partway through or ask for a new response. If you handle all of that with the same loading, success, and error states you’d use for a REST call, the interface can feel broken long before anything has actually gone wrong. This guide is about the states in between, and how to handle each one so the user knows what’s happening and stays in control while the model works.
Map the Generation Lifecycle
A generation goes through more states than a fetch does, and each one needs to look different on screen. In my experience, the states that get skipped, thinking and stopped, are the ones users notice.
- Name every state: A single generation is
idlebefore it starts,submittedonce the request is sent,thinkingwhile the model works before any text appears,streamingas tokens arrive, and finallycomplete,stopped, orerror. If the user regenerates a response, it goes back tosubmitted. If you treat all of this as one state machine, your rendering logic stays consistent. - Separate “sent” from “streaming”: There’s often a pause of a second or more between sending the request and the first token, and a reasoning model can think for much longer than that. Users are most likely to wonder whether something is wrong during that pause, so a generic spinner isn’t enough. The pause should get its own Thinking Indicator.
- Design the end states together:
complete,stopped, anderrorall end a generation, but each one needs different controls. After a finished message, the user will usually want to ask a follow-up. After a stopped one, they might want to start again, and after a failed one, they’ll want to retry. It’s worth deciding what each one looks like before you write the happy path.
Fill the Pause Before the First Token
Users often assume the app has locked up in the pause between submitting a prompt and seeing the first token, so something on screen needs to change right away.
- Acknowledge immediately: As soon as the request is sent, the user’s prompt should appear in the transcript, with a Thinking Indicator where the model’s reply will go. If nothing changes on screen until the first token arrives, even a fast model feels unresponsive.
- Announce it to assistive tech: A screen reader user gets nothing from a silent animation. If the thinking state is in a polite live region, the screen reader announces it, and the streamed text can replace it once it starts arriving.
Stream Tokens as They Arrive
Once text starts coming back, you can render it a piece at a time. A model that has produced three paragraphs shouldn’t look identical to one that has produced nothing.
- Render partial content as it arrives: With Streaming Response, tokens are added to the message as they come in, instead of the whole reply being buffered and shown at the end. Half-finished text is already useful to the user, so there’s no reason to hide it behind a spinner.
- Keep the latest text in view: Long replies scroll off-screen while they’re being generated. The transcript should stay anchored to the bottom as content grows, but it should stop following the moment the user scrolls up to read. Pulling the view back down while they’re reading is more annoying than letting the newest lines scroll out of sight.
- Parse incremental markdown carefully: Model output is usually markdown, and a half-finished code fence or list can render as broken markup if you re-parse on every token. It’s safer to render the text that’s complete, and leave out the unfinished fragment at the end until the rest of it arrives.
Give Users Control Over the Generation
Users will often want to stop a response or run it again, because it can take several seconds and come out different each time, so both should be easy to do.
- Always offer a stop button: While a generation is streaming, the send button should become a Stop Generation button, which calls
abort()on anAbortControllerso the underlying request is actually canceled. The model only stops generating if your server also cancels its own call to the model provider when the connection closes. If the stop button only hides the output while the model keeps generating tokens in the background, you’re still paying for those tokens, and the user thinks the request has stopped when it hasn’t. - Let users ask again: Because the response changes from one run to the next, users will often want Response Regeneration as a normal part of using the app, and not only when something goes wrong. They should be able to re-run the last prompt, and if you keep the previous response, they can compare the two instead of losing the first one.
- Lock the input while generating: A Prompt Input should disable submission while a response is in progress, but it should never disable the stop button. If submission stays enabled, users can send a second prompt while the first response is still streaming, and get two responses mixed together.
Handle AI-Specific Failures
AI requests fail in ways a JSON endpoint doesn’t, and several of those failures can happen mid-stream, after the user has already seen half a response.
- Recover from mid-stream breaks: A dropped connection can leave a message half-written. It’s better to keep the tokens that have already arrived and mark the message as interrupted, with an option to continue or regenerate, than to throw away text the user can already read.
- Treat refusals and moderation as normal output: A reply blocked by a content filter, or a refusal from the model, is still a valid response. It should appear as a message that explains what happened and what the user can do next, instead of a red error toast.
- Back off on rate limits and overload:
429and model-overloaded responses are common when the service is busy. Retry with exponential backoff can handle these, with a calm “the model is busy, retrying” message on screen instead of a hard failure. - Distinguish context-length errors: If a conversation gets too long for the model’s context window, retrying won’t help, and the app has to summarize or trim the history instead. That error should get its own message, separate from network errors.
Keep the Conversation Coherent
Each generation is part of a longer conversation, and where it appears changes how its states should look.
- Anchor state to the right message: In a Message Thread, each generation’s state belongs to a specific turn. If you track state per message, regenerating one response won’t affect any of the others.
- Distinguish a first response from a follow-up: A new conversation can show a larger empty state with suggested prompts, while a continuing one should stay focused on the transcript.
Test the Parts You Can’t Reproduce
Most of the states in an AI interface are hard to reproduce on demand, so it helps to build in hooks that let you trigger each one yourself.
- Drive the states from fixtures: If your components are wired to a fake stream that you control, it can emit tokens on a timer, stop early, stall before the first token, or fail mid-response. You should be able to see every state without calling a real model.
- Test slow and interrupted paths explicitly: If you develop locally against a fast, reliable model, you’ll rarely run into the states this guide is about. It’s worth writing test cases that check the thinking indicator appears and that stopping aborts the request. A mid-stream error needs its own test case too, which checks that the partial text is still there afterward.
Remember: An AI response has more states than a fetch. There’s a pause before the first token, then the text arrives a little at a time, and the user might stop it or ask for a new one at any moment. If you model each of those states explicitly, the interface stays responsive and the user stays in control. If you flatten them into loading/done, the app looks like it’s locked up while the response is being generated, and then the whole response appears at once, which is jarring.