Pattern Resilience Intermediate
Stop Generation
A stop button aborts the fetch with an AbortController but keeps whatever text has already streamed in. The server needs to stop generating too, so you’re not billed for tokens the user will never read.
The problem
The model got the question wrong and is now generating six paragraphs about the wrong thing. The user can watch it going wrong as the text streams in, but there’s no stop button, so they either wait for it to finish or refresh the page and lose the whole conversation.
The pattern
As a model response starts generating, you create an AbortController and pass its signal to fetch, then keep the controller while the response streams in. If the user clicks the stop button, your code calls controller.abort(), and whichever call is still in progress, the fetch or the next reader.read(), rejects with an AbortError. That ends the read loop, and your catch block should treat the AbortError as a normal stop, separately from real errors.
Problem
Even when there’s a stop button, it often only looks like it works. Clicking it might stop new text from appearing and re-enable the prompt input, but the request is still open, and the server is still generating tokens that you’re billed for.
A stop button is more useful on a model response than on most things in a UI. The response is slow, so the user has time to react, and because the model gives a different response each time, it often goes somewhere the user doesn’t want. You’re also billed for each token, so the tokens generated after the user has given up on a response still cost you money.
Solution
The tokens that already arrived should stay when the user stops a response. Your code can mark the message as stopped and leave its text in place, so the user keeps what they’ve read so far, and can use Response Regeneration if they want to try again. While the response streams, the stop button takes the place of the Prompt Input’s send button, and the send button comes back when the generation ends, whether it finished, was stopped, or failed.
Aborting the fetch stops the browser from reading the response, but on many setups the server keeps generating unless your code tells it to stop. Your server needs to detect when the connection closes, for example with res.on('close') in Express, and abort the upstream model call too. Without that, the user stops seeing new tokens, but you’re still paying for them.
Example
Each framework version below keeps the controller for the current response and has a stop function that aborts it.
Wiring the Abort
function useGeneration() {
const [text, setText] = useState('');
const [status, setStatus] = useState('idle'); // idle | streaming | complete | stopped | error
const controllerRef = useRef(null);
const start = async (prompt) => {
const controller = new AbortController();
controllerRef.current = controller;
setText('');
setStatus('streaming');
try {
const res = await fetch('/api/chat', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ prompt }),
signal: controller.signal,
});
const reader = res.body.getReader();
const decoder = new TextDecoder();
while (true) {
const { value, done } = await reader.read();
if (done) break;
setText(prev => prev + decoder.decode(value, { stream: true }));
}
setStatus('complete');
} catch (err) {
// Abort is an expected outcome, not an error
setStatus(err.name === 'AbortError' ? 'stopped' : 'error');
}
};
const stop = () => controllerRef.current?.abort();
return { text, status, start, stop };
} <script setup>
import { ref } from 'vue';
const text = ref('');
const status = ref('idle');
let controller = null;
async function start(prompt) {
controller = new AbortController();
text.value = '';
status.value = 'streaming';
try {
const res = await fetch('/api/chat', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ prompt }),
signal: controller.signal,
});
const reader = res.body.getReader();
const decoder = new TextDecoder();
while (true) {
const { value, done } = await reader.read();
if (done) break;
text.value += decoder.decode(value, { stream: true });
}
status.value = 'complete';
} catch (err) {
status.value = err.name === 'AbortError' ? 'stopped' : 'error';
}
}
function stop() { controller?.abort(); }
defineExpose({ text, status, start, stop });
</script> <script>
let text = $state('');
let status = $state('idle');
let controller = null;
export async function start(prompt) {
controller = new AbortController();
text = '';
status = 'streaming';
try {
const res = await fetch('/api/chat', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ prompt }),
signal: controller.signal,
});
const reader = res.body.getReader();
const decoder = new TextDecoder();
while (true) {
const { value, done } = await reader.read();
if (done) break;
text += decoder.decode(value, { stream: true });
}
status = 'complete';
} catch (err) {
status = err.name === 'AbortError' ? 'stopped' : 'error';
}
}
export const stop = () => controller?.abort();
</script> class Generation extends HTMLElement {
#controller = null;
async start(prompt) {
this.#controller = new AbortController();
this.textContent = '';
this.dataset.status = 'streaming';
try {
const res = await fetch('/api/chat', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ prompt }),
signal: this.#controller.signal,
});
const reader = res.body.getReader();
const decoder = new TextDecoder();
while (true) {
const { value, done } = await reader.read();
if (done) break;
this.textContent += decoder.decode(value, { stream: true });
}
this.dataset.status = 'complete';
} catch (err) {
this.dataset.status = err.name === 'AbortError' ? 'stopped' : 'error';
}
}
stop() { this.#controller?.abort(); }
}
customElements.define('ai-generation', Generation); The Stop Control
The button only appears while the response is streaming, and its label tells screen reader users what it does.
function GenerationControls({ status, onStop }) {
if (status !== 'streaming') return null;
return (
<button type="button" className="stop" onClick={onStop} aria-label="Stop generating">
<StopIcon aria-hidden="true" /> Stop generating
</button>
);
}
Keeping the Partial Response
Stopping a response shouldn’t clear the message. The message is marked as stopped and keeps its text, and there’s a button to regenerate it.
function AssistantMessage({ text, status, onRegenerate }) {
return (
<article data-status={status}>
<div className="content">{text}</div>
{status === 'stopped' && (
<footer className="stopped-note">
<span>Stopped.</span>
<button onClick={onRegenerate}>Regenerate</button>
</footer>
)}
</article>
);
}
Stopping the Work on the Server
Aborting the fetch closes the connection. The server can detect that and cancel the upstream model call, so the model stops generating too.
// Express handler proxying a streaming model API
app.post('/api/chat', async (req, res) => {
const upstream = new AbortController();
// When the browser aborts, the connection closes before the response
// finishes, so stop generating too
res.on('close', () => {
if (!res.writableFinished) upstream.abort();
});
try {
const modelStream = await model.chat({
messages: [{ role: 'user', content: req.body.prompt }],
signal: upstream.signal,
});
for await (const chunk of modelStream) {
res.write(chunk);
}
res.end();
} catch (err) {
if (err.name !== 'AbortError') throw err;
}
});
Benefits
- Users can stop a response that’s going the wrong way, instead of waiting for it to finish.
- If the server also cancels the model call, generation stops, so you don’t pay for tokens the user never reads.
- The partial text stays when the user stops, so they keep what was useful and can regenerate the rest.
- It uses the same reader and signal as Streaming Response, which you already have, so it takes very little code to add.
Tradeoffs
- Aborting the fetch doesn’t stop generation on the server, and it’s easy to forget the connection-close handling, because the UI looks exactly the same without it.
AbortErrorarrives as a rejected promise, so if your code doesn’t handle it separately, a normal stop shows up as an error.- There’s a race condition if the user clicks stop just as the last token arrives, so you need to decide whether that counts as
stoppedorcomplete. - A partial response that’s cut off mid-sentence can be misleading, so users need to be able to see clearly that it was stopped.
- These examples assume the send button is disabled while a response is streaming. If your app lets a new generation start before the last one ends, the old stream keeps adding its text to the new response, so
startshould abort the previous controller first and ignore any late updates from the old request.
Summary
Streaming model responses often go somewhere the user doesn’t want, and you pay for each token, so users should be able to stop them. You keep an AbortController while the response streams, call abort() when the user clicks stop, handle the AbortError as a normal stopped outcome, and keep the partial text. The server also has to cancel the upstream call when the connection closes, otherwise the button only stops new tokens from appearing, and you’re still paying for them.
See also
- Streaming Response — Resilience
- Response Regeneration — Resilience
- Prompt Input — Components
- Message Thread — Components
- AbortController - MDN
- AbortSignal - MDN
- Aborting a fetch - MDN