Skip to main content
Saved

Pattern Resilience Intermediate

Response Regeneration

A model gives a different response each time, so people regenerate on purpose. If your app keeps each earlier attempt as a variant that users can switch back to, they can compare responses without losing the previous one.

Examples
React, Vue, Svelte and native web
Read
4 min

The problem

A user gets a response that’s almost right, with a good structure but the wrong tone. In a normal web app, there’d be nothing to do, because the same request returns the same response. A model gives a different response when you ask again, so it’s worth asking for another one.

The pattern

With regeneration, your app re-runs the same prompt and stores the new result next to the previous one, instead of replacing it. Each assistant turn stores a list of variants and the index of the one that’s on screen. Regenerating adds a new variant to the end of the list and makes it the one on screen, and a small ”‹ 2 / 3 ›” control lets the user page back and forth between attempts, so they never lose the previous response.

Problem

Some apps don’t have a regenerate option at all, so the user has to retype or rephrase the prompt to try again. Others have a regenerate button that overwrites the previous response, so if the new one is worse, the good-enough version they had is gone. The user might end up regenerating again and again, hoping the next one is better, with no way to get back to the one they liked.

Often nothing went wrong with the first response at all. The user just wants to compare two responses and keep the better one, and they can’t do that if the second one replaces the first.

Solution

Under the hood, regeneration goes through the same generation steps as the original response, so the Streaming Response, Stop Generation, and Retry patterns all apply. The only difference is where the result is stored. A Message Thread already identifies each turn by its id, so adding a variants array to the assistant message is a small change.

Regenerating reuses the exact same prompt. Editing a prompt and resubmitting it is a different action, because the prompt changes, so everything after it is out of date. The thread is cut off at that point and generation starts again from there, and the old branch is only kept if your app supports branching. If your API allows it, you can also nudge the model toward a different response when the user regenerates, by raising the sampling temperature or adding an explicit “try a different approach” instruction, so the new variant doesn’t come back almost the same as the last one.

Example

In these examples, all the attempts for a turn are stored in one assistant message, so the rest of the thread doesn’t change when the user regenerates.

Modeling Variants

An assistant message stores every attempt and the index of the one that’s on screen.

// An assistant turn with switchable attempts
// { id, role: 'assistant', variants: [{ id, content, status }], activeVariant: 0 }

function messageReducer(message, action) {
  switch (action.type) {
    case 'regenerate': // append a fresh, streaming variant and switch to it
      return {
        ...message,
        variants: [...message.variants, { id: crypto.randomUUID(), content: '', status: 'streaming' }],
        activeVariant: message.variants.length,
      };

    case 'token': // target the streaming variant by id, so paging away mid-stream is safe
      return {
        ...message,
        variants: message.variants.map(v =>
          v.id === action.variantId ? { ...v, content: v.content + action.chunk } : v
        ),
      };

    case 'switch': // page between attempts without losing any
      return { ...message, activeVariant: action.index };

    default:
      return message;
  }
}

Regenerate and Switch Between Attempts

function AssistantTurn({ message, onRegenerate, onSwitch }) {
  const { variants, activeVariant } = message;
  const active = variants[activeVariant];
  const many = variants.length > 1;

  return (
    <article className="assistant-turn">
      <div className="content">{active.content}</div>
      <footer className="actions">
        {many && (
          <div className="variant-pager" role="group" aria-label="Response versions">
            <button onClick={() => onSwitch(activeVariant - 1)} disabled={activeVariant === 0}>‹</button>
            <span>{activeVariant + 1} / {variants.length}</span>
            <button onClick={() => onSwitch(activeVariant + 1)} disabled={activeVariant === variants.length - 1}>›</button>
          </div>
        )}
        <button onClick={onRegenerate} disabled={active.status === 'streaming'}>
          Regenerate
        </button>
      </footer>
    </article>
  );
}

Nudging the Model to Differ

If each regeneration comes back with almost the same text, raising the sampling temperature or adding an explicit instruction can make the new variant different.

function regenerate(messages, previousAttempts) {
  return model.chat({
    messages,
    // Raise the temperature a little with each attempt, so the responses vary more.
    // Check your provider's allowed range: some cap temperature at 1.
    temperature: Math.min(0.7 + previousAttempts * 0.1, 1),
  });
}

Edit-and-Resubmit Truncates the Thread

Changing an earlier prompt makes everything generated after it out of date, so the thread is cut off at that turn and generation starts again from there.

function editAndResubmit(messages, editedId, newText) {
  const index = messages.findIndex(m => m.id === editedId);
  // Everything after the edited prompt is now stale
  const kept = messages.slice(0, index);
  return [...kept, { ...messages[index], content: newText }];
  // …then start a new generation from this truncated history
}

Benefits

  • Users can keep asking for a better response without retyping the prompt.
  • Because the app stores each attempt as a variant, regenerating doesn’t throw anything away, and the previous response is still a click away.
  • The pager lets users compare versions and pick the one they prefer, which is why they asked the model the same thing twice in the first place.
  • Regeneration reuses the streaming and stop code you already have, so most of the work is changing how the state is structured.
  • With edit-and-resubmit, users can fix their original prompt instead of arguing with the model in follow-up messages.

Tradeoffs

  • Each variant you keep is a full response stored in memory, and in storage too if you save conversations, so memory usage and storage use can add up in long conversations.
  • The pager adds UI and state that have to stay in sync while a new attempt is still streaming.
  • If you don’t nudge the model, regeneration can return almost identical responses, and the feature might feel broken to users.
  • Edit-and-resubmit has to delete the turns that came after the edited prompt, so users lose them unless you build full conversation branching.
  • Once a turn has several variants, you have to decide which one to save or export as the response, and the right choice isn’t always clear.

Summary

A model responds differently each time, so regenerating is a normal part of using it. Each regeneration is stored as a new variant next to the previous attempts, and a pager lets users compare them and choose one. The generation itself reuses your streaming and stop code, and a nudge to the sampling temperature or the prompt helps the variants come out different. Editing an earlier prompt is the one case where something gets thrown away, because the thread is deliberately cut off at that point.

See also

Next in Resilience Retry Retrying failed requests automatically with exponential backoff, so users don't see errors for temporary network problems.

A monthly email
from Den Odell.

What I'm thinking about frontend patterns, plus news when the site changes.

No spam. Unsubscribe anytime.