Web Dev to AI Frontend Engineer: Streaming Chat UX + Tools

The core shift is this: classic web UIs assume the backend returns complete, correct data. AI UIs assume the “backend” returns partial, probabilistic text that may change mid-flight and may need to be constrained. As an AI frontend engineer, you still care about layout, performance, accessibility, and state management—but you add new responsibilities: token streaming UX, interruption controls, message lifecycle state, tool result rendering, and safety-oriented affordances.

UI patterns unique to AI apps show up immediately. You need a “typing/streaming” state that is not just a spinner; the user expects to see incremental progress. You need an explicit cancel/stop action because responses can be long or wrong. You need retry and “regenerate” flows that preserve context but allow alternatives. And you need to represent uncertainty: the model may say “I’m not sure,” or it may produce conflicting content that must be presented with guardrails (e.g., disclaimers, citations, or constraints for certain domains).

Common mistakes when transitioning from web dev: treating the model as deterministic, over-trusting raw model output, and coupling UI state directly to the current text in the textarea. In AI chat, a single user submission can produce multiple events (assistant tokens, tool calls, tool results, final answer), and your state must survive network hiccups and user interrupts. Engineering judgment starts with choosing a reference stack that makes these flows explicit and observable. In this course we standardize on Next.js plus server routes so secrets stay server-side and streaming is controlled.

Before you write code, define the primitives your UI must support. A “turn” is not just one message bubble; it’s a transaction with stages. A user turn (input) is usually stable once sent. An assistant turn is often staged: created → streaming → completed, with possible branches like interrupted, errored, or needs_confirmation (for risky tool actions). Modeling this explicitly pays off later when you add streaming, retries, and tool use.

Context is also a UI concern. The model’s context window is limited, so you must decide what conversation history to send on each request. Even if you keep “all messages” in the client state, you might send only a summarized subset to the server. Your UI should communicate what the model is using: e.g., showing “Using last 10 messages” or providing a “Clear chat” action that resets context. A subtle but important outcome: user trust increases when the UI clarifies what the assistant remembers and what it does not.

Uncertainty shows up in three places: latency, correctness, and actionability. Latency is handled with streaming (visible progress) and interrupt (user agency). Correctness is handled with citations, tool grounding, and clear error states. Actionability is handled with confirmations: the assistant can propose an action (“I can create a ticket”) but must wait for the user to approve before invoking a tool. Even in this chapter’s baseline UI, structure your message type so it can later hold partial text, final text, and metadata like sources or tool calls. If you skip that now, you’ll repaint your state model under pressure later.

• Turn state: idle, streaming, completed, interrupted, error
• Message fields: id, role, content, createdAt, status, error, metadata
• UX controls: Send, Stop, Retry, Clear, Copy

Those primitives become your contract between UI and server and keep your implementation from devolving into ad-hoc conditionals.

You have two broad options: call the model provider directly from the browser (client-only) or route requests through your server (server-assisted). Client-only can be tempting for prototypes because it’s fewer files and you see network calls immediately. But it breaks down fast: you cannot safely ship provider API keys to browsers, you lose control over rate limiting, you can’t enforce consistent output constraints, and you struggle to add tool execution safely.

Server-assisted architecture—our reference approach—puts a Next.js route (or server action) in the middle. The browser sends a message payload to your route. The route reads secrets from environment variables, calls the model provider, and streams results back to the client. This adds a hop, but it centralizes the decisions that matter: authentication, rate limiting, prompt assembly, tool schema validation, logging, and output filtering.

Engineering judgment: use client-only only when you have a secure intermediary like a provider-issued ephemeral token with strict scope and short TTL, and even then expect to move server-side as soon as tools or guardrails appear. For most real products, server-assisted is the default because it enables safe execution and consistent observability.

Practical workflow tip: define a stable request/response contract early. Even in the baseline (non-streaming) stage, have the client post JSON like { messages: [...], model: "...", options: {...} } to a single route. Later, you can switch the route from returning a complete JSON response to streaming SSE without rewriting the client’s entire data model—only the transport layer changes.

Streaming is not optional in modern chat UX. Users interpret “nothing happening” as “broken.” You need incremental tokens, and you need the ability to stop generation. Two common transports are WebSockets and Server-Sent Events (SSE). WebSockets are bidirectional and powerful, but they add operational complexity (connection management, proxies, scaling, and stateful coordination). SSE is unidirectional (server → client), built on plain HTTP, easy to debug in browser devtools, and maps well to token streaming.

For a chat UI, the request direction is naturally HTTP POST (client → server) and the response direction is a stream (server → client). SSE fits that shape: you POST to a route that returns an event stream. The client listens and appends tokens as they arrive. Cancel can be handled by aborting the fetch on the client (with AbortController) and having the server respect disconnects.

Common mistake: choosing WebSockets by default because “chat apps use sockets.” Human-to-human chat benefits from bidirectional realtime updates; model streaming is mostly a single streamed response per request. Unless you need presence, shared rooms, or server push notifications independent of user actions, SSE is the simpler and more robust default.

• Use SSE when: single-response streams, simpler infra, easier debugging, HTTP-friendly deployments
• Use WebSockets when: multi-party realtime, server-initiated pushes, complex bidirectional protocols

In later chapters, SSE will also carry structured events (token, tool_call, tool_result, error, done). Planning for multiple event types now prevents you from treating “streaming” as “just text.”

Set up your project so you can iterate without leaking secrets or losing debuggability. In Next.js (App Router), create a server route under app/api/chat/route.ts. This file is your controlled boundary: it receives user messages, constructs the provider request, and returns either JSON (baseline) or an SSE stream (later). Keeping the provider call here ensures your API key stays on the server.

Environment variables: store provider keys in .env.local and never prefix them with NEXT_PUBLIC_ (that prefix exposes values to the browser). Add .env.local to .gitignore. Validate required env vars on server startup or at request time and return a clear 500 error if missing. This seems basic, but it prevents a huge class of “works on my machine” failures when teammates or CI run the app.

Secrets handling also affects logging. Log request IDs, timing, and high-level event types—but do not log full message content in production by default. A practical compromise is to log message counts, token counts, and truncated previews. In development, add a debug mode that prints more detail. Later, when you add tool execution, also log tool names and validation results (pass/fail) without logging sensitive arguments unless explicitly allowed.

Project skeleton checklist:

• Routes: / for UI, /api/chat for model proxy
• Types: shared message types in a lib/types.ts
• Config: .env.local with provider key; optional DEBUG_AI=1
• Safety baseline: basic rate limit placeholder and request size limits (even before guardrails chapter)

The practical outcome is a clean separation: the browser owns interaction and rendering; the server owns secrets, provider I/O, and later, validation and tool safety.

Build a baseline non-streaming chat UI first. This is your control group: you’ll use it to feel the difference once streaming and tool events are added. The UI should have three major regions: a scrollable transcript, a composer (textarea + send button), and a small status/control row (e.g., “Sending…”, “Stop”, “Retry”). Keep the layout resilient: long messages should wrap, code blocks should scroll horizontally, and the transcript should auto-scroll only when the user is already near the bottom.

Accessibility is not optional in chat. Use semantic roles and labels: the transcript can be a <section aria-label="Conversation"> with messages as list items; the composer needs a <label> (even if visually hidden). Ensure keyboard behavior: Enter to send, Shift+Enter for newline, and focus management after sending (return focus to textarea). Announce streaming updates later via an aria-live region, but in the baseline you can at least announce errors (“Request failed. Retry.”).

State shape matters more than you think. Avoid a single string for the assistant message; instead, store messages as an array of objects with stable IDs and a status field. Even before streaming, include fields you’ll need later: status, error, and meta (for citations/tool results). This prevents a rewrite when you add partial outputs and retries.

• Message: { id, role: 'user'|'assistant'|'system', content, status, createdAt, meta? }
• Request state: isSending, activeAssistantMessageId, lastError

Finally, establish your debugging workflow now: use browser network panels to inspect the POST to /api/chat, add server logs with a request ID, and capture timing (start/end). When something breaks later with streaming, you’ll rely on these habits: verify the route returns what you think, verify the UI updates state predictably, and keep rendering decoupled from transport details.

Section 2.1: Streaming mental model: tokens, chunks, and deltas

Streaming output arrives as a sequence of small payloads over time, but the units you see in logs (tokens) are not the same units you receive over the network (chunks). Tokens are model-internal; chunks are transport-level packets; deltas are the semantic “diff” you apply to your UI state. Keeping these concepts separate prevents common bugs like duplicated text, missing spaces, or broken Unicode characters.

In practice, an SSE stream usually sends events that contain a JSON object with a delta field (text to append) or a structured action (like a tool call). Your UI’s job is to apply deltas to a “draft assistant message” while preserving the final committed messages list. That implies two layers of state: (1) committed history for multi-turn context, and (2) an in-flight message with partial content and metadata (startedAt, lastChunkAt, status).

A useful discipline: treat streaming as an append-only log. You never “replace the whole message”; you append deltas until you receive an explicit “done” marker. If you need to support edits (e.g., re-ranking citations), represent them as separate event types rather than rewriting text. This mental model also makes retries safer: you can resume from a known committed state and discard a partial draft without corrupting the transcript.

• Token: model output unit; not guaranteed to align with characters.
• Chunk: bytes received; may contain partial JSON lines.
• Delta: parsed semantic increment (append text, set role, tool_call, etc.).

Common mistake: assuming each network chunk is a complete JSON object. Your parser must handle splitting across chunk boundaries. Another mistake: concatenating blindly and causing double-appends when reconnecting. Always attach an id (messageId / streamId) to the in-flight message so the UI knows exactly what it is updating.

Section 2.2: Server route streaming: headers, flush, and timeouts

An SSE endpoint is a normal HTTP response that stays open and emits text frames in the text/event-stream format. The minimal requirements are correct headers, no buffering, and a clean termination path. If you deploy behind proxies or serverless platforms, these details are the difference between “works locally” and “randomly stalls in production.”

At the route level (for example, a Next.js Route Handler), set:

• Content-Type: text/event-stream; charset=utf-8
• Cache-Control: no-cache, no-transform (avoid proxy transformations)
• Connection: keep-alive

Then write SSE frames like:

• event: delta
• data: {"delta":"Hello"}
• blank line to terminate the event

Engineering judgment: add heartbeats. Many platforms silently close “idle” connections. A tiny comment line every ~15 seconds (e.g., : ping\n\n) keeps the pipe open and gives you a timestamp to diagnose stalls. Also enforce timeouts: if the model provider stops producing tokens, you should abort upstream and end the SSE with an error event so the client can transition out of “typing.”

Flushing matters. Some runtimes buffer output until a threshold; if you don’t flush, the user sees nothing for seconds and your “streaming” looks like a single blob. In Node-based streaming, favor TransformStream/ReadableStream patterns that flush per enqueue. Finally, always send a terminal marker (e.g., event: done) and then close the stream. Clients rely on this to commit the draft message as final.

Common mistakes: forgetting no-transform (some proxies compress/merge chunks), not handling client disconnect (wasting tokens), and not propagating AbortSignals upstream. If the user hits stop, your server should stop the model request immediately.

Section 2.3: Client stream reader: incremental parsing and updates

On the client, your streaming reader has two responsibilities: (1) parse SSE incrementally from arbitrary byte chunks, and (2) apply parsed events to chat state without causing jank. The browser provides a ReadableStream from fetch; you decode bytes with TextDecoder and split events by the SSE delimiter (blank line). Because boundaries are arbitrary, you keep a rolling buffer string and only parse complete events.

A practical parsing approach:

• Start a request with fetch('/api/chat', { signal }).
• const reader = res.body.getReader() and loop reader.read().
• Append decoded text into buffer.
• While buffer contains \n\n, extract one event block and parse its event: and data: lines.
• JSON.parse the data payload and dispatch by event type (delta, error, done).

State modeling is the make-or-break decision. Represent an in-flight assistant message as { id, role:'assistant', content:'', status:'streaming'|'done'|'error', startedAt, tokens?:number }. When a delta arrives, append to content. When done arrives, mark status='done' and move it into your committed messages array. This separation lets you handle retries cleanly: on retry, discard the draft and start a new stream id without mutating history.

Stable layout is a UX feature, not a cosmetic detail. As content grows, maintain scroll anchoring: if the user is near the bottom, auto-scroll; if they scrolled up, don’t yank them down. Use a placeholder bubble for the assistant as soon as the request starts so the page doesn’t jump when the first token appears. That placeholder also supports typing indicators and perceived latency improvements.

Section 2.4: AbortController for stop/regenerate and UX rules

Stop/regenerate is where streaming becomes product-grade. Implement it with AbortController and a clear set of UX rules so the user always understands what happened to the partial text. The core mechanism: create an AbortController per in-flight request, pass signal into fetch, and keep a reference in state so a Stop button can call controller.abort().

Define your rules explicitly:

• Stop: abort the network request; keep the partial assistant text as-is; mark status as cancelled (or done with a “stopped” badge) so it’s clear why it ended.
• Regenerate: abort any current stream; create a new assistant draft; resend the same user prompt (and conversation context) with a new stream id.
• Send while streaming: either disable send (simple) or allow it but auto-stop the current stream first (more fluid). Pick one and keep it consistent.

Cancellation should propagate. When the client aborts, the server route should detect disconnect (or receive an AbortSignal if you forward it) and abort the upstream model call. Otherwise you pay for tokens that nobody will see. Also add “backpressure awareness”: if your UI can’t render as fast as tokens arrive, you should buffer deltas and apply them in batches (covered in Section 2.6) rather than blocking the read loop.

Common mistakes: immediately deleting partial text on stop (users often want to copy it), leaving the typing indicator on after abort, and allowing multiple concurrent streams that interleave into the same message. Concurrency control is easiest if your reducer checks that incoming events match the current streamId before applying them.

Section 2.5: Error states: partial output recovery and fallbacks

Streaming increases the surface area for failure: mid-stream 502s, timeouts, JSON parse errors, and provider rate limits. Your job is to convert these failures into understandable message states with safe recovery paths. Start by classifying errors into: (1) request never started (no response), (2) started but failed mid-stream (partial output exists), and (3) completed but post-processing failed (e.g., citations parse).

For mid-stream failure, keep the partial assistant message and mark it error. Add a small inline banner such as “Connection lost—partial response shown.” Provide two actions: Retry (resend the request) and Continue (send a follow-up user message referencing the partial text). Retrying should not duplicate the partial content: create a new draft message and keep the failed one as a separate transcript entry or replace it only if your product explicitly chooses that behavior.

Implement fallbacks when streaming is unavailable. Some environments buffer responses or strip SSE; detect this (e.g., missing text/event-stream content-type) and gracefully switch to a non-streaming response mode with a spinner. Also handle parse robustness: your incremental parser should ignore unknown event types, tolerate heartbeat comments, and fail “closed” (stop applying deltas) if data cannot be parsed, surfacing a clear error.

Finally, align error handling with guardrails you’ll expand later: rate limit errors should present a cooldown message; prompt injection detection should refuse with a safe response; and output constraints violations should end the stream with an error event that the UI can render as “formatting issue—regenerate.” The key is consistency: every terminal path must end typing indicators and leave the chat in a stable, actionable state.

Section 2.6: Performance: batching UI updates and avoiding re-render storms

Naively calling setState for every delta can cause hundreds of renders per response—especially with fast models—leading to input lag and scroll jank. The fix is to decouple “network read speed” from “UI paint speed.” Your stream reader can append deltas to an in-memory buffer immediately, then schedule UI updates at a controlled cadence (for example, every 16–50ms).

A practical pattern in React:

• Accumulate text in a mutable ref (pendingTextRef).
• Use requestAnimationFrame or a short setTimeout to flush the ref into state in batches.
• On done, force a final flush and commit the message.

This batching also helps with backpressure: you keep reading from the network (so buffers don’t grow uncontrollably), but you avoid overwhelming React. Combine batching with memoization: render each message bubble as a memoized component keyed by message id, and ensure only the in-flight message re-renders during streaming. A common mistake is storing the whole messages array in a way that causes every bubble to re-render on each token; reducers that replace the entire array each time can trigger that if child components aren’t memoized.

Typing indicators and stable layout affect perceived latency. Show the assistant bubble immediately, then a subtle “typing” state until the first delta arrives. Measure perceived latency with timestamps: record t_request_start, t_first_byte (when headers arrive), t_first_delta, and t_done. Often the biggest win is not model speed but eliminating buffering (server flush), reducing middleware overhead, and ensuring your UI paints the first delta quickly. Treat these metrics as part of your engineering definition of done for streaming.

Section 3.1: Message modeling: roles, ids, timestamps, metadata

A robust message schema is the difference between a demo and a product. Streaming makes this obvious: the assistant message starts empty, accumulates tokens, may pause for a tool call, and might never finish if the user stops it. Your schema must represent these transitional states without guessing.

Start with a normalized message shape that supports multiple content parts, not a single string. Even if you only render text today, tool calls and citations will become first-class content. A practical baseline is:

• id: stable UUID for the message.
• role: one of system, user, assistant, tool.
• createdAt and updatedAt: ISO timestamps for ordering and UI “edited” badges.
• status: draft | streaming | complete | cancelled | error.
• contentParts: array like {type: 'text'|'tool_call'|'tool_result'|'citation', ...}.
• parentId: for branching (regeneration or edit) without destroying history.
• conversationId: allows persistence and multi-tab sync.
• meta: model name, token counts, latency, safety flags, and UI hints.

Two common mistakes: (1) using array index as identity, which breaks when you insert or branch; and (2) storing assistant output as a single mutable string without status fields, which makes it hard to differentiate “still streaming” from “finished but empty” or “cancelled mid-way.”

In streaming, treat the assistant message as an entity whose content is appended by events. For example, SSE events might include delta tokens, tool_call payloads, and a terminal done event. Map those events into your message model deterministically: append text deltas into a text part, and create new parts when a tool call begins. When the user hits Stop, set status to cancelled and record why in metadata (e.g., meta.cancelReason = 'user_stop'). This clarity pays off later when you build retries, tool-result UX, and evaluation logs.

Section 3.2: Prompt assembly: truncation, summaries, and context windows

Prompt assembly is your “compiler.” It takes a conversation graph plus preferences and produces a linear set of messages for the model. If prompt assembly is ad-hoc, your system will feel inconsistent: sometimes it remembers, sometimes it forgets, and debugging becomes folklore. Good assembly is explicit about what it includes, what it summarizes, and what it drops.

Start by defining a prompt budget in tokens. You rarely know exact tokenization on the client, so implement a conservative heuristic (character count) client-side and enforce true token limits server-side. A practical workflow:

• Pick a target budget: e.g., 75% of model context window for history + system + tools, leaving 25% for the answer.
• Always include: current system instruction, user preferences, tool schemas (if using function calling), and the last user message.
• Include recent turns in reverse chronological order until you hit the budget.
• If older context matters, replace it with a summary message generated earlier (or on demand).

Summaries should be stored as separate messages with role system or a dedicated memory type in metadata, so you can audit them. Avoid “mystery memory” hidden in code. A common mistake is summarizing too aggressively and losing constraints (e.g., “use TypeScript” or “prefer concise answers”). Preserve stable preferences in a dedicated user-preferences object rather than relying on historical mentions.

For truncation, don’t chop arbitrary text mid-message; you may cut off a tool result or a safety constraint. Prefer turn-level truncation: drop the oldest whole turns first, then fall back to summaries. When tool results include long payloads, store the full data for UI, but send a compact version to the model (e.g., top rows, key fields, and a pointer/citation id). This keeps context windows healthy and reduces hallucinations from overwhelmed prompts.

Engineering judgment: make prompt assembly deterministic for a given conversation snapshot. That means: stable ordering by timestamps, stable branch selection (which path is “active”), and stable inclusion rules. Determinism enables caching, debugging, and regression testing later in Section 3.6.

Section 3.3: UI for editing: revise a turn and re-run downstream turns

Editing is where chat becomes a real work surface. Users will correct a requirement (“Actually, use PostgreSQL, not SQLite”), fix a typo that changed meaning, or replace an earlier message entirely. Your UX and state model must handle the consequence: everything after the edited turn may now be invalid.

There are two common strategies: overwrite and branch. Overwrite is simpler but destroys history and makes debugging impossible. Branching keeps the original message and creates a new message with parentId pointing to the original, then marks a new “active path” for the conversation. In practice, branching is worth it even in small apps because regeneration already creates branches.

A practical UI flow:

• User clicks “Edit” on a user message.
• You show an inline editor and a “Save & re-run” action.
• On save, create a new user message node (same position in the logical timeline, new id) and set it as active.
• Soft-delete or mark downstream assistant/tool messages as stale in metadata (don’t erase them).
• Trigger a new run from the edited message forward: assemble prompt from the active branch and stream a fresh assistant reply.

Common mistake: keeping downstream messages visible as if they’re still valid. Instead, render them with a “stale” banner and offer choices: “Re-run from here” or “Keep old branch.” This reduces confusion and makes branching feel intentional rather than buggy.

For regeneration, treat it as a special branch of an assistant message: same preceding context, new assistant node with parentId to the previous assistant message. Store generation parameters (model, temperature) in metadata so users can understand why results differ and you can reproduce issues. If you support interruptions, ensure the “Stop” action only cancels the active streaming message, not the entire conversation state. That’s easiest when your store tracks activeRunId and associates SSE events with that run.

Section 3.4: Persistence patterns: localStorage, DB, and session ids

Persistence is not just about “saving chat.” It’s about aligning identity across client, server, and streaming runs. The most important identifier is a conversationId, created once and reused for every message and tool result in that thread.

Three practical persistence tiers:

• localStorage (or IndexedDB): fastest to ship. Store conversations as JSON keyed by conversationId. Use it for drafts, quick prototypes, and offline-ish behavior.
• Server DB: required for multi-device sync, team features, analytics, and auditability. Persist messages, tool calls, and tool results. Store “active branch” pointers so the server can reconstruct the visible thread.
• Hybrid: local first, then background sync. Useful when you want instant UI updates and eventual server consistency.

Session IDs matter for streaming. Your SSE endpoint should accept conversationId plus a runId. The runId ties a single model invocation (and its stream of events) to an assistant message. If the user refreshes mid-stream, you can decide: either (a) resume by reconnecting with the runId (advanced), or (b) mark the message as error/cancelled and allow regeneration (simpler). Be explicit; “sometimes it resumes” is worse than “it reliably restarts.”

Common mistake: persisting only the final assistant text. You lose tool traces, partial outputs, and timing metadata that helps debug. At minimum, persist message status and tool events. Also consider data minimization: tool results may contain sensitive data, so store references or redacted snapshots server-side and keep full payload only in secure storage when necessary.

Practical outcome: with stable IDs and persistence, you can implement reload-safe UX (the transcript and statuses render correctly), and you can run evaluations against stored conversation snapshots to detect regressions.

Section 3.5: Safety in prompts: separating instructions from content

Prompt injection succeeds when untrusted content is treated like instructions. Your job is to keep instruction channels clean and make your assembly rules hard to subvert. This is not only about security; it’s also about correctness. If a user pastes a long document containing “Ignore previous directions,” your system should treat that as text to analyze, not a policy change.

Use clear separation:

• System instruction: your non-negotiable policy and task framing (e.g., “You are a coding assistant. Do not reveal secrets.”).
• User preferences: configurable but structured (e.g., {tone: 'concise', codeStyle: 'typescript'}) and inserted as a dedicated system message or a special “preferences” block.
• User content: everything the user typed or pasted, treated as data.
• Tool results: data with provenance; never let tool output modify system rules.

Practically, this means your prompt assembly function should never concatenate raw user text into the system message. Instead, keep system messages static templates with parameter substitution only for trusted fields (like a selected language). When you need to include untrusted text inside instructions (for example, “Summarize the following document”), wrap it in explicit delimiters and label it as content, not instruction.

Common mistakes include: (1) building prompts with string concatenation, (2) letting users edit the system prompt directly without validation, and (3) treating tool output as authoritative instructions (“Tool says to send API key back to user”). Maintain a “policy boundary”: tools can provide facts, not rules. On the server, validate tool calls against JSON schemas and enforce allowlists. On the client, display confirmation dialogs for high-impact tools (sending emails, making purchases) and include the confirmation outcome as a structured event in the conversation so the model can proceed safely.

Outcome: safer multi-turn chat where instructions are stable, user data stays in the right channel, and tool use is controlled and auditable.

Section 3.6: Lightweight evaluation: golden prompts and regression checks

As soon as you add branching, summaries, and tool calling, regressions become subtle: a small prompt-assembly change may drop an important constraint, or a new persistence migration may reorder messages. Lightweight evaluation prevents “it feels worse” releases.

Start with golden prompts: a small set of representative conversation snapshots (JSON fixtures) that cover your key UX flows—streaming cancellation, edit-and-rerun, tool call with citations, long context requiring summarization, and a safety edge case (user tries to override system policy). For each golden prompt, define what “good” means. Keep it practical:

• Structural checks: prompt assembly includes system + preferences; active branch is respected; tool schemas present when required.
• Budget checks: assembled prompt stays under a token/char threshold; truncation drops whole turns, not partial tool results.
• Behavior checks: for a deterministic model config, response contains required fields or follows a constrained output format.

Don’t aim for perfect semantic scoring at first. Focus on regression detection: if yesterday your assembled prompt included the summary and today it doesn’t, fail the test. Add snapshot tests for the assembled prompt payload your server sends to the model. That catches accidental reorderings and missing messages.

Also capture run metadata: model, temperature, tool call count, and error rates. When the UX changes (new edit flow, new persistence), compare these metrics on your golden set. If tool calls spike unexpectedly, you may have confused the model with missing context. If cancellations leave messages stuck in streaming, your SSE terminal event handling is wrong.

Outcome: you can iterate quickly—add memory strategies, improve safety boundaries, refine branching UI—without losing reliability. In later chapters, these same fixtures become your safety net as you add more tools and stricter guardrails.

Section 4.1: Why tools: reliability, grounding, and actions

Tools exist to close the gap between “sounds plausible” and “is correct.” Without tools, a model may invent citations, guess totals, or describe actions it never performed. With tools, you can ground answers in real data sources (search, database, internal APIs) and delegate computations (currency conversions, date math, scoring) to code that you control and can test.

There are three practical categories of tools you’ll implement in an AI frontend engineer role:

• Read tools: fetch information (search, docs lookup, DB query). These improve accuracy and let you attach citations.
• Transform tools: deterministic processing (summarize a document you already fetched, compute a diff, validate a form). These reduce token waste and lower hallucination risk.
• Act tools: side effects (create a ticket, send a message, place an order). These require confirmations and strong safety constraints.

In a streaming UX, tools also improve user trust. Instead of waiting for a long paragraph, the UI can show “Searching docs…”, “Calculating…”, or “Creating draft email…” as intermediate steps. This makes latency feel purposeful and allows interruption: if the user sees the assistant searching the wrong thing, they can stop it before any action happens.

Common mistake: treating tools as a hidden backend detail. If a tool influences the answer, your UI should reflect it—at least through a compact tool-result panel and, when relevant, citations. The outcome you want is a conversation that is both responsive (streaming tokens) and accountable (traceable tool calls).

Section 4.2: Tool contracts: schemas, enums, and constraints

A tool contract is a promise: “If you give me inputs in this exact shape, I will return outputs in that shape.” The model must follow the input rules, and your server must enforce them. This is why strict JSON schema validation is non-negotiable. You are not validating to help the model—you are validating to protect your system.

Define each tool with: a name, a human-readable description (for the model), and a JSON Schema for arguments. Use enums and constraints aggressively. If a parameter is a fixed set, use enum. If a string must be short, set maxLength. If an array can’t be huge, set maxItems. If a field must be present, list it in required. Disallow surprise fields with additionalProperties: false.

• Prefer enums over free text for modes like sort, timeRange, language, format.
• Use patterns for IDs (ticket keys, UUIDs) and dates (ISO strings) to stop “creative” arguments.
• Separate “user intent” from “execution details”: for example, allow the model to request query but not raw SQL.

On the implementation side (Node/Next.js), validate tool args with a schema library like Ajv or Zod (with JSON schema interop). Treat validation failures as expected runtime events: return a structured tool error that the model can recover from, and show a compact “Tool input invalid” message in the UI so users understand why the assistant paused.

Common mistake: making schemas too loose “so the model can be flexible.” Flexibility belongs in your prompt and dialog; execution belongs in strict contracts. The practical outcome is fewer hallucinated arguments, clearer retries, and safer defaults.

Section 4.3: Execution layer: allowlists, timeouts, and sandboxing

Once you have tool definitions, you need an execution layer that is boring, predictable, and safe. Think of it as a router: it receives a tool call request, validates arguments, checks permissions, runs the tool, and returns a structured result. This layer should live server-side (Next.js route handler, Node server, or edge function where appropriate) because it holds credentials and enforces policy.

Start with an allowlist of tool names. Never execute arbitrary functions based on a string without mapping it to known implementations. If the model requests an unknown tool, return a controlled error. Next, enforce timeouts per tool. A search call might time out at 8–10 seconds; an internal DB query at 3–5 seconds; a long job should be turned into an async task with polling.

• Sandbox external effects: tools that write data should use restricted service accounts and scoped tokens.
• Limit resources: cap response sizes, restrict network domains, and block filesystem access unless explicitly needed.
• Audit logs: record tool name, validated args, user ID, and outcome (success/error) for debugging and compliance.

In practice, your tool executor returns a typed envelope like { ok: true, data, meta } or { ok: false, error: { code, message, details } }. Keep errors structured, not prose. This matters because the model can react to code (e.g., RATE_LIMIT, VALIDATION_FAILED, NOT_AUTHORIZED) and attempt a safe next step, while the UI can render consistent states.

Engineering judgment: not every tool should run inline in a streaming request. If a tool can take long or can be retried later, consider queuing and streaming a “job created” event. The goal is to keep your SSE stream responsive while preserving correctness and safety.

Section 4.4: Tool-result UX: panels, citations, and structured output

Users should be able to tell when the assistant is “thinking” versus when it is “doing.” Tool-result UX is how you earn trust: show progress, intermediate steps, and sources without dumping raw JSON into the chat transcript.

A practical pattern is a tool panel per assistant message. While streaming, the panel can display a timeline:

• Queued → Running → Succeeded/Failed states
• Short labels like “Searching docs”, “Reading ticket #1234”, “Calculating estimate”
• Expandable details: validated arguments, duration, and a summarized result

For read tools, add citations as first-class UI elements: title, snippet, URL, and an optional “used in answer” marker. Then instruct the assistant to reference citations by index. This creates a tight loop: tool returns sources → UI renders sources → model responds with grounded text. If a tool fails, show a small error row with a retry button, and keep the assistant’s partial streamed text separate from the tool output to avoid confusing users.

Structured output matters here too. If your tool returns { items: [...] }, don’t render it as a blob. Map it to cards, tables, or inline badges. Your chat system already tracks message state (streaming, interrupted, retrying); extend that state model to include tool-call state so interruptions are clean: stopping the stream should cancel in-flight tool calls when possible, and the UI should reflect “Cancelled by user.”

Common mistake: hiding tool failures and letting the assistant “smooth over” errors in prose. Instead, surface a clear tool error and guide the user: “Search timed out—retry or narrow the query.” The outcome is a chat UX that feels dependable under real network conditions.

Section 4.5: Multi-tool workflows: chaining, retries, and idempotency

Real tasks rarely fit one tool call. You might: search docs → fetch a specific page → extract key fields → create a ticket. This is where multi-tool workflows can either feel magical or spiral into loops. Your system needs rules.

Chaining: let the model call tools step-by-step, but constrain it with a maximum number of tool invocations per user message (for example, 3–6). If it hits the limit, instruct it to ask the user a clarifying question or summarize progress. This prevents infinite “search again” loops.

Retries: separate “safe to retry” failures (timeouts, transient 503) from “don’t retry” failures (validation, authorization). Your executor should attach retry hints in the tool error envelope, like retryable: true and a backoff suggestion. On the UI side, show a Retry button that replays the exact same validated args, not a regenerated guess.

Idempotency is critical for action tools. If the model calls createTicket twice due to a stream reconnect or a client retry, you don’t want two tickets. Use idempotency keys derived from the conversation turn (user message ID + tool name) and store them server-side. If the same request arrives again, return the original result. For “send email” or “charge card,” require explicit user confirmation (next section) and bind the idempotency key to that confirmation event.

• Cap tool chains per turn.
• Make retries deterministic (replay validated args).
• Use idempotency keys for side effects.

Practical outcome: your app handles streaming reconnects, user interruptions, and transient failures without duplicating actions or losing the narrative thread of the conversation.

Section 4.6: Guarding against prompt injection targeting tools

Prompt injection becomes more dangerous when tools exist, because an attacker can try to trick the assistant into calling privileged functions or smuggling malicious arguments. Your defense is layered: the model prompt is only one layer, and it is not the strongest one. The strongest layers are schema validation, allowlists, permission checks, and confirmation gates.

Start by assuming tool inputs may be hostile even if they appear to come from the model. Validate strictly (Section 4.2), then apply policy checks based on user identity and context. For example: only managers can call approveInvoice; only the ticket owner can call closeTicket. Do not let the model “decide” permissions.

Next, build confirmation gates for risky actions. A confirmation gate is a UI step that pauses execution until the user explicitly approves. The assistant can draft the action (“I’m ready to send this email to 3 recipients”) and the UI shows a confirmation card with exact recipients and content. Only after the user clicks Confirm does the server execute the tool. This defeats many injections because hidden instructions can’t silently trigger side effects.

• Tool instruction isolation: treat retrieved text (web pages, documents) as untrusted data, not system instructions.
• Argument sanitization: strip or reject unexpected URLs, commands, or long payloads; cap lengths.
• Loop protection: detect repeated failing tool calls with the same args and force a user-visible stop.

Finally, align UI and backend: if the assistant claims it performed an action, it must be backed by an actual tool result. Teach your assistant to speak in conditional language until confirmation (“I can send… once you confirm”). The practical outcome is a tool-using assistant that remains useful under adversarial inputs, not just under happy-path demos.

Section 5.1: Trust UX: citations, grounded answers, and user control

Trust is earned at the moment the model might be wrong. In chat UIs, the most effective trust signals are: (1) grounding (what the answer is based on), (2) uncertainty (how confident it is), and (3) control (what the user can do next). Build these into the message layout instead of hiding them in a footer.

Citations and sources. If you use retrieval (RAG) or tool-based browsing, render citations as first-class UI elements. Good defaults: show 1–3 citations inline, with a “show all” affordance; display title + domain + timestamp; and link to the exact section (anchored URL) when possible. If your tool returns snippets, show the snippet on hover/focus and include a clear “Quoted from …” label. Common mistake: dumping raw URLs with no context—users can’t evaluate relevance and it looks like a cover-up.

Disclaimers that don’t annoy. Avoid generic “AI may be wrong” banners on every message. Instead, tie disclaimers to specific risk categories and only show them when relevant (e.g., medical, legal, financial). Use a compact callout: “This is not medical advice. Consider consulting a clinician.” Pair it with a user-control action such as “Show sources” or “Ask me clarifying questions.”

• Uncertainty UI: show “I’m not sure” states explicitly, such as a confidence label (“Low confidence”) or a “Needs verification” badge when citations are thin.
• User control: add “Regenerate,” “Ask follow-up,” “Report issue,” and “Stop generating” controls that are keyboard-accessible and persist even while streaming.
• Grounding contract: if your model uses tools, show a compact tool timeline (“Searching…”, “Reading sources…”, “Calculating…”) so users understand where the answer came from.

Engineering judgment: decide what “grounded” means in your app. For example, you might require at least one citation for factual claims, or you might constrain the model to answer only from retrieved documents. Enforce it with output constraints and post-generation checks (e.g., if no citations are present, the UI prompts: “I couldn’t find sources—do you want me to answer from general knowledge?”). That choice—asking permission—is a powerful trust pattern because it makes the boundary explicit.

Section 5.2: Cost & limits: caching, quotas, and model routing

Streaming chat can quietly become your most expensive endpoint. Production cost control is a product feature: it keeps the app available, prevents surprise bills, and protects shared infrastructure. Start by making costs observable, then apply limits at multiple layers.

Rate limits and quotas. Implement per-user and per-IP limits on the server route that initiates SSE streaming. Use a fast store (Redis or a managed rate-limiting service) and enforce: requests/minute, concurrent streams, and tokens/minute. Return a clear 429 response with a retry-after hint, and in the UI render a calm, actionable state (“You’ve hit today’s limit. Try again in 2 minutes or upgrade.”). Common mistake: limiting only by IP—mobile networks and offices share IPs and you’ll block legitimate users.

Caching and reuse. Cache expensive tool results (search queries, embeddings, retrieval results) with short TTLs, keyed by normalized inputs. For conversational UX, cache is most useful for deterministic tool calls (“lookup product policy”) rather than model completions. When you do cache model responses, store them by (prompt hash + system version + model version) and treat it as an optimization, not a source of truth.

• Budget caps: enforce daily token budgets per org/user; stop generation early with a UI message explaining why.
• Max tokens: set conservative defaults and allow user-initiated expansion (“Continue”) rather than one huge generation.
• Concurrency: limit tool invocations per request; prevent runaway tool loops with a max tool-call count.

Model routing. Route requests to cheaper/faster models when the task is low-stakes (summaries, formatting) and reserve premium models for hard reasoning or tool orchestration. A practical routing rule uses: message length, presence of tools, and user tier. Keep routing transparent in logs and stable for users—frequent model switching can change tone and quality unexpectedly.

Workflow tip: treat “cost controls” like auth—tested, versioned, and reviewed. Add unit tests for rate-limit keys, integration tests for 429 UI states, and dashboards for spend per endpoint and per feature flag.

Section 5.3: Observability: tracing a request across UI and server

When users say “it hung,” you need to know whether the model was slow, SSE broke, a tool timed out, or the browser tab lost connectivity. Observability for streaming chat means correlating events across the client, your server route, and downstream providers.

Correlation IDs. Generate a requestId at the server when the chat turn starts, return it immediately (first SSE event), and attach it to every log line, tool call, and provider request. On the client, store that ID in the message state so “Report issue” can include it automatically.

Structured logging. Log in JSON with consistent fields: requestId, userId/orgId (if applicable), model, route, latencyMs, tokensIn/tokensOut, finishReason, toolNames, errorCode. Avoid logging full prompts by default; instead log prompt length and a hash, and gate full payload logging behind a short-lived debug flag with strict access controls.

• Token and latency metrics: record time-to-first-token (TTFT), total stream duration, tool latency, and SSE disconnect rate.
• Client logs: capture key UI events—send click, stop click, stream started, stream ended, retry, hydration errors—then sample and upload with privacy filters.
• Tracing: adopt OpenTelemetry (or a vendor equivalent) and create spans for: API route, provider call, each tool call, and post-processing/validation.

Common mistakes. (1) Measuring only total request time, which hides TTFT regressions that users feel as “nothing is happening.” (2) Not distinguishing “model still generating” from “connection dropped,” leading to misleading spinners. (3) Logging too much sensitive data, creating a security incident disguised as “debugging.”

Practical outcome: with TTFT, stream duration, and disconnect rate on a dashboard, you can spot regressions after deployments and know whether to fix UI buffering, server backpressure, or provider settings.

Section 5.4: Reliability: timeouts, retries, and circuit breakers

Reliable chat UX is mostly about predictable failure. Users will tolerate an error that explains itself and offers a next step; they won’t tolerate silent hangs or duplicated tool actions. Reliability requires coordinated policies between the client and server.

Timeouts with intent. Set explicit timeouts for each stage: provider connect, time-to-first-token, per-tool execution, and overall request. If TTFT exceeds your threshold, stream a status event (“Still working…”) and offer “Stop” and “Try again” actions. On the server, abort upstream requests when the client disconnects to avoid paying for orphaned generations.

Retries that don’t multiply side effects. Retry network failures, not validation failures. For tool calls, make them idempotent where possible (e.g., read-only queries) and include an idempotency key when they mutate state. In the UI, a “Retry” should create a new message attempt linked to the previous attempt, so the user sees history and you keep observability clean.

• Circuit breakers: if a provider or tool endpoint has elevated error rate, trip a breaker and temporarily route to a fallback model or disable the tool.
• Fallback UX: degrade gracefully: “Search is unavailable; I can answer from general knowledge if you confirm.”
• Error budgets: define acceptable failure rates for stream starts, tool calls, and end-to-end completions; use them to decide when to freeze releases.

Feature flags. Put high-risk capabilities—new tools, new model routing rules, aggressive prompt changes—behind flags. A good flag system supports gradual rollout (1% → 10% → 50% → 100%), quick rollback, and per-tenant overrides. Common mistake: using flags as permanent configuration sprawl. Retire flags after the feature stabilizes.

Engineering judgment: don’t chase zero errors. Chase fast detection, clean rollback, and user-visible clarity. A stable “we’re having trouble—retry” state is better than a half-broken stream that produces misleading partial answers.

Section 5.5: Security basics: secrets, CORS, and data handling

Production AI UX expands your attack surface: prompts can contain malicious instructions (prompt injection), tool calls can be exploited, and logs can leak sensitive data. Security basics are not optional; they’re required to ship responsibly.

Secrets management. Keep provider keys and tool credentials on the server only—never in the browser bundle. Use environment variables managed by your hosting platform, rotate keys, and scope them to least privilege. If you run multiple environments, ensure staging keys cannot access production data.

CORS and origin controls. Lock down your SSE route to your known origins. If you expose APIs publicly, use auth tokens and CSRF protections where applicable. Don’t rely on CORS alone as an auth mechanism; it’s a browser control, not a server security boundary.

Data handling and retention. Decide what you store: user messages, model outputs, tool results, and telemetry. Minimize by default. If you store chat history, encrypt at rest, limit access by role, and define retention windows. In logs, redact obvious PII patterns and provide a “do not log content” mode for sensitive tenants.

• Prompt injection defenses: treat tool outputs as untrusted input; separate system/tool/user channels; and validate tool arguments against schemas.
• Output constraints: enforce JSON schemas for tool calls, apply allowlists for tool names, and reject unknown fields.
• Safe execution: tools that touch files, networks, or databases need additional permission gates and, for risky actions, a user confirmation step in the UI.

Common mistake. Letting the model decide what tool to call and passing the tool result back without sanitization. Instead, validate arguments, constrain tool capabilities, and log tool decisions with requestId so you can audit behavior.

Section 5.6: Accessibility for chat: focus management and ARIA patterns

Accessibility is a production requirement because chat is inherently dynamic: content appears while streaming, controls change state, and errors can happen mid-message. If you don’t design focus and announcements intentionally, keyboard and screen-reader users will be lost.

Keyboard-first workflow. Ensure the full flow works without a mouse: focus lands in the input on page load; Enter sends (with Shift+Enter for newline); Escape stops streaming; Tab navigates message actions (copy, retry, report). After send, keep focus in the input so users can continue; don’t steal focus to the streaming message unless the user requests it.

ARIA for streaming content. Use an ARIA live region carefully. A common pattern is: (1) a polite live region announcing high-level status (“Assistant is typing”, “Response complete”, “Error—retry available”), and (2) the message content itself is not continuously announced token-by-token (that becomes noise). Provide a “Read latest response” button that moves focus to the message container for users who want it.

• Roles and labels: mark the transcript as a list, each message as a listitem/article with clear author labeling (“User: …”, “Assistant: …”).
• Focus management: when a retry creates a new attempt, focus the retry status, then return to input; avoid focus jumps during streaming.
• Motion sensitivity: respect prefers-reduced-motion; avoid auto-scrolling animations and provide a manual “Jump to latest” control.

Error and confirmation flows. Tool confirmations (e.g., “Send email?”, “Delete record?”) must be reachable and understandable via screen reader. Use a modal dialog with proper focus trap and descriptive button labels (“Confirm send email”, not “OK”). When an error occurs mid-stream, announce it in the status live region and present a single primary action (“Retry”) plus a secondary (“Copy partial output”) so users can recover.

Practical outcome: an accessible chat UX is calmer for everyone: fewer surprise scroll jumps, clearer state changes, and controls that behave consistently while streaming.

Start by freezing scope. The most common capstone failure is expanding features while core behaviors remain brittle (especially streaming + tool calls). Write requirements in user-visible terms and pair each with an acceptance test you can execute manually in under two minutes.

Minimum feature set (requirements): (1) streaming responses via SSE from a server route to a React/Next.js UI; (2) interruptions (stop generating) that immediately halt UI updates and server streaming; (3) message state that supports partial outputs, retries, and multi-turn context; (4) tool use with schemas, validation, and safe execution; (5) tool-result UX that shows progress, citations/links when relevant, and errors; (6) persistence (at least local storage or a simple DB table) so the thread survives refresh and can be resumed.

• Acceptance test examples: Start a long answer, click Stop within 1s; verify the last assistant message is marked “stopped” and no more tokens append. Trigger a tool call; verify UI shows “Running…” then a structured result block; if tool errors, verify a friendly error state plus “Retry” that replays tool call safely. Refresh mid-thread; verify messages reload with correct roles and timestamps. Simulate slow network (throttle); verify partial tokens still appear and UI doesn’t freeze.

Non-goals (explicitly exclude): multi-agent orchestration, file uploads, user accounts, and “perfect” design polish. Those are valuable, but they tend to hide broken fundamentals. Recruiters reward reliability over breadth.

Finally, define your “definition of done.” A practical standard is: all acceptance tests pass, no TypeScript errors, lint passes, and you can deploy from a clean clone using only documented steps. This clarity prevents endless tinkering and forces you to deliver a coherent, reviewable artifact.

Production deployment is part of the skill. The goal is environment separation: local development should use local keys and mock-friendly settings; staging/preview should be safe for demos; production should be locked down with rate limits and stable observability.

Hosting options: For a Next.js capstone, Vercel is the most straightforward (preview deployments per branch, built-in environment variables). Alternatives include Render/Fly.io (more control over long-lived processes) or a container on a cloud VM. Choose one you can explain. If you use SSE, confirm your host supports streaming responses without buffering; test it from the deployed URL, not just locally.

Environment configuration: create clear variable sets: OPENAI_API_KEY (or provider key), MODEL, APP_ENV (development/staging/production), RATE_LIMIT_RPM, and any tool-specific keys (e.g., search API). Use separate keys for staging vs production. In Next.js, keep secrets server-only; never expose them to the client bundle (avoid NEXT_PUBLIC_ for secrets).

• Secrets management: store secrets in the hosting provider’s encrypted env store; do not commit .env. Rotate keys after sharing the repo.
• Operational guardrails: add basic rate limiting per IP/session, cap max tokens, and enforce tool allowlists. Treat guardrails as part of “production-ready,” not “extra credit.”

Common mistake: deploying a build that “works” but silently fails on streaming because a proxy buffers responses. Your deployment checklist should include verifying that token-by-token updates appear in production and that Stop/Cancel still works under real network conditions.

Final QA for AI chat apps is less about pixel-perfect UI and more about state correctness under chaotic input. You’re testing concurrency: users can send twice, cancel mid-stream, open multiple tabs, or lose connectivity. A professional capstone shows you anticipated those realities.

Edge cases to test: (1) double-submit (press Enter twice quickly): ensure only one request is accepted or the UI queues intentionally; (2) cancel during tool execution: ensure the UI stops streaming and the tool result does not append to the wrong message; (3) retry after failure: ensure you keep prior messages but generate a new assistant message with a clear “retry” label; (4) partial output persistence: if the page reloads, do you store partial tokens or mark the message as incomplete?

• Concurrency model: track a per-message requestId or runId; append tokens only if IDs match the active run. This prevents stale streams from corrupting the latest message.
• Streaming robustness: handle SSE disconnects by marking the assistant message as “interrupted” with a “Resume” or “Retry” affordance rather than leaving the UI in a spinner state.

Mobile behavior: test on an actual phone (or device emulator) for keyboard overlap, scroll anchoring during streaming, and tap targets for Stop/Retry. A classic bug: new tokens push content while the user scrolls up to read; solve with “smart autoscroll” (autoscroll only when the user is already at the bottom).

Top issues to fix: duplicate messages, stuck loading indicators, mis-ordered tool results, and any scenario where the user cannot recover without refreshing. Your capstone should make recovery obvious: visible error text, a clear retry path, and no silent failures.

Your README is your first technical interview. It should explain what the app does, how to run it, and why the architecture is designed the way it is. Include an architecture diagram (even a simple box-and-arrow diagram) that shows: React UI → Next.js API route (SSE) → model provider → tool executor → persistence layer.

API contracts: document your server endpoints and streaming protocol. For the chat route, specify the request shape (messages, conversationId, optional tool settings) and the streamed events you emit (e.g., token, tool_call, tool_result, error, done). If you use JSON Lines over SSE, show an example event payload. This demonstrates you understand your own system boundaries.

• Tool registry: maintain a single source of truth for tools: name, description, JSON schema, permission level, timeouts, and error mapping. Document what “safe execution” means: validation before execution, allowlisted tools only, and sanitized inputs/outputs.
• Runbooks: add short operational notes: how to rotate API keys, how to disable a tool quickly, what to check when streaming breaks in production, and where logs live. This is the difference between a toy and an application.

Common mistake: a README that lists features but not decisions. Include a “Tradeoffs” section: why SSE over WebSockets, why you chose your state model (message status enums, run IDs), and what you would improve next (e.g., background jobs for long tools). Those notes help recruiters see engineering judgment.

To market this skill, you must explain it clearly. Practice a two-minute walkthrough: “The UI maintains message state with statuses (streaming, stopped, complete, error). The server streams tokens via SSE. Tool calls are validated against JSON schemas and executed in a safe allowlist. Results are rendered as structured UI blocks with progress and recoverable errors.” That summary signals you understand both UX and systems.

Explain streaming like an engineer: SSE is a single long-lived HTTP response that pushes events from server to client. The UI appends tokens incrementally, and cancellation works by aborting the client request and having the server stop writing. Mention why you chose SSE: simpler than WebSockets for one-way streaming, compatible with many hosting platforms, and easier to reason about in serverless contexts (with caveats).

• Explain tools (function calling): the model proposes a tool name + arguments; your server validates arguments (schema), executes the tool, then feeds the tool result back into the model or directly to the UI. Emphasize safety: no arbitrary code execution, strict timeouts, and clear user confirmation for sensitive actions.
• Tradeoffs to articulate: SSE vs WebSockets; client-side vs server-side persistence; optimistic UI vs strict serialization; how you prevent prompt injection from tool outputs (e.g., delimiting tool results, instruction hierarchy, refusing to execute new tools requested by tool output).

Portfolio story (before/after + measurable impact): quantify improvements you made while building: “Reduced time-to-first-token from 1.8s to 400ms by streaming,” “Cut user-perceived failures by adding retry states and error boundaries,” or “Prevented runaway costs with token caps and rate limiting.” Even if the numbers come from your own testing, they show product thinking. Pair the story with links: deployed app, GitHub repo, README, and a short demo video that shows Stop/Retry and a tool run.

Once your capstone is stable, you can extend it without undermining the core. The key is to add one “advanced” capability at a time and keep the same discipline: schemas, observability, and UX states for partial and failing behaviors.

RAG (Retrieval-Augmented Generation): add a retrieval step before generation: embed documents, store vectors, retrieve top-k passages, and cite them in the UI. The engineering pitfall is treating retrieved text as trusted instructions. Keep retrieved content clearly delimited, apply prompt-injection defenses, and render citations with source metadata (title, URL, chunk id). Add a “no sources found” state rather than hallucinating citations.

• Agents: if you add multi-step planning, make it observable. Show a run timeline (plan → tool calls → final answer). Add user controls: Stop, “Approve tool action,” and “Show reasoning summary” (avoid exposing sensitive chain-of-thought; prefer concise step labels and tool logs).
• Eval pipelines: build a small evaluation harness: a set of prompts with expected properties (must cite sources, must not call restricted tools, must follow output constraints). Run it in CI to catch regressions. Track metrics like tool-call success rate, refusal correctness, and latency percentiles.

These next steps are how you evolve from “frontend engineer who built a chat UI” to “AI frontend engineer who ships reliable systems.” But don’t skip the capstone discipline: each new capability must come with UX states, guardrails, and documentation updates. The best career signal is not novelty—it’s that your system stays understandable as it grows.