From Idea to App: Build a Tiny AI Feature (Beginner)

Section 1.1: What an AI feature is (and what it is not)

An AI feature is a small, well-defined capability inside an app that uses a model to transform input into output. It has a clear trigger (when it runs), a clear interface (what you pass in), and a clear contract (what comes out and what happens when it fails). Treat it like any other feature: it must be testable, observable, and safe enough for your users.

What it is not: an “AI vibe” added to your product, a chat box that answers anything, or a promise that the model will always be correct. Beginners often confuse AI features with general intelligence. In practice, you are designing a constrained tool that helps with one slice of work, for one type of user, under specific rules.

• Good AI feature: “Given a customer email, draft a 3-sentence reply that matches our friendly tone and includes our refund policy link.”
• Not a feature: “Answer customer questions.” (Too broad, no constraints, no success criteria.)

Engineering judgment begins with boundaries. If you can’t describe your feature as an input → output transformation in one sentence, it’s likely not a tiny feature yet. Also note that “AI” doesn’t replace product thinking: you still need to decide who uses it, where it appears in the UI, and what the user does with the result (copy, edit, approve, discard).

Finally, plan for non-AI fallback behavior. An AI feature should degrade gracefully: if the model fails, the user should still be able to complete the task manually, or you should return a safe default that doesn’t cause harm.

Section 1.2: Tiny features vs full AI products

A full AI product tries to own an entire workflow. A tiny AI feature improves one step inside an existing workflow. That difference is what makes the “build in a week” goal realistic. Full products require broad data coverage, deep UX exploration, and extensive safety evaluation. Tiny features can be shipped with a small example set, a simple evaluation checklist, and careful constraints.

Use this rule: if your feature needs users to explain their whole situation in a long conversation, you are probably building a product. If your feature can run on a single artifact the user already has (an email, a note, a ticket, a paragraph, a photo), you are closer to a tiny feature.

• Tiny feature candidates: summarize a meeting note; classify a support ticket into 5 categories; extract action items; rewrite text for clarity; detect whether a message contains scheduling intent.
• Full product signals: unlimited topics; multiple tools and permissions; long-term memory; complex multi-step planning; “agent” behavior that can take actions without approval.

Why this matters for MLOps: tiny features let you control variability. You can log inputs/outputs, set a cost ceiling per request, and write targeted tests. A full product pushes you toward open-ended evaluation and harder safety problems.

To keep scope tight, decide up front what you will not handle in version 1. For example: “English only,” “max 1,000 characters,” “no medical or legal advice,” or “only these 5 ticket categories.” These constraints are not limitations—they’re what enable you to deliver predictable quality.

Section 1.3: Picking a problem with clear users and stakes

Choose a real problem you can solve in a week by focusing on (1) a clear user, (2) a repeated moment of friction, and (3) stakes that are meaningful but manageable. “Clear user” means you can name them and their context: “a customer support agent handling 30 tickets/day,” not “everyone on the internet.”

Start with a short list of workflows you personally experience or can observe. Then ask: where do people copy/paste text, rephrase the same idea, categorize items, or scan long text for key details? Those are high-signal spots for a tiny AI feature because the input already exists and the output is easy to consume.

• Good week-sized problems: drafting replies, summarizing, extracting structured fields, routing, tone rewriting.
• Hard week-sized problems: diagnosing, predicting rare events, making final decisions with legal/medical impact, or tasks needing large proprietary datasets.

Define the stakes explicitly. If the feature is wrong, what happens? A harmless typo is low stakes; a wrong refund decision is higher stakes. Your first build should be assistive: the user reviews and chooses what to do. This reduces risk and makes evaluation simpler because “helpful draft” is easier to accept than “final answer.”

Common mistake: picking a “cool” problem with no real usage frequency. A feature used once a month won’t generate enough feedback to improve. Prefer problems where you can collect examples quickly and iterate often.

Section 1.4: Inputs, outputs, and constraints in plain language

Turn the problem into one clear AI task by writing the input → output contract in plain language. Avoid model jargon. Your future self (and your teammates) should be able to read it and know exactly what to build.

Use a simple template:

• User intent: what the user is trying to achieve at this moment.
• Input: the exact fields you will send (text, metadata, options).
• Output: the exact shape you want back (text, JSON fields, label).
• Constraints: length limits, tone, forbidden content, required inclusions.
• Failure behavior: what you show if the model refuses, times out, or returns garbage.

Example (plain language): “Input: a support email (up to 1,200 characters) and a selected product name. Output: a reply draft under 90 words, friendly tone, includes exactly one link to the refund policy, and never invents order details.” This is vastly more buildable than “write a good reply.”

Constraints are where engineering judgment lives. They control cost (shorter outputs), reduce hallucinations (don’t invent), and improve safety (don’t provide prohibited advice). They also help evaluation: if you require “must include refund link,” you can write a simple test that checks for it.

Common mistake: forgetting the user interface realities. If the output will be shown in a small card, specify the length. If it will be inserted into an email, specify greeting/sign-off rules. Design the output for where it will land.

Section 1.5: Example-driven specs (good vs bad outputs)

Define what “good” looks like with simple examples. This is the fastest way to align expectations and the most practical way to improve prompts and tests later. You do not need hundreds of samples to start; you need a small, consistent set that covers the typical cases.

Create 10–30 example inputs and label them consistently. “Label” might mean: the correct category, the extracted fields, or a reference draft. The key is consistency: write down your rules and apply them the same way each time. If you can’t label consistently, your feature definition is still fuzzy.

• Good output examples show the format, tone, and required elements.
• Bad output examples clarify boundaries: too long, missing key info, invented facts, wrong tone, unsafe content.

For a drafting feature, include at least:

• A straightforward request (happy path).
• An angry or emotional message (tone control test).
• A message missing key details (should ask a clarifying question or provide a safe placeholder).
• A policy-sensitive case (refund/returns/medical/legal) where the draft must avoid overpromising.

Common mistake: using only “clean” examples. Real inputs are messy: typos, multiple topics, conflicting information, pasted signatures. Add a few messy samples early so you don’t overfit your design to perfect text.

These examples become your first evaluation set. Later, when you call a model via API, you’ll run the same examples repeatedly to see whether changes improve or degrade quality. That repeatability is the foundation of practical AI engineering.

Section 1.6: Risks, edge cases, and acceptance criteria

Before you build, set your success checklist: quality, speed, cost, and safety. This prevents you from declaring victory based on one impressive output. Acceptance criteria should be simple enough to check repeatedly and strict enough to protect users.

• Quality: correct category/fields; follows format; addresses the user’s intent; avoids hallucinated facts; readable and on-tone.
• Speed: e.g., 95% of requests under 2 seconds end-to-end (or whatever fits your app).
• Cost: e.g., average under $0.01 per request; enforce max input length; cap output length.
• Safety: no disallowed advice; no sensitive data leakage; polite refusals when required.

Identify edge cases up front: empty input, extremely long input, non-English text, user includes personal data, user asks for prohibited content, or the model returns invalid format. Decide how you’ll handle each: truncate, ask for revision, refuse, or fall back to a template.

Add basic safety checks and failure handling as part of the feature contract. Practical patterns include: validating output structure (e.g., must be JSON), checking for required phrases/links, limiting length, and using a safe fallback message when validation fails. Also plan observability: log request metadata (not sensitive content), model latency, and whether validations passed.

Common mistake: treating “model refusal” as an error. Sometimes refusal is the correct safe behavior. Your UI should explain what happened and offer the user a next step (edit input, remove sensitive info, or do it manually).

With a clear task, example-driven specs, and acceptance criteria, you now have a buildable one-page brief: problem, users, input/output, constraints, examples, and checklist. That brief will guide your API integration and testing in the next chapter.

Section 2.1: Data basics for beginners (samples, fields, labels)

Start by defining what one “sample” is for your feature. A sample is a single example the AI will process, along with what you expect the AI to produce. For a tiny feature, think in rows. Each row should be complete enough that someone else could understand it without extra context.

Most samples have three parts: input (what your app receives), fields (structured columns that describe the input), and a label or expected output (what “good” looks like). Fields are optional but helpful. If you’re classifying an email, fields might include subject, body, and channel. If you’re extracting data, fields might include message_text and the expected extracted JSON.

Labels can be categories (e.g., billing, bug, feature_request), or they can be text outputs (e.g., a rewritten sentence), or a structured object (e.g., {"due_date":"2026-03-10","amount":49.99}). Choose the simplest label type that matches your product requirement. If your UI needs a category, label with categories, not paragraphs of explanation.

Engineering judgment: define labels at the level of stability you can maintain. Beginners often create labels that are too nuanced (“annoyed but polite” vs. “slightly frustrated”), then discover they can’t label consistently. Prefer fewer, clearer labels over many fuzzy ones. When in doubt, add an other label and revisit later.

• Sample: one real case your feature should handle
• Fields: columns that store the input and helpful metadata
• Label: the expected output your feature should produce
• Schema: the list of fields and their meaning (write it down!)

Your practical outcome for this section: a one-paragraph description of your dataset schema and a draft list of labels (or output format) that you can apply to 30–100 samples without constantly revising the rules.

Section 2.2: Where examples come from (without violating privacy)

Your dataset should reflect reality, but not at the cost of privacy, security, or licensing violations. For a beginner-friendly project, there are safe sources that still give you “real” variety.

First, check if you can use data you already control: your own notes, messages you wrote, or synthetic examples you authored. If you do use internal business data (support tickets, chats), get explicit permission and remove personal identifiers. A good rule: if you would feel uncomfortable pasting it into a public issue tracker, you should not put it into your dataset.

Second, use public-domain or appropriately licensed datasets. Many repositories provide example text for tasks like sentiment analysis, toxicity detection, spam classification, or intent detection. Record the license and attribution requirements in a DATA_SOURCES.md file so you can prove you collected the data legally.

Third, create “realistic but fake” examples. You can write samples that mimic your expected inputs while avoiding real names, phone numbers, addresses, order IDs, and medical or financial details. This is often enough for a tiny AI feature, especially early in development.

Common mistakes include scraping websites without permission, copying private conversation logs, or mixing data with incompatible licenses. Also watch for hidden sensitive data inside logs (API keys, tokens, internal URLs). If you must include identifiers for debugging, replace them with stable placeholders like USER_001 or ORDER_123.

• Use your own authored examples or permitted internal data
• Prefer public datasets with clear licensing
• Redact or avoid PII: names, emails, phone numbers, addresses
• Document sources and rules so future you can verify compliance

Your practical outcome: a short list of 2–3 approved example sources and a redaction checklist you will apply before any sample enters the dataset.

Section 2.3: Labeling rules and consistency checks

Labeling is where your “tiny AI feature” becomes measurable. Without consistent labels, you can’t tell whether a model call or a prompt update actually improved anything. The trick is to write labeling rules that are concrete enough to follow, then apply quick consistency checks as you label.

Create a one-page labeling guide. For each label, include: (1) a plain-language definition, (2) 2–3 positive examples (“this is the label”), and (3) 1–2 boundary examples (“this is not the label”). Boundary examples reduce confusion later. If your output is structured, define the format precisely: required keys, allowed values, and how to represent “unknown” (e.g., null or empty string).

While labeling, keep notes on uncertain cases. Beginners often force every sample into a category even when it doesn’t fit. That silently damages your dataset. Instead, mark ambiguous items with a temporary flag like needs_review=true. Later, you can either refine rules or move those samples into other.

Consistency checks can be simple and still powerful:

• Re-label 10%: after you finish, re-label a random 10% without looking at the old labels. If you disagree with yourself often, your rules are too vague.
• Look for label imbalance: if one label dominates, confirm that reflects reality and not a bias in how you collected examples.
• Spot-check boundaries: review the hardest cases for each label and ensure your decisions follow your written guide.

The practical outcome: a labeling guide you can hand to a teammate and get similar labels back. This is the foundation for later evaluation and basic safety checks, because you can only validate safety behavior if “safe vs unsafe” is labeled predictably.

Section 2.4: Train-like vs test-like sets (why splitting matters)

Even if you never “train” a model, you still need a split between build and test examples. Your build set is what you use while iterating—tuning prompts, adjusting output parsing, adding guardrails, and fixing edge cases. Your test set is what you keep aside to measure whether those changes generalize to new inputs.

Without a split, you will overfit your prompt and code to the examples you’ve seen. It feels like progress (“all my examples pass now!”) but it’s an illusion. A test set gives you a reality check: does the feature work on samples you did not use to design it?

For 30–100 samples, a practical split is 80/20 or 70/30. Keep the test set representative: include a similar mix of labels and difficulty. If you have rare labels (only 3–5 examples), ensure at least one lands in the test set so you notice failures. If your data has groups (e.g., multiple messages from the same thread), keep groups together in either build or test—otherwise you leak near-duplicates into test and inflate your results.

Workflow tip: create two files early, even if small: dataset_build.jsonl and dataset_test.jsonl. When you call an AI model through an API later, you will run the same evaluation script against the test file each time you change prompts or safety logic.

• Build set: for iteration, debugging, and prompt design
• Test set: for honest measurement and comparisons over time
• Avoid leakage: keep related/near-identical items in the same split

Your practical outcome: a frozen test set that you do not “clean up” just to make scores look better. Fix the feature, not the exam.

Section 2.5: Cleaning: duplicates, missing fields, odd formats

Mini datasets still need cleaning. In fact, small datasets are more sensitive: a handful of duplicates or malformed examples can distort your evaluation and mislead you during iteration.

Start with duplicates. Exact duplicates are easy (identical text). Near-duplicates are trickier: copied templates, forwarded messages, or the same request with tiny edits. Near-duplicates can cause leakage between build and test if you split randomly. A practical approach is to create a fingerprint field—like a normalized version of the input (lowercased, whitespace collapsed)—and then review items with matching or very similar fingerprints.

Next, handle missing fields. Decide which fields are required for your feature. If your API call expects message_text, then samples missing it are invalid and should be removed or repaired. For optional fields, use a consistent representation (empty string or null) rather than a mix, because inconsistent null handling causes annoying bugs later.

Watch for odd formats: unexpected encoding, stray HTML, multi-line text that breaks CSV rows, or timestamps in different formats. Cleaning isn’t about making data “pretty”; it’s about making it predictable. If your app will see messy input in production, keep some messy samples, but store them consistently and label them clearly. A common beginner mistake is to delete all “hard” examples. Instead, keep them and mark them with a field like hard_case=true so you can ensure your feature improves on real-world pain points.

• Remove exact duplicates; review near-duplicates
• Validate required fields; standardize nulls
• Normalize formats (dates, whitespace) without erasing real messiness

Your practical outcome: a dataset that loads cleanly every time and a short cleaning checklist you can re-run whenever you add new samples.

Section 2.6: Storing data as CSV/JSON and versioning the file

Once your samples are collected, labeled, split, and cleaned, store them in a simple format you can reuse. Two beginner-friendly choices are CSV and JSON. CSV is great for spreadsheet viewing and quick edits, but it struggles with nested outputs and multi-line text. JSON (especially JSON Lines, .jsonl) is better for AI tasks because it naturally stores long text and structured expected outputs.

A practical JSONL row might look like: {"id":"s_042","input":{"text":"..."},"label":"billing","notes":"","hard_case":false}. Add a stable id so you can track a sample across edits. Avoid using row number as ID because it changes when you sort or filter.

Version your dataset like code. Put it in a repository and commit changes with messages such as “add 12 refund examples” or “fix label rules for shipping vs billing.” If the dataset contains sensitive but permitted internal data, store it in a private repo and restrict access. For teams, consider a lightweight changelog: DATASET_CHANGELOG.md where you record additions, removals, and labeling rule updates.

Also store your schema and labeling guide next to the dataset file. Future you will forget why a field exists or what other was supposed to mean. The point of versioning is not bureaucracy—it’s repeatability. When you evaluate model outputs later, you need to know which exact dataset version produced which results.

• CSV: simple, spreadsheet-friendly; be careful with commas and newlines
• JSON/JSONL: flexible for long text and structured outputs
• Version control: commit dataset changes and keep a labeling guide alongside

Your practical outcome: one reusable dataset file (build and test), a labeling guide, and a versioned history of changes—so your tiny AI feature can improve in a controlled, testable way.

Before writing prompts, it helps to understand what you are actually integrating. A hosted model API is a service that converts input tokens (your text plus system instructions plus examples) into output tokens (the model’s response). The service does not “know” your application context unless you include it. There is no memory unless you provide prior messages, and there is no guarantee of correctness unless you validate outputs.

From an engineering perspective, treat the model like a probabilistic function: the same input can yield slightly different outputs, especially when randomness is enabled (often via a “temperature” setting). This is why your “tiny AI feature” should have clear inputs/outputs and a narrow task. It’s also why you should decide early whether you need prompt-based generation or a simple classifier approach.

• Prompt-based generation is best when you need text creation or transformation: summaries, rewrites, suggestions, extraction into fields.
• Simple classifier is best when you need one of a small set of labels: “billing vs. technical vs. account,” “urgent vs. non-urgent,” “spam vs. not spam.” You can still use a generative API, but constrain the output to labels.

Common mistake: choosing generation when you actually need a deterministic label. If your product needs a stable category, a classifier-style prompt with a fixed label set will be easier to test and safer to ship. Common mistake: sending the entire user history “just in case.” More text means higher cost and more chances the model follows irrelevant details. Start with the minimum information needed for the task, then expand only if the quality tests fail.

Practical outcome: you can describe your AI call like a contract: Input (what the user provides + what your app adds), Output (exact fields your app needs), and Failure modes (timeouts, empty output, invalid JSON, unsafe content). That contract will guide everything else in this chapter.

A prompt is not “magic wording.” It is a compact specification: instructions (what to do), constraints (what not to do), and examples (what good looks like). For beginners, the fastest path to a working feature is to write a prompt that reads like a short operating procedure. Your job is to remove ambiguity.

Use a two-part structure: (1) a role + goal statement, and (2) explicit output requirements. If your feature is “summarize a support ticket and propose next steps,” a prompt might define: audience (support agent), length limits (max 3 bullets), and forbidden behavior (don’t invent order numbers). If your feature is “classify,” list the labels and give one-sentence definitions for each label.

Then add a few examples. Examples teach the boundary lines your instructions didn’t fully capture. Even one or two can dramatically improve reliability because they anchor the response format and the level of detail. Keep examples small and representative: choose the cases that previously caused confusion, not the easiest cases.

• Instruction style that works: “Given INPUT, produce OUTPUT. Follow these rules. If the information is missing, return null for that field.”
• Constraint style that works: “Use only the labels listed. If uncertain, choose ‘other’.”

Common mistakes: writing prompts that are goals but not constraints (“be helpful!”), or adding examples without consistent formatting. Your app needs consistency more than creativity. Also avoid leaking product secrets or personal data into prompts. Treat prompts like code: review them, version them, and keep them minimal.

Practical outcome: by the end of this section, you should have a first working prompt that produces plausible results in a manual test (copy/paste a few inputs and inspect outputs). Do not optimize yet; just get a stable baseline you can wrap with structure and error handling.

Few-shot prompting means you include a small set of example inputs and the exact outputs you want. The key is not the number of examples; it’s the consistency. Your examples should share the same fields, the same casing, and the same order. If you want a classifier, every example should return exactly one label (not a label plus commentary). If you want extraction, every example should include the same keys even when values are missing.

Choose examples strategically. A good starter set is 4–8 items that cover: (1) a typical case, (2) a tricky case with missing information, (3) a borderline case that could be misclassified, and (4) a “should refuse” or “should be cautious” case if relevant (for example, requests for medical/legal advice). Label them consistently using your definitions from Chapter 2.

• Formatting tactic: use clear separators like “### INPUT” and “### OUTPUT” so the model can parse the pattern.
• Stability tactic: keep your example outputs minimal and repeatable; avoid unnecessary adjectives or variable-length prose.
• Classifier tactic: include at least one example per label so the model sees the full label space.

Common mistakes: adding too many examples (increases cost and can confuse the pattern), using contradictory labels, or including examples that are unrealistic compared to production inputs. Another frequent reliability issue is “format drift,” where one example uses a bullet list and another uses sentences. Models often imitate whatever pattern is most recent, so format drift can cascade into your real requests.

Practical outcome: you end up with a prompt that behaves like a lightweight program: it maps common inputs to a predictable output shape. At this stage you should begin logging a small set of real inputs (with privacy-safe handling) so you can expand your few-shot set based on actual failure cases.

Humans can read free-form text; apps need structure. Your feature becomes far more usable when the model returns JSON that your code can parse. Instead of “Here’s a summary…,” you want something like:

• summary: a short string
• category: one label from a fixed set
• confidence: a number 0–1 (optional, treat as advisory)
• action_items: an array of short strings

In your prompt, explicitly require JSON only, with no surrounding text. Define allowable values (for enums like category) and define what to do when unknown (use null or “other”). If your API/tooling supports “JSON mode” or schema-based outputs, enable it; otherwise, you can still instruct the model to output strict JSON, but you must expect occasional invalid responses.

Validation is not optional. After the API call, your code should:

• Parse JSON strictly (fail fast if invalid).
• Validate required keys exist and types match (string vs array vs number).
• Validate enums (category is in the allowed set).
• Clamp or reject out-of-range values (confidence between 0 and 1).

If validation fails, you have options: retry with a “repair JSON” instruction, fall back to a simpler prompt, or return a safe default. Common mistakes: trusting the model’s “confidence,” letting unvalidated fields flow into your UI, or silently accepting partial JSON that later breaks downstream code.

Practical outcome: you can now treat the model response like an internal API response. Your frontend and database code can rely on stable keys, and you can write tests that compare parsed objects rather than subjective text.

Even a perfect prompt won’t prevent operational failures. Network calls can time out, rate limits can trigger, and the model can return empty or malformed content. A production-ready tiny AI feature assumes failure will happen and makes failure safe.

Start with timeouts. Set a firm request timeout that matches your user experience: for an interactive UI, you might target 3–10 seconds. Next add retries, but only for retryable errors (timeouts, transient 5xx, rate limits). Use exponential backoff and jitter so you don’t create a thundering herd. Do not retry on validation errors indefinitely; that’s how costs explode.

• Retry policy example: up to 2 retries for network/timeouts; 1 retry for invalid JSON with a “return valid JSON only” repair instruction.
• Fallback example: if structured extraction fails, fall back to returning a plain-text summary and skip category, or choose category “other.”
• Graceful UI: show a helpful message like “We couldn’t generate suggestions right now—here’s a basic summary,” instead of crashing.

Also plan for empty responses. Your handler should treat empty string, missing “content,” or missing fields as an error state and trigger the fallback path. Log failures with enough context to debug (request id, timing, model name, error type), but avoid logging sensitive user text unless you have a privacy plan.

Common mistakes: retrying too aggressively, hiding errors until users complain, or failing open (displaying unvalidated model text in places where it could be unsafe or misleading). Practical outcome: your feature remains usable under real-world conditions and you can measure and improve it from logs rather than guesswork.

API-based AI is priced by usage, typically proportional to tokens processed and generated. Cost control is therefore an engineering habit, not a finance problem. You should measure tokens in, tokens out, and latency per request from day one, even in a prototype.

First, control prompt size. Keep instructions short, keep examples minimal, and avoid sending unnecessary context. Second, cap output length with a max tokens setting or explicit brevity constraints (“summary max 40 words”). Third, set rate limits per user or per IP to prevent accidental or malicious spikes.

• Token budget mindset: decide a target cost per successful request and design backward (shorter prompt, fewer examples, smaller output).
• Speed tradeoff: bigger models can be higher quality but slower and more expensive; start with a model that meets your baseline tests, then upgrade only where it improves outcomes.
• Caching idea: cache results for identical inputs (or normalized inputs) to avoid paying twice. For example, if a user reopens the same ticket, reuse the prior summary.

Measure in simple terms: log the elapsed time, tokens used (if the API provides it), and whether the response passed validation on the first try. That gives you a practical scorecard: “p50 latency,” “cost per 100 calls,” and “first-pass success rate.”

Common mistakes: ignoring output limits (the model writes an essay), keeping every few-shot example forever, and not noticing retries are doubling your cost. Practical outcome: you can iterate with confidence—each prompt change can be evaluated not only for quality but also for cost and speed, which is exactly what makes a tiny AI feature shippable.

Before writing rules, name the failures you’re defending against. For tiny AI features, three show up repeatedly: hallucination, bias, and leakage. Hallucination means the model produces plausible but incorrect information, or invents details that were not provided. If your feature summarizes a message thread, hallucination can look like adding an action item that nobody mentioned. If your feature extracts structured fields, hallucination can look like filling a missing “date” with a guess.

Bias shows up when outputs differ unfairly across groups or when the model uses stereotypes. Beginners often miss bias because they only test with their own examples. If your feature rewrites text “more professional,” it may change tone differently depending on dialect or cultural phrasing. If your feature ranks items, it may consistently prefer certain categories without justification.

Leakage includes private data flowing where it shouldn’t: user secrets, API keys, emails, phone numbers, addresses, or internal company data. Leakage can happen in two directions: you might send sensitive user input to a third-party model without realizing it, and the model might echo or transform that sensitive data into the output.

• Hallucination symptom: confident details with no source in the input.
• Bias symptom: different behavior for similar inputs phrased by different people.
• Leakage symptom: output includes credentials, personal identifiers, or internal content.

Write these failure modes into your engineering checklist. Each time you change your prompt, model, or UI, re-run a small test suite targeting these risks. A common mistake is to treat guardrails as a one-time task; in reality they’re a living part of your feature, just like logging and monitoring.

A safety policy is a short list of “do not” rules that define what your feature will refuse to do or will handle carefully. The key is to write them in plain language that a non-expert teammate can review. Avoid vague statements like “don’t be unsafe.” Instead, name categories and give examples.

Start with what your app context makes risky. If your tiny AI feature helps draft messages, you should block harassment and threats. If it provides recommendations, you should avoid medical, legal, or financial instructions unless you have strong controls. If it helps students, you should consider cheating policies. You’re not trying to cover the entire internet—just the realistic misuse paths for your feature.

• Do not generate: instructions for violence, self-harm encouragement, illegal wrongdoing, or targeted harassment.
• Do not provide: personalized medical/legal/financial advice (offer general info + encourage professional help instead).
• Do not request or reveal: passwords, API keys, payment card numbers, or private identifiers.
• Do not impersonate: a real person or claim actions you didn’t take (e.g., “I contacted your bank”).

Then decide what happens when users ask anyway. For a beginner project, your safest default is refuse + redirect: briefly say you can’t help with that request and offer a safe alternative (for example, “I can help you write a respectful complaint email” rather than “How do I threaten my landlord?”). Another common approach is safe completion: answer only the allowed part of the request and omit prohibited instructions.

Common mistakes: writing policies that are too broad (blocking normal use), policies that are too subtle (hard to test), or policies hidden only in the prompt (easy to bypass). Put your “do not” rules in code as well, so you can enforce them even if prompts change.

Input guardrails happen before the model call. They protect your system, reduce garbage-in/garbage-out, and prevent sending sensitive data unnecessarily. Think of input handling as three steps: validate, clean, and redact.

Validate the basics: required fields, maximum length, allowed characters (when relevant), and content type. If your feature expects “a short customer message,” reject a 50,000-character paste. If you accept file uploads, restrict file types and size. Validation is not only security—it also protects your token budget and improves predictability.

Clean for usability: trim whitespace, normalize line endings, collapse repeated punctuation, and handle empty input. A common mistake is sending “empty but not empty” strings (like spaces) to the model and then treating the model’s output as meaningful.

Redact private data when you don’t need it. For many tiny features, you can replace identifiers with placeholders while keeping the meaning. Practical redaction targets include email addresses, phone numbers, home addresses, social security numbers, and access tokens. You can implement simple regex-based detection as a first pass, and add more patterns over time.

• Example redaction: “Email me at jane@site.com” → “Email me at [EMAIL].”
• Example redaction: “My number is (555) 123-4567” → “My number is [PHONE].”
• Example redaction: “Here’s my API key: sk-...” → “[SECRET]” and block the request.

Engineering judgment: don’t over-redact. If your feature is “extract the user’s contact details,” then you obviously need those details—so you might instead store them locally and send only the minimum context to the model, or avoid the model entirely for extraction. Also decide where redaction happens (client vs server). Server-side redaction is more consistent; client-side redaction can reduce exposure earlier but is easier to bypass.

Finally, log carefully. Avoid logging raw user inputs by default. If you must log for debugging, log redacted versions and use short retention periods. Many privacy incidents come from logs, not from the model itself.

Output guardrails happen after the model call. They ensure the response matches what your UI and downstream code expect. Beginners often treat model output as “final text,” but in a real app you need to treat it as untrusted and verify it.

Start with format checks. If you asked for JSON, parse it and fail gracefully if parsing fails. If you asked for a bulleted list, ensure it is actually a list. If you asked for a classification label, enforce that it is one of the allowed labels. If parsing fails, you can retry once with a stricter prompt, or fall back to a safe default (“I couldn’t confidently classify this; please rephrase.”).

Next add length limits: maximum characters, maximum items, and maximum sentences. This protects your UI and prevents the model from returning a rambling essay when you wanted a one-line suggestion. If you enforce a strict limit, prefer truncation with a clear indicator over silent cutting that changes meaning.

Then add banned-content filters. This is where your “do not” rules become enforceable. Use a simple approach first: a small list of banned terms/phrases for your domain plus pattern checks for secrets (e.g., token prefixes). When a violation is detected, replace the output with a refusal message or route to human review.

• JSON schema validation: required keys, allowed enums, types (string/number/array).
• Safety scan: detect harassment slurs, self-harm encouragement, doxxing patterns, secret-like strings.
• Quality heuristics: reject “As an AI...” boilerplate, repeated text, or missing required sections.

Common mistakes: relying only on prompt wording (“Please output valid JSON”) without parsing; silently accepting partial fields; and blocking too aggressively with naive keyword filters. Keyword filters should be paired with context where possible and should fail “safe” (no harmful output) rather than “silent” (harmful output reaches the user).

A practical outcome for this chapter is a small “postprocessor” function in your code that takes raw model text and returns either (a) validated structured output, (b) a safe refusal, or (c) a review request.

No beginner guardrail system catches everything, and some cases should not be automated at all. A simple human-in-the-loop (HITL) path is how you handle “tricky” inputs and uncertain outputs without pretending the model is always right. The goal is to create a clear branch in your workflow: normal cases are automatic; edge cases pause and request review.

Define triggers for review. You can use deterministic triggers (certain topics) and probabilistic triggers (low confidence). If your feature is a classifier, route to review when the top label probability is below a threshold. If you don’t have probabilities, use heuristics: the model output failed validation, the content includes redaction placeholders, or the user asked for something close to your policy boundaries.

• Policy-adjacent topics: self-harm, violence, illegal activity, sexual content involving minors (always block), medical/legal/financial situations.
• Privacy signals: output includes [EMAIL]/[PHONE] placeholders, or input had many identifiers.
• Uncertainty signals: parse failures, contradictory statements, or “I’m not sure” responses.

Design the review experience to be fast and consistent. Reviewers should see the redacted input, the model output, and the reason it was flagged. Provide three buttons: approve, edit, or reject. Store reviewer decisions as labeled examples—you can use them later to improve prompts, add new rules, or expand your small dataset.

Common mistakes: routing too many cases to humans (making the feature unusable), or routing too few (letting harmful outputs through). Start conservative for high-risk domains and gradually automate more as you collect evidence that your guardrails work. Also clarify the user experience: tell users when a response is delayed for review and provide an expected timeframe or an alternative action.

Your final guardrail is documentation. A short “feature limits” note sets correct expectations and reduces misuse. It should be visible where users interact with the feature (not buried in a legal page) and written in the same plain language as your safety rules.

A good limits note answers: what the feature does, what it does not do, what data it uses, and what the user should do if the output seems wrong. This improves safety and quality because users are less likely to treat the AI output as authoritative, and more likely to provide better inputs.

• Capability: “This tool drafts a short reply based on the text you provide.”
• Non-capability: “It can be incorrect or incomplete and may miss context.”
• Safety boundary: “It won’t help with harassment, wrongdoing, or instructions for harm.”
• Privacy boundary: “Do not enter passwords, API keys, or payment information. We may redact personal data before processing.”
• User action: “Review before sending. If it looks wrong, edit it or try again with more context.”

Keep it short, specific, and aligned with your actual behavior. A common mistake is writing aspirational limits that don’t match the app (for example, claiming “we never store data” while your logs retain raw input). Another mistake is overpromising: “always accurate,” “guaranteed safe,” or “bias-free.” Instead, state what checks you do and what users are responsible for.

Practical outcome: add the limits note next to your feature UI, link to a longer page if needed, and include a “Report an issue” path. That report channel becomes an input to your guardrail backlog: each report is a candidate test case, a new “do not” rule, or a new redaction pattern.

A tiny AI feature still benefits from a clear separation of responsibilities. Think in three parts: the client (your UI), the server (your backend), and the API (the contract between them). The client collects input and displays results. The server enforces rules: input validation, calling the model, applying safety checks, and returning a structured response. The API is simply the agreed request/response format—usually JSON.

Why not call the model directly from the browser? Because you would have to expose your API key, you’d lose centralized logging, and you’d make it harder to add safeguards consistently. Keeping the model call on the server lets you rotate keys, rate-limit abusive traffic later, and update prompts without redeploying the UI. Even if you’re the only user today, this architecture prevents “prototype debt” that becomes painful tomorrow.

• Client: HTML form (or a small React/Vue page) that POSTs text to your server and renders JSON results.
• Server: a small web service (often FastAPI for Python beginners) that exposes a single route like POST /predict.
• AI feature function: one clean function (e.g., run_feature(input_text) -> FeatureResult) that the endpoint calls.

Engineering judgment: keep boundaries strict. The endpoint should not contain your prompt engineering in-line, and the UI should not contain business rules. When something goes wrong, you’ll know which layer is responsible—and you can test layers independently.

Your goal is a single endpoint that turns an input into an output. Use a POST request so you can send structured data and avoid URL length issues. In FastAPI, you typically define a request model and a response model so your API is self-documenting and consistent.

Wrap the AI logic into one clean function first. For example, you might create tiny_feature.py with:

• Input normalization: trim whitespace, cap length, reject empty input.
• Model call: send prompt + user text to the model provider API.
• Safety checks: block disallowed content, require citations, or enforce a format.
• Failure handling: timeouts, provider errors, and a user-friendly message.

Then the endpoint becomes a thin wrapper:

• Parse JSON into a request object (e.g., {"text": "...", "mode": "..."}).
• Call run_feature().
• Return a JSON response (e.g., {"ok": true, "result": "...", "warnings": [...]}).

Common mistakes: returning raw provider responses (they change shape), leaking internal errors to the client, and not setting timeouts. Your response should be stable and small. Include enough detail to debug (request_id, warnings), but don’t dump entire prompts or secrets. Practical outcome: once /predict works with curl or Postman, your UI becomes much easier—you’re just calling an API that you control.

The UI for a tiny feature should be intentionally boring. A text area, a submit button, and a box for results is enough. The UI’s job is to: (1) capture input, (2) send it to /predict, (3) show the output, and (4) show errors in a human-readable way. Avoid adding extra fields until you’ve proven the core feature is useful.

A practical pattern is:

• Input: a <textarea> for text and an optional dropdown for a small set of modes (keep it to 2–3).
• Submit: disable the button while the request is in flight to prevent accidental double submits.
• Status: show “Working…” and then either a result panel or an error panel.
• Rendering: treat output as plain text first; only render HTML/Markdown if you sanitize it.

Common mistakes: assuming the model will always return something usable, or failing silently when the server returns an error. Your UI should handle at least three cases: success (ok=true), expected failure (validation error like “text too long”), and unexpected failure (server error). Keep the response schema consistent so the UI can rely on it.

Practical outcome: when you can paste one of your saved examples into the form and reliably see the expected kind of output, you’ve reached a real milestone. You now have a demo-able feature, not just a script.

As soon as you connect a UI and server, you’ll be tempted to paste API keys “just for now.” Don’t. Secret handling is not an advanced topic—it’s a beginner survival skill. If a key ends up in Git history, it may be harvested quickly, and rotating it later is painful.

Rules of thumb for a tiny AI app:

• Never place provider API keys in front-end code. Anything shipped to the browser is public.
• Store secrets in environment variables (e.g., OPENAI_API_KEY) and read them from the server at runtime.
• Use a local .env file for development, but add it to .gitignore.
• Keep prompts and configuration separate from secrets. Prompts can live in version control; keys cannot.

Engineering judgment: treat logs as semi-sensitive too. Even if you protect API keys, you can still leak user data or prompt details. Avoid logging raw user input unless you have a clear reason and you’ve considered privacy. If you do log input for debugging, consider truncating it or removing obvious personal identifiers.

Practical outcome: you can deploy or share the project safely. Anyone can run the app by setting environment variables, without editing code and without risk of committing secrets.

When a user says “it didn’t work,” you need more than guesses. Logging is your memory. For a tiny AI feature, good logs answer: what happened (event), when (timestamp), and why (context like input size, chosen mode, provider latency, and the error category).

Start simple and structured. Use a unique request_id per call, and include it in both server logs and the API response. Then you can correlate a UI report (“request_id=abc123 failed”) with exactly one server trace.

• Log at INFO: request received, validation passed, model call started/finished, response returned.
• Log at WARNING: safety filter triggered, output format repaired, user input near limits.
• Log at ERROR: provider timeout, parsing failure, unhandled exception.

Common mistakes: logging too little (no clue what failed) or too much (dumping full prompts, full user text, or entire model outputs). A practical compromise is to log lengths, hashes, and short snippets (e.g., first 120 characters) while keeping the full content out of logs.

Practical outcome: you can measure latency (“model_call_ms”), detect flaky behavior, and identify the top failure modes. This feeds directly into better safety checks and more focused improvements to your prompt or post-processing.

You already created a small labeled set of examples earlier in the course. Now you’ll use it like a professional: as a repeatable end-to-end check. The goal is not perfect accuracy; it’s regression prevention. When you change a prompt, add a safety rule, or refactor code, you want to know if outputs got worse.

A practical end-to-end test workflow:

• Store your test set as a small JSON or CSV file with fields like input, expected_behavior, and notes.
• Write a tiny test runner that iterates over examples, calls POST /predict, and saves responses to a timestamped file.
• Evaluate with a simple checklist: format valid, no disallowed content, answer addresses the request, and any required fields present.

Include both “happy path” and “tricky” examples: empty input, overly long input, ambiguous phrasing, and content that should trigger a safety refusal. This is where your failure handling proves its value: the app should fail cleanly and consistently.

Common mistakes: testing only in the UI (slow and inconsistent), or only unit-testing the AI function without hitting the real endpoint. You want both: unit tests for deterministic parts (validation, formatting) and end-to-end tests for the full pipeline. Practical outcome: you can run a demo with confidence, backed by a small repeatable script that shows the feature works on real examples—not just one cherry-picked prompt.

Deployment is the process of running your app somewhere other than your laptop, so other people (or future you) can access it. For a tiny AI feature, deployment usually means: (1) your UI and API are reachable from the internet, (2) secrets are not stored in code, (3) you can reproduce the same behavior across environments, and (4) you can roll forward safely when you make changes.

Pick a simple hosting option that matches your app. If you built a small web app, common beginner-friendly paths are: a serverless platform (fast to deploy, scales automatically) or a managed app host (straightforward “deploy from Git”). If your app is a single API endpoint, serverless is often easiest. If your app includes a small UI plus an API, a managed web host works well. What matters is not the brand—it’s that you can deploy consistently and you have logs.

After deployment, a few things change immediately. First, failures become normal: rate limits, timeouts, and user inputs you didn’t anticipate. Second, latency matters: a feature that feels instant locally can feel slow on the internet, especially when calling an AI API. Third, you are accountable for cost: a single prompt that’s fine in testing can become expensive with real usage.

Common mistakes at this stage include deploying without any way to view errors, hard-coding API keys in the repo, and assuming the model will behave “about the same” forever. The practical outcome you want is a working URL and a deployed environment where you can see what happened for every request (even when it fails).

• Practical outcome: You can open the deployed app, submit an input, and get a response. If it errors, you can find the error message in logs.
• Engineering judgment: Start with a simple hosting path that gives you logs and easy redeploys. Optimize later.

The fastest way to accidentally leak secrets is to paste an API key into code and push it to Git. The fastest way to create a “works on my machine” problem is to bake configuration into your source. Deployment forces you to separate code (stable logic) from configuration (values that differ per environment).

Use environment variables for anything sensitive or environment-specific: AI API keys, model names, base URLs, feature flags, and even your safety thresholds. In practice, your code should read configuration at startup and validate it. If a required variable is missing, fail fast with a clear error message. This is better than deploying a broken app that only fails after a user clicks a button.

Keep your configuration minimal and explicit. Beginners often over-configure too early. A good tiny-feature set might be: AI_API_KEY, MODEL, MAX_OUTPUT_TOKENS, REQUEST_TIMEOUT_MS, and ENV (local/staging/prod). If your feature includes a “strict mode” for safety, consider a SAFETY_MODE=strict|standard flag so you can test changes in staging before production.

Also treat prompts as configuration in the sense that they will evolve. Store prompts in version-controlled files (not only inline strings) so you can review changes like any other code change. If you keep prompts in code, at least isolate them in one module and add a version label (for example, PROMPT_VERSION=2026-03-26). That version label becomes useful later when you compare feedback and metrics across versions.

• Common mistake: Logging full user text plus API keys “for debugging.” Never log secrets, and be careful with personal data.
• Practical outcome: You can deploy the same code to staging and prod by changing only environment variables.

Monitoring is not fancy dashboards. Monitoring is answering four questions quickly: Is it up? Is it failing? Is it slow? Is it getting expensive? For a tiny AI feature, you can start with basic logging plus a small set of counters and timers.

Uptime: at minimum, know whether requests are reaching your server and returning a response. Even if you don’t run an external uptime checker, your host logs can show whether traffic is being served. A simple “health” endpoint can help (for example, returns 200 OK if the app can start and has required config).

Errors: count failures by category. Separate user errors (bad input, missing fields) from system errors (timeouts, model API failures) and safety blocks (content filtered, policy refusal). This breakdown matters because the fix differs. If 30% of “errors” are actually users submitting empty text, the fix is UI validation, not model tuning.

Latency: track total request time and the AI API call time. AI calls often dominate latency. Set a timeout and handle it gracefully—return a helpful message and allow the user to try again. Latency data also guides prompt optimization: shorter prompts and smaller outputs often speed things up.

Cost: even a demo can surprise you. Track usage in a way you can audit: number of requests, approximate tokens (if available from the API), and the model used. Cost monitoring is a safety feature for your budget. Add a basic rate limit (per IP or per user session) and a max input length to prevent abuse.

• Minimal monitoring checklist: request ID per call, status code, error category, total latency, AI latency, model name, output token count (if available), and a redacted input size (length, not full text).
• Common mistake: logging raw user inputs forever. Prefer short retention and redaction; store only what you truly need to improve.

The practical outcome: when something goes wrong in production, you can answer “what happened” in minutes, not hours.

Your v1 evaluation was based on a small set of examples you created. Real users will produce different inputs, different expectations, and different edge cases. A feedback button turns those moments into data you can learn from—but you must collect it responsibly.

Implement a simple feedback flow: next to the AI output, provide “Helpful” / “Not helpful” plus an optional comment box. When feedback is submitted, store a compact record tied to a request ID: timestamp, model/prompt version, user rating, and (optionally) the input/output text. “Optionally” is important. Do not store personal or sensitive data by default. Prefer storing redacted text, partial snippets, or a hashed identifier, and ask for explicit consent if you plan to retain full text for improvement.

Create a review workflow. Feedback is only useful if someone looks at it. A lightweight workflow for a solo builder is: review new “Not helpful” items weekly, label the failure type, and decide an action. Failure types might include: wrong format, hallucination, too verbose, missed constraint, unsafe content, or unclear user input. Store the label alongside the feedback so you can see patterns.

Be careful not to build a “shadow dataset” that violates privacy expectations. Set retention limits, restrict access, and document what you collect. If your feature targets a domain like health, finance, or education, be extra conservative: store less, not more. The practical outcome is a steady stream of improvement examples without creating a risk you can’t manage.

• Practical outcome: A user can submit feedback in one click, and you can review a list of issues with enough context to act.
• Common mistake: collecting feedback but not capturing prompt/model version—then you can’t connect improvements to outcomes.

After you collect feedback, the temptation is to “just tweak the prompt.” Sometimes that works, but it can also introduce regressions: fixing one case breaks three others. The safer approach is an iteration loop that updates prompts, tests, and rules together.

A practical loop looks like this: (1) pick the top 5 failure examples from feedback, (2) classify them by type, (3) propose a change (prompt instruction, output schema, input validation, or post-processing rule), (4) add or update test cases that represent the failures, (5) run the full checklist, and (6) deploy to staging before production.

Prompts are not magic; they are an interface contract. If your AI feature needs a specific output format, enforce it. Ask for structured output (like JSON) and validate it. If validation fails, retry once with a stricter instruction, then fall back to a safe message. This is basic failure handling, and it’s often more effective than endless prompt edits.

Update your rules when you see repeated safety issues. For example, if users frequently paste personally identifying information, add a warning and auto-redact patterns before sending text to the model. If users attempt disallowed content, block and explain. Track how often blocks occur; a high block rate might indicate unclear UI or misaligned user expectations.

The practical outcome is confidence: you can improve quality without guessing, because every change is tied to real examples and protected by repeatable tests.

• Common mistake: changing prompt wording without updating tests—then you can’t prove improvement.
• Engineering judgment: fix the system around the model (validation, UX, guardrails) before you reach for larger models or higher temperature.

Shipping is a checklist discipline. It’s how you turn “cool demo” into “trustworthy feature.” Before you call v1 done, walk through a short, repeatable list. This list also makes your project easy to present to stakeholders because you can explain what you built and how you reduced risk.

• Deployment: app has a stable URL; redeploy process is documented; staging (optional) exists for testing changes.
• Configuration: secrets are in environment variables; missing config fails fast; prompt and model versions are tracked.
• Safety: input limits, timeouts, and clear user messages exist; basic content checks and refusal handling are implemented.
• Monitoring: logs include request IDs; you track error rate and latency; you can estimate usage and cost.
• Feedback: “Helpful/Not helpful” works; review workflow is defined; example storage is minimized and documented.
• Evaluation: you have a small regression test set; a checklist defines “good enough”; changes are measured before/after.

To package your project for a portfolio or stakeholder demo, include three artifacts: a one-page README (what the feature does, inputs/outputs, safety limits, and how to run it), a short demo script (two good cases and one failure case with graceful handling), and a “learning log” (top issues found from feedback and what you changed). These artifacts signal engineering maturity: you didn’t just build; you shipped responsibly.

Next steps in MLOps, once you’re ready, are incremental: add automated deployments, structured tracing, a privacy review checklist, and a more formal dataset curation process. But don’t skip the basics. For a tiny AI feature, the biggest wins come from clear contracts, simple monitoring, and a tight feedback loop. That is how you improve safely—one small, measurable iteration at a time.