Non-Technical to AI Builder: Ship a RAG App in 30 Days

RAG is a simple idea: before the model answers, you first look up relevant information from your own documents, then you ask the model to answer using only that retrieved information. Think of the model as the “writer” and your document store as the “library.” The system retrieves a few pages (or chunks) from the library, and the model writes an answer that references those chunks. If you do it well, the model becomes more accurate, more up-to-date, and more auditable because it can cite sources.

RAG is not “training the model on your data.” Your documents are not permanently baked into the model; they are fetched at question time. This is good for beginners: you can update content without retraining, and you can often avoid expensive or risky fine-tuning. It also enables guardrails: when the retriever finds nothing relevant, your app can say “I don’t know” rather than guessing.

When should you use RAG vs fine-tuning? Use RAG when the answer must match specific internal policies, manuals, contracts, knowledge base articles, or changing information. Use fine-tuning when you need a consistent writing style, structured outputs, domain-specific formatting, or behavior changes (e.g., a customer support voice) that don’t depend on constantly updated facts. Many real systems do both: RAG for facts, fine-tuning (or strong prompting) for tone and format.

Engineering judgment: RAG quality is usually limited by retrieval, not generation. If the right chunk is not retrieved, even the best model can’t answer correctly. So your plan must cover ingestion, chunking, metadata, retrieval settings, and evaluation—not just prompt crafting. Common mistakes include uploading messy PDFs without cleaning, using huge chunks that bury the relevant sentence, and skipping citations so you can’t diagnose errors.

Pick one high-value use case with a clear user and a clear decision it supports. “Chat with our documents” is too broad. A good 30-day RAG use case has three properties: narrow scope (one domain), measurable success criteria (you can tell if it worked), and demo-ability (a short live walkthrough proves value).

Start by writing a one-sentence problem statement: “Help who answer what kind of questions using which documents in what context.” Examples: “Help HR coordinators answer benefits eligibility questions using the 2026 benefits guide and policy PDFs.” Or: “Help sales engineers find approved security answers using our security questionnaire responses and SOC2 excerpts.”

Define success criteria before you build. Create 10–20 representative questions (real ones if possible) and specify what a good answer looks like. Include constraints like: must cite sources, must refuse when the policy doesn’t cover the question, and must not invent numbers or dates. Your criteria can be simple at first: “At least 80% of answers are supported by a correct citation and require fewer than two follow-up clarifications.”

Map the RAG pipeline end-to-end for this use case: inputs (PDFs, docs, web pages), processing (cleaning and chunking), storage (embeddings + metadata), retrieval (top-k + filters), optional reranking, response generation (prompt + citations), and outputs (final answer + source links). Keeping this map visible prevents you from over-optimizing one step while ignoring missing steps elsewhere.

Common mistakes: choosing a use case that requires private data you can’t access yet, selecting a domain with no stable “source of truth,” or picking a task that is really automation (workflows) rather than retrieval (knowledge). If your use case requires the model to calculate or execute transactions, keep that out of scope for the first demo—make it “answer with citations” first.

Your demo is only as credible as its data—and data has rules. In week one you should decide what documents you can legally ingest and whether they can be sent to a hosted API. The safest early path is to use documents you own or that are explicitly licensed for your use. If you work at a company, treat internal documents as sensitive by default and get written permission for any demo that leaves your laptop.

Practical categories of acceptable sources include: publicly available documentation from your organization, PDFs you authored, internal policies approved for internal use, and web pages with terms that allow access and reuse. For third-party sites, don’t assume “public” means “free to scrape.” Review terms of service, copyright notices, and robots.txt. For a course project, you can also use open licenses (Creative Commons) or official government publications.

Design your ingestion plan around legality and privacy. If documents contain personal data (names, addresses), credentials, customer info, or regulated content, either exclude them or anonymize them before ingestion. If you’re using a hosted embedding or LLM service, verify whether they retain data for training and whether you can opt out. If you’re unsure, keep the demo local using openly licensed documents until you get clarity.

• Rule of thumb: if you can’t share the document with a coworker, don’t upload it to an external API without explicit approval.
• Keep provenance: record where each file came from, when it was downloaded, and its license/permission status.
• Prefer a “single source of truth”: pick documents that represent policy, not informal notes.

Common mistakes include mixing “official” and “draft” versions, ingesting duplicate PDFs, and ignoring versioning. Your RAG app may confidently cite the wrong version if you don’t track effective dates. Plan to store metadata like document title, URL or file path, version/date, and access level, because you’ll use it later for filtering and better citations.

You can ship a RAG demo with either a hosted stack (cloud LLMs + managed vector database) or a local stack (open-source models + local vector store). Your choice should match your constraints: privacy, budget, speed to implement, and reliability. In a 30-day timeline, prioritize “works end-to-end” over “perfect architecture.”

Hosted stack advantages: fast setup, strong model quality, fewer performance headaches, and easy sharing of a demo. Typical components: a hosted LLM API, an embeddings API, and a managed vector database (or a lightweight hosted service). Tradeoffs: data governance, recurring costs, and dependency on external uptime. Hosted is usually the best option for a first build if your documents are non-sensitive or approved for external processing.

Local stack advantages: maximum privacy and offline capability. You can run embeddings and even generation locally, and store vectors in a local database. Tradeoffs: you must manage performance, model downloads, GPU/CPU limitations, and sometimes lower baseline quality. Local is appropriate if you handle sensitive documents or can’t rely on external services.

Engineering judgment: keep the stack minimal. You need four capabilities: (1) document parsing and chunking, (2) embedding generation, (3) vector search with metadata filters, and (4) a chat/generation layer that can cite sources. Avoid adding a second reranker model, a complex agent framework, or multiple databases until you can answer your test questions with citations.

Set up accounts and tooling early: create API keys, store them in environment variables, and confirm you can run a “hello world” embedding + query before ingesting real data. Many beginners lose days on permissions, billing limits, or mismatched SDK versions. Your goal this week is a minimal project workspace that can: index 3–5 documents, retrieve 3 chunks for a question, and show them on screen. That is your first end-to-end proof.

A RAG app is a small system, not a single script. A clean project structure makes debugging and iteration easier, especially when you start evaluating quality. Even if you are new to coding, you can follow a simple layout that separates data, configuration, and pipeline steps.

Use a minimal repository structure like: a data/ folder for raw documents (or links), a processed/ folder for cleaned text and chunks, an index/ folder for vector store artifacts (or scripts that build them), and a app/ folder for your retrieval + generation code. Keep configuration (model names, chunk size, top-k) in a single config file so you can change settings without editing multiple scripts.

• Never commit secrets: keep API keys in .env and add it to .gitignore.
• Log everything important: for each query, log retrieved chunk IDs, scores, and final citations. This is your debugging lens.
• Version your data: record document versions and ingestion dates; otherwise you can’t reproduce results.

Repo hygiene is not bureaucracy—it’s how you move fast. When an answer is wrong, you need to know whether the failure came from parsing (text missing), chunking (wrong boundaries), embeddings (poor semantic match), retrieval settings (top-k too small), or generation (prompt too loose). If your artifacts are scattered, you will “prompt your way out” of retrieval problems and get inconsistent behavior.

Practical outcome for this chapter: create your repo and run a single command (even a simple script) that builds the index from a small document set. Then run a second command that answers a question and prints: answer, citations, and retrieved passages. This is the skeleton you will improve in later weeks.

Your 30-day plan should have weekly deliverables and a clear “definition of done” for the demo. A useful pattern is to plan backwards from demo day. First, write your demo script: the exact story you will tell in 3–5 minutes, the 3–6 questions you will ask live, and what you will show on screen (retrieved sources, citations, refusal behavior). Then turn that into an acceptance checklist: conditions that must be true for you to call it shipped.

Suggested weekly milestones: Week 1—choose use case, success criteria, legal data sources, and run an end-to-end skeleton on a tiny dataset. Week 2—build ingestion properly: cleaning, chunking, metadata, indexing, and basic retrieval. Week 3—add reranking (if needed), citations, prompts and guardrails to reduce hallucinations, and refusal rules when evidence is weak. Week 4—evaluation: run your test questions, inspect failures, tune chunking/retrieval, and polish the demo UI and script.

Track risks explicitly. Common risks include: document parsing errors (tables, scanned PDFs), scope creep (adding agents, multi-step workflows), weak evaluation (no test set), and data governance delays. Mitigations are practical: start with clean documents first, keep a small “golden set” of questions, and require citations for every answer. If citations look wrong, fix retrieval before rewriting prompts.

Your definition of done should be concrete. Example checklist items: (1) answers include 1–3 citations with working links or doc/page identifiers, (2) the system refuses when no relevant chunks are retrieved, (3) 80%+ of test questions are answered with correct supporting evidence, (4) logs show retrieved chunks and scores for each query, and (5) a new document can be added and indexed in under 10 minutes. If you can meet this checklist, you have a real RAG app—not a chat toy.

Section 2.1: Data inventory: formats, size, and update cadence

Before you parse anything, inventory your data like a product owner would: what sources exist, how trustworthy they are, and how often they change. This prevents a classic mistake in RAG projects: spending days perfecting ingestion for a dataset that updates weekly, without a plan to rerun it.

Create a simple “data register” (spreadsheet, Notion page, or YAML/JSON file) that lists each source with: (1) source type (PDF, docx, web page, wiki, ticket export), (2) location (URL, S3 path, Drive folder), (3) owner (who can clarify content), (4) licensing/permissions, (5) update cadence (static, monthly, daily), (6) approximate size (#files, total MB, or #pages), and (7) intended use in the app (policy answers, product specs, troubleshooting).

Document provenance from day one. For each source, record where it came from and when you pulled it. If you scrape web pages, store the crawl timestamp and the exact URL. If you export from a tool, store the export settings. Provenance is how you earn trust: when someone asks “Why did the bot say that?”, you can point to the source and version.

• Starter dataset tip: begin with 20–100 representative documents. Choose breadth (different topics and formats) over sheer volume.
• Update cadence rule: if content changes more than monthly, plan ingestion as a scheduled job from the start—even if it’s a simple nightly script.
• Common mistake: mixing “official” and “unofficial” docs without labeling them. In RAG, ambiguity becomes retrieval noise.

Your inventory also guides chunking and metadata later: manuals need sectioning; support tickets need timestamps and tags; web pages need canonical URLs. Treat this as your map before you start digging.

Section 2.2: Text extraction basics (PDFs, HTML, docs)

RAG retrieval works on text, not “documents.” Your first technical job is converting files and pages into normalized plain text while keeping enough structure to cite sources later. Different formats fail in different ways, so plan for format-specific extraction.

PDFs: PDFs are the most deceptive format. Some are “digital text” (copy/paste works), others are scans (images). For digital PDFs, extract text plus page numbers. For scanned PDFs, you need OCR (optical character recognition). OCR introduces errors (e.g., “1” vs “l”), so you should flag OCR-derived text in metadata to handle it cautiously in evaluation.

HTML/web pages: HTML contains navigation, ads, cookie banners, and repeated UI elements. Extract only the main article content. Preserve headings when possible (H1/H2/H3) because they become useful boundaries for chunking. Always store the final resolved URL and, if available, the canonical URL to avoid duplicates.

Docs (docx, Google Docs exports): These usually extract cleanly, but watch for tables and lists. Tables can flatten into unreadable text if you don’t handle them; sometimes the best “good enough” approach is to convert tables into a simple line format (e.g., “Field: Value”) or keep them as Markdown-like rows.

• Normalization step: convert all text to UTF-8, normalize whitespace, and standardize newlines.
• Structure retention: keep paragraph breaks and headings; don’t over-flatten everything into one blob.
• Common mistake: losing page/section boundaries. Without them, your citations become vague (“somewhere in the PDF”).

At the end of extraction, aim for a consistent intermediate representation: one record per document with fields like doc_id, source_uri, title, raw_text, and optional page_map or section_map. This becomes the input to cleaning and chunking.

Section 2.3: Cleaning rules: headers, footers, duplicates, noise

Cleaning is where you remove the “stuff humans ignore but retrievers don’t.” Vector search will happily match on repeated footer text, boilerplate disclaimers, or navigation menus. If those tokens dominate your chunks, your RAG app retrieves irrelevant passages even when the correct answer exists.

Start with explicit, explainable rules rather than opaque heuristics. For PDFs, remove repeated headers/footers by detecting lines that appear on most pages (e.g., the same company name, document code, or “Confidential”). For HTML, strip navigation blocks, cookie notices, and “related articles” lists. For docs, remove repeated templates like “Revision history” if it’s not useful for answering user questions.

Next, address duplicates. Duplicates come from multiple exports of the same doc, mirrored web pages, or “print view” vs “web view.” Compute a lightweight fingerprint (e.g., hash of normalized text) per document and flag near-identical ones. Keep one canonical version and record the relationship in metadata so you can trace where it came from.

• Noise to remove: page numbers, isolated single-character lines, legal boilerplate that appears everywhere, table-of-contents dumps (unless you need them).
• Noise to keep: definitions, parameter limits, warnings, and exception notes—even if they look repetitive. These are often what users ask about.
• Common mistake: “over-cleaning” that deletes headings or bullet structure. That structure improves retrieval and helps the model quote accurately.

Make cleaning auditable. Store both raw_text and clean_text (or keep a diff log) so you can debug later when someone asks why a line disappeared. Cleaning is not a one-time step; it evolves as you discover failure modes in evaluation.

Section 2.4: Chunking strategies and overlap heuristics

Chunking is the bridge between documents and retrieval. If chunks are too large, retrieval returns a lot of irrelevant text and the model may miss the key sentence. If chunks are too small, you lose context and the model may misinterpret details (like conditions, exceptions, or scope). Good chunking is an engineering tradeoff, not a fixed rule.

A practical default is “structure-first” chunking: split by headings and paragraphs, then enforce size limits. If you have headings, treat each heading section as a candidate chunk, then subdivide if it exceeds your target length. If you don’t have headings (common in messy PDFs), split by paragraphs and fall back to character/token windows.

Use overlap to preserve context across boundaries. Overlap helps when an answer depends on the end of one chunk and the start of the next (common with definitions followed by constraints). A simple heuristic: 10–20% overlap of your chunk size. For example, if your chunk target is ~800 characters (or ~200–300 tokens), overlap ~80–160 characters (or ~30–60 tokens). Adjust based on observed failures: if answers frequently miss prerequisites, increase overlap; if retrieval returns too many near-duplicates, decrease it.

• Chunk IDs: generate stable IDs like {doc_id}::chunk::{index} so you can re-ingest without breaking references.
• Preserve boundaries: avoid cutting mid-sentence when possible; it harms embedding quality and quotation.
• Common mistake: chunking purely by fixed length and ignoring headings. This destroys topical coherence, leading to “semantically mushy” embeddings.

Finally, test chunking with real queries. Pick 10 questions users will ask and manually confirm that at least one chunk contains a crisp, self-contained answer. If you can’t find it by skimming the chunks, the retriever won’t either.

Section 2.5: Metadata design: source, section, timestamps, tags

Metadata is what turns “retrieved text” into “retrieved evidence.” It powers citations, filtering, access control, and debugging. Many first-time builders skip metadata and later can’t answer basic questions like: Which document did this come from? How old is it? Is it the official policy or a draft?

Design metadata as a small, consistent schema that you attach to every chunk. At minimum, include: source_uri (file path or URL), source_type (pdf/html/docx), title, doc_id, chunk_id, and ingested_at timestamp. If you can, add published_at or last_modified_at from the source system. For PDFs, include page_start/page_end. For HTML, include canonical_url. For docs with headings, store section_path like “Onboarding > Security > Password Reset.”

Tags are your leverage. Add tags for product area, audience (“internal”, “customer”), region, or document status (“draft”, “approved”). These tags let you implement retrieval filters later (e.g., “only approved policies”) without redoing your pipeline.

• Citations: metadata should be sufficient to generate a human-clickable citation (URL or file + page).
• Debugging: include provenance fields like retrieved_from (“crawl”, “export”), and optionally a content_hash to detect changes.
• Common mistake: stuffing the entire document into metadata. Keep metadata small; store content in the chunk text field.

Good metadata also reduces hallucinations indirectly: when the model can cite a specific section and date, you can enforce guardrails like “answer only from retrieved chunks” and “include citations,” which makes it harder for unsupported claims to slip through.

Section 2.6: Ingestion checkpoints and reproducibility

A one-off ingestion run is a prototype; a rerunnable ingestion pipeline is a product. Your goal is to make data intake repeatable so you can update the knowledge base, fix parsing bugs, and regenerate chunks without guesswork. This is especially important in a 30-day build, where requirements and sources change midstream.

Break ingestion into checkpoints with artifacts you can inspect: (1) raw capture (downloaded PDFs, saved HTML, exported docs), (2) extracted text (one file/record per document), (3) cleaned text, (4) chunked dataset (chunks + metadata). Store these in a predictable folder structure or object store paths, versioned by run date. If a later step fails, you can resume without re-scraping or re-OCRing everything.

Make each run identifiable. Write a small “run manifest” that logs: ingestion version (git commit or script version), input sources and counts, start/end time, and basic stats (documents processed, chunks produced, failures). When you discover an issue—like a PDF that extracted empty text—you can trace exactly which run produced the bad chunks.

• Validation spot checks: sample 5–10 documents per format and compare raw vs cleaned text; ensure key sections exist.
• Edge cases to test: scanned PDFs, pages with two columns, documents with tables, very short pages, and duplicated web URLs.
• Common mistake: ignoring extraction failures. Empty or garbage text still gets embedded later, polluting retrieval.

End each run with a “quality gate”: fail the pipeline (or at least warn loudly) if too many documents have empty text, if average chunk length is far outside your target, or if duplicates spike unexpectedly. This habit—small, automated checks—will save you from shipping a RAG app that looks fine in demos but breaks in real usage.

Section 3.1: Embeddings explained for non-technical builders

An embedding is a numeric representation of text (a long list of numbers) that captures meaning. Two pieces of text with similar meaning end up with embeddings that are “close” to each other. This is why a user can search for “cancellation policy” and still retrieve a chunk that says “how to terminate your subscription” even if none of the words match exactly.

In practice, you’ll embed chunks of your documents, not whole PDFs. Each chunk becomes a record: (1) chunk text, (2) embedding vector, (3) metadata like document title, URL, date, section heading, and permission scope. When a user asks a question, you embed the question and perform a nearest-neighbor search to retrieve the most similar chunks.

• Cost basics: embedding cost typically scales with characters/tokens embedded. Chunking too small increases the number of chunks and the number of embeddings you pay for; chunking too large can reduce retrieval precision.
• Latency basics: query-time latency comes from (a) embedding the user query and (b) vector search. Embedding the entire corpus happens offline and should not impact user latency once stored.

Common mistake: treating embeddings as magic. Embeddings can only retrieve what you stored. If your chunk text is messy (headers repeated, navigation junk, broken sentences), the embedding will faithfully encode that mess. Another mistake: assuming higher top-k always helps. Pulling too many chunks increases noise and can confuse later steps.

Practical workflow: (1) embed 50–200 chunks first, (2) run a few representative queries, (3) inspect the top results manually. If results look off, fix chunking and metadata before you embed the entire corpus.

Section 3.2: Choosing an embedding model: quality vs cost

Embedding model choice is one of the highest-leverage decisions in a RAG system because it determines how well “meaning” is captured. You’re balancing three things: retrieval quality, cost per embedded token, and latency. As a non-technical builder, you don’t need to memorize model internals—focus on selecting a model that fits your use case and validating it with a small experiment.

Use a higher-quality embedding model when: your domain language is specialized (legal, medical, engineering), your queries are short and ambiguous, or your documents contain many near-duplicates where subtle differences matter. Use a cheaper model when: documents are straightforward (FAQs, simple policies), your budget is tight, or you can compensate with strong metadata and reranking.

• Rule of thumb: optimize for retrieval quality first, because poor retrieval forces you to spend more on generation and manual support later.
• Batching: embed documents in batches offline. This reduces overhead and makes cost predictable.
• Idempotency: store a content hash in metadata so you only re-embed chunks that actually changed.

Common mistake: re-embedding everything every time. Instead, treat embeddings like a build artifact. Your pipeline should detect changes and only update what’s necessary. Another mistake: comparing models by “how good answers feel” rather than retrieval quality. In this chapter, you’ll build a retrieval scorecard so you can compare embedding models using the same set of test queries and expected sources.

Practical outcome: choose one embedding model for v1, document why you chose it, and keep the door open to upgrading later. Changing embedding models later is possible—but it usually requires re-embedding the corpus—so make it a conscious decision.

Section 3.3: Vector database concepts: indexes and similarity

A vector database stores embeddings and supports fast “nearest neighbor” search. Conceptually, you are asking: “Which stored chunk vectors are closest to this query vector?” Most vector databases also store metadata and support filtering, which is critical for real apps (for example, show only content a user is allowed to see, or only the latest policy version).

Two terms matter: similarity metric and index. Similarity metric is how closeness is computed (often cosine similarity or dot product). An index is a data structure that makes search fast at scale. Many vector databases use approximate nearest neighbor indexing to trade a tiny amount of accuracy for large speed gains. For your first version, you don’t need to tune index parameters deeply, but you do need to confirm retrieval works end-to-end.

• Loading vectors: upsert records with a stable ID, the embedding vector, the chunk text (or a pointer to it), and metadata.
• Confirm retrieval: run a query you know should match a specific chunk and verify it appears in the top-k results.
• Metadata filters: filter by document type, product line, date range, language, permission tier, or source URL.

Common mistake: skipping metadata. Without metadata you can’t filter, and without filters your app will retrieve irrelevant or unauthorized content. Another mistake: storing only embeddings and not the source reference. You need citations later, so store fields like doc_id, source_url, page_number, and chunk_index from day one.

Practical outcome: you should be able to answer, with evidence, “Yes—vector search returns the right chunks for these ten queries,” before you ever add generation.

Section 3.4: Query expansion and hybrid search overview

Vector search is powerful, but it’s not the only retrieval signal. Two techniques often improve real-world relevance: query expansion and hybrid search. Query expansion means rewriting or enriching the user’s query to better match your corpus language. Hybrid search means combining vector similarity with keyword-based search (like BM25) to handle exact terms, identifiers, and edge cases.

Query expansion is helpful when users ask vague questions (“What’s the policy?”) or use different phrasing than your documents. You can expand the query by adding synonyms, product names, or a more explicit intent. This can be done with a small prompt to a language model that outputs 2–5 alternate queries, which you then search and merge.

• When hybrid search shines: part numbers, error codes, API parameter names, legal clause numbers, and names that must match exactly.
• Engineering judgement: start with pure vector search; add hybrid when you observe failures involving exact tokens.
• Top-k tuning: increase top-k slightly when using multiple query variants, but cap it to avoid flooding downstream steps with noise.

Common mistake: expanding queries without constraints. If expansion introduces incorrect product names or unrelated topics, retrieval degrades. Keep expansions short, domain-specific, and logged. Another mistake: assuming hybrid search always improves results. If your documents contain boilerplate repeated on every page, keyword signals can overweight noise. This is why your chunk cleaning work matters.

Practical outcome: a retrieval strategy you can evolve—vector-only for v1, then optional expansion and hybrid signals to address specific, observed failure modes.

Section 3.5: Reranking fundamentals and when it matters

Reranking is a second-stage retrieval step that takes an initial set of candidates (say top 20 from vector search) and re-orders them using a stronger, more expensive relevance model. Think of vector search as “fast recall” and reranking as “precise selection.” This often produces a noticeable jump in answer quality because the generator sees cleaner, more relevant context.

When reranking matters most: your corpus is large, many chunks are semantically similar, queries are nuanced, or the cost of a wrong answer is high (support, compliance, healthcare). When it matters less: small corpora, highly structured FAQs, or when your metadata filters already narrow the space dramatically.

• Typical flow: retrieve top-k=20–50 quickly, rerank to top-n=5–10, then pass only those to the language model.
• Latency tradeoff: reranking adds a network call and computation; measure the impact on user experience.
• Practical guardrail: if the reranker confidence is low, return fewer chunks and ask a clarifying question instead of forcing an answer.

Common mistake: reranking too few candidates. If you only rerank the top 5, you may never give the reranker a chance to fix a slightly-missed retrieval. Another mistake: reranking too many and blowing up latency. Start with 20–30 candidates, measure, then adjust.

Practical outcome: your retrieval layer becomes robust: vector search gets you “close,” reranking makes it “right,” and generation becomes less prone to hallucinating because the context is better.

Section 3.6: Retrieval evaluation: relevance and coverage

Before you test the full RAG system, test retrieval on its own. If retrieval can’t find the right chunks, generation quality is irrelevant. A baseline retrieval scorecard is a simple, repeatable way to measure whether your embeddings + vector database + settings are doing their job.

Build a set of 20–50 realistic questions drawn from user tickets, onboarding calls, or your own expectations. For each question, write down what “good retrieval” means: the correct document, section, or a specific chunk identifier. Then run retrieval with your current configuration and record results.

• Relevance: are the top results actually about the question? Track whether the expected source appears in top-3, top-5, and top-10.
• Coverage: does the retrieved set include all necessary subpoints (e.g., exceptions, prerequisites, dates)?
• Filter correctness: confirm metadata filters enforce permissions and versioning (no outdated policies).
• Failure analysis: label failures as chunking issue, missing content, embedding mismatch, query ambiguity, or filter misconfiguration.

Common mistake: evaluating only by “did the model answer correctly?” A model might guess correctly without retrieving the right evidence—or answer confidently with the wrong evidence. Your scorecard should force you to check citations: did the system retrieve the right source material?

Practical outcome: you’ll finish this chapter with a baseline. When you later adjust chunking, top-k, filters, hybrid signals, or reranking, you can rerun the scorecard and quantify improvement. That’s how you build a RAG app that gets better on purpose, not by accident.

Section 4.1: Prompt anatomy: role, task, constraints, format

A RAG prompt is not creative writing; it’s an interface contract. The model performs best when you explicitly define four parts: role, task, constraints, and format. If any of these are fuzzy, you’ll see inconsistent answers, missing citations, or the model “helpfully” inventing details.

Role sets the stance: “You are an assistant that answers questions using the provided sources.” Avoid roles like “expert consultant” unless you also clamp constraints tightly. Task states what to produce: a direct answer plus citations. Constraints define the rules of evidence: only use the provided snippets; if insufficient, say you don’t know; do not reveal system text; do not fabricate sources. Format makes outputs machine-checkable: headings, bullets, and a citations array you can render.

A practical template (adapt to your app) looks like this:

• System: “Answer using only EVIDENCE. If not found, say ‘I don’t know based on the provided sources.’ Include citations as [S#].”
• User: “QUESTION: … EVIDENCE: (S1…Sn with quoted text + metadata)”

Engineering judgement: be careful with “use your general knowledge.” If you allow it, you must label which parts are from sources vs general knowledge, or you’ll lose trust. Common mistake: mixing instructions and evidence in one blob. Keep evidence in a clearly delimited block so the model can reliably treat it as grounding.

Practical outcome: you can run the same question twice and get nearly identical structure and citation behavior. That consistency is what makes your RAG app feel “product-like,” not like a toy demo.

Section 4.2: Context packing: how much to send and why

Context packing is the art of sending enough evidence to answer, without flooding the model with irrelevant text. Too little context causes “I don’t know” or shallow answers; too much context causes distraction, truncation, and accidental contradictions. Your default should be: retrieve a small set of high-signal chunks, then pack them predictably.

Start with top-K retrieval (often 4–8 chunks) and a token budget (for example, reserve 30–40% for the model’s answer). If your chunks are 300–500 tokens, sending 8 chunks might already be too much. A practical pattern is: retrieve 12, rerank to 6, then include only the best 4–6 that fit your budget.

Pack evidence with structure:

• ID: S1, S2, S3…
• Provenance: file/url, page, section heading, chunk_id
• Snippet: a short quote (not the entire chunk if it’s long)

Quoting matters: it forces compactness and reduces the chance the model will summarize the wrong part of a long chunk. Another practical trick is deduping: many vector searches return near-duplicate chunks from the same page. Deduplicate by (document_id, page, heading) or by similarity between chunks before packing.

Common mistake: sending raw chunks without titles or page numbers. The model can’t cite well if you don’t give it citation-friendly metadata. Another mistake: packing the user’s entire chat history plus all retrieved chunks—this is a fast path to context overflow and weird answers. Keep evidence tight; keep history separate and condensed (we’ll cover that in Section 4.5).

Practical outcome: you can explain exactly why a chunk is included (“top reranked evidence within budget”), and you can reproduce failures by logging the packed evidence used for each answer.

Section 4.3: Citation patterns: snippets, links, and provenance

Citations are your main anti-hallucination tool because they make the model “show its work.” But citations only help if they’re real, verifiable, and connected to the claim. Aim for claim-level citations: when the answer states a fact, it tags the supporting source(s). Avoid dumping a list of sources at the end with no mapping—that’s citation theater.

Use a simple pattern in the answer text: “... according to the policy, the retention period is 30 days [S2].” Then provide a citations block your UI can render:

• [S2] Document title, link or file path, page/section, and a short quoted snippet.

The quoted snippet is crucial. It lets users verify quickly, and it gives you a debugging handle: if the snippet doesn’t actually support the claim, the issue is either reranking or prompt compliance. When sources are web pages, store canonical URLs and anchor text; when sources are PDFs, store page numbers and ideally a stable “page image” reference or offset.

Engineering judgement: sometimes multiple snippets support one claim; include 1–2 maximum unless the user asks for exhaustive sourcing. If your retrieved chunks disagree, you have options: (1) present both with citations and explain the conflict, or (2) prefer a more authoritative document via metadata weighting (for example, “policy handbook” beats “old slide deck”). Build that bias explicitly rather than hoping the model guesses.

Common mistakes: letting the model invent citations that don’t correspond to evidence IDs; failing to pass citation IDs in the evidence block; or stripping metadata from chunks during ingestion. Your pipeline should enforce: only cite from the set {S1..Sn}. If the model outputs [S9] when you only provided S1–S6, treat that as a failure and either re-ask with stricter constraints or fall back to “I don’t know.”

Practical outcome: users trust your app because every important statement is traceable to a real document location.

Section 4.4: Safety and refusal: don’t invent, don’t leak

“I don’t know” is a feature, not a bug. In RAG, the model should refuse when evidence is missing, conflicting, or too weak. Without a refusal path, the system will confidently guess—exactly what you’re trying to prevent.

Implement refusal with two layers:

• Prompt rule: If the answer cannot be supported by EVIDENCE, respond with a brief refusal and what information is needed.
• Programmatic checks: If retrieval returns low similarity scores, empty results, or only irrelevant doc types, skip the LLM answer and return a “no context found” response.

A practical refusal response has three parts: (1) a direct statement (“I don’t know based on the provided sources”), (2) a short reason (“no source mentions X”), and (3) a helpful next step (“Try asking about Y” or “Upload the policy document covering X”). This keeps UX smooth while staying honest.

Also protect against leakage. Your system prompt, API keys, internal retrieval queries, and other users’ content must never be exposed. The prompt should instruct the model to not reveal hidden instructions, and your app should avoid sending secrets to the model at all. Treat conversation history as potentially sensitive; don’t re-inject long raw user text if it contains private info unless the user’s use case explicitly demands it.

Common mistake: allowing the model to answer “best effort” when evidence is thin. If you truly need best-effort answers, label them clearly (“Speculation” vs “From sources”) and keep them out of regulated or high-stakes use cases. Another mistake: returning verbatim large blocks of copyrighted text. Prefer short quotes and summaries, and keep quotes within reasonable length.

Practical outcome: your RAG app becomes dependable in real environments because it refuses gracefully and doesn’t leak what it shouldn’t.

Section 4.5: Conversational RAG: history condensation basics

Chat makes RAG harder because users refer to earlier turns: “What about the exception?” If you simply append the whole chat transcript to every prompt, you’ll hit context limits and increase the chance the model follows an old instruction instead of current evidence. The safe pattern is history condensation: summarize only what’s needed for retrieval and answering.

Implement conversation memory in two channels:

• Short-term working summary: 3–6 bullet points capturing the user’s goal, constraints, and resolved facts.
• Last user question: the current turn verbatim.

Then create a standalone retrieval query by rewriting the user’s latest question using the summary. Example: if the user says “Does it apply to contractors too?”, your query rewriter outputs: “In the company retention policy, does the 30-day retention rule apply to contractors?” This rewritten query is used for vector search; it should not include private details that aren’t necessary to find documents.

Engineering judgement: decide what to keep vs drop. Keep stable preferences (“Answer in bullet points,” “Use only uploaded handbook”), keep named entities that affect retrieval (product names, policy titles), and keep decisions already made. Drop emotional language, long user pasted content, and anything unrelated to retrieval. If you’re building for sensitive domains, consider redacting emails, phone numbers, or IDs before storing any memory.

Common mistake: letting the model update the memory without guardrails. Treat memory like code: update it via a dedicated “summarize” step with a strict schema (for example, JSON with keys like goal, constraints, entities). If the memory step fails or produces garbage, fall back to minimal memory rather than polluting future turns.

Practical outcome: your chat RAG remains fast, grounded, and coherent over many turns without blowing token budgets.

Section 4.6: UX basics for Q&A: clarity, follow-ups, and tone

A working local demo isn’t just backend plumbing; it’s a user experience. The interface should make it obvious what the system knows, where it got the answer, and what to do next. This is where non-technical builders can create an advantage: clear UX often beats fancy modeling.

Start with an answer layout that users can scan:

• Direct answer first (1–3 sentences).
• Key points (bullets) for details and edge cases.
• Citations rendered as clickable source cards with page/section and quoted snippet.

When context is missing, don’t just refuse—ask a precise follow-up: “Are you asking about the employee handbook or the contractor agreement?” This both improves retrieval and feels helpful. If the user’s question is ambiguous, ask clarifying questions before retrieving; otherwise you’ll retrieve broadly and pack noisy context.

Tone matters: be calm, factual, and consistent. Avoid overconfident language (“definitely,” “guaranteed”) unless the source is explicit. Also avoid long “AI disclaimers.” A simple, repeatable phrasing is enough: “I don’t know based on the provided sources.”

For your local demo, include a “debug drawer” that shows: the rewritten retrieval query (if using chat), the top retrieved chunk titles with scores, and the exact evidence sent to the model. This is invaluable when a stakeholder says “why did it answer that?” It also speeds up iteration on chunking, metadata, and reranking.

Common mistakes: hiding citations behind extra clicks; showing sources without quotes; or not indicating when an answer is partially supported. If only one part is grounded, cite that part and explicitly mark the rest as unknown.

Practical outcome: you can demo a real question-answer flow end-to-end on your laptop, with transparent citations and graceful follow-ups—exactly the kind of credibility you need to “ship” rather than just prototype.

A RAG app is only as good as the questions you test it with. Your goal is not to create a perfect benchmark; your goal is to create a small dataset that reflects real user needs and forces the system to use your knowledge base (not general web knowledge). Start with 25–60 questions you expect from actual users. Collect them from support tickets, Slack/Teams threads, onboarding emails, product FAQs, or by asking 3–5 stakeholders to write “what I would ask the bot” prompts.

For each question, add two fields: (1) the expected source(s) and (2) the acceptance criteria. Expected sources are usually document titles, URLs, or stable IDs for chunks/pages (not just “the employee handbook”). If you do not have stable IDs yet, add them now in ingestion (e.g., document_id, section, page, paragraph index). Acceptance criteria should be short and testable: what must be included, what must be excluded, and whether the answer must cite at least one source.

Keep the dataset honest by including edge cases: ambiguous questions (“What’s the policy?”), multi-hop questions (“What do I do first, then where do I file it?”), and “negative space” questions where the answer should be “I don’t know based on these docs.” This prevents overconfident hallucinations from appearing “good” in testing.

• Tip: Aim for coverage, not volume. 40 well-chosen questions often uncover more than 400 repetitive ones.
• Tip: Store the dataset in a simple CSV/JSON with columns: question, expected_doc_ids, notes, must_cite, category.

Once you can run this dataset end-to-end with one command, you have the foundation for continuous improvement.

Most RAG failures fall into two buckets: retrieval failures (wrong or missing evidence) and generation failures (bad use of evidence). Learning to label failures correctly saves days of trial-and-error.

Retrieval failures look like: the retrieved chunks are unrelated; they are on the right topic but miss the key detail; the right document exists but never appears in top results; or the query is interpreted too broadly. Common causes include poor chunking (chunks too big or too small), missing metadata filters (e.g., user region, product version), embedding mismatch (wrong model or mixed languages), or top-k being too low.

Generation failures happen when the model had enough evidence but still produced a wrong, vague, or policy-violating answer. Common causes include prompts that do not force grounding (“use the context”), prompts that do not require citations, lack of refusal behavior (“if not in sources, say you don’t know”), or context overload (too many chunks so the model misses the relevant line). You may also see citation errors: citing a source that doesn’t contain the claim, or failing to cite at all.

A practical method: for every failed test question, inspect the top retrieved chunks first. If the expected source is not in the retrieved set, label it retrieval. If it is present and clearly contains the answer, label it generation. If it is present but buried in noise, label it retrieval+ranking (the model may never notice it). This simple taxonomy makes your next action obvious.

Debugging RAG requires visibility into what the system saw. For each test query, capture a “retrieval report” containing: the normalized query, applied filters, the top-k chunks (with document_id and section/page), similarity scores, and optionally reranker scores. If you use hybrid search (keyword + vector), log both contributions. Your goal is to answer: “Did we retrieve the right evidence, and if not, why not?”

Start with three quick checks. Check 1: coverage. Is the expected document present anywhere in the top-k? If yes but low-ranked, you likely need reranking, better chunk boundaries, or query rewriting. Check 2: specificity. Are the retrieved chunks too general (table of contents, overview sections)? That often indicates chunks are too large or that you are embedding boilerplate repeatedly. Check 3: duplicates. Are multiple chunks nearly the same (headers/footers, repeated navigation text from web pages)? Duplicates waste context window and reduce answer quality.

Do not overinterpret raw similarity scores across different queries; treat them as relative within a query. What matters operationally is whether relevant chunks consistently appear above irrelevant ones. If you see relevant chunks with scores close to irrelevant ones, a reranker can help. If you see consistently low scores for everything, your query may not match doc language; consider adding synonyms, domain terms, or a light query rewrite step (“expand acronyms, include product names, remove chit-chat”).

Finally, always inspect the final prompt sent to the model (system + user + retrieved context). Many “mystery” bugs are simply truncated context, wrong formatting, or sources not being passed in the way the model can reliably use.

Once you can classify failures and inspect retrieval, you can tune quality using a small set of high-leverage knobs. Change one lever at a time, run the same test set, and compare results. This prevents placebo improvements.

Chunking: If answers miss key details, try smaller chunks (e.g., 200–500 tokens) with overlap (10–20%). If answers lack context or become fragmented, try slightly larger chunks. Use structure-aware chunking for PDFs and docs (headings, bullet lists, tables). A common mistake is splitting in the middle of a procedure: steps 1–3 in one chunk and steps 4–7 in another; the model then answers incompletely. Fix by chunking on section boundaries and keeping complete procedures together.

Top-k and reranking: Increasing top-k can improve recall (more chances to include the right chunk) but can hurt precision and increase token cost. Many teams use a two-stage approach: retrieve 20–50 candidates cheaply, then rerank down to 5–10. Reranking is especially valuable when your domain has similar-sounding sections (policies, terms, product variants).

Prompts and guardrails: Your prompt should explicitly require grounding and citations: “Answer only using the provided sources. If the sources do not contain the answer, say so and request clarification.” Add formatting rules (short answer + bullets, or step-by-step) and citation rules (“cite after each major claim”). If hallucinations persist, reduce the model’s freedom: shorter responses, avoid speculative language, and prioritize quoting exact lines for compliance-sensitive topics.

Metadata filters: Filters often create the biggest quality jump with the least cost. Examples: filter by customer plan, product version, region, language, document status (published vs draft), or recency. Without filters, retrieval can pull outdated policies and the model will confidently mix them. Tie filters to user context when available; otherwise, ask a brief clarifying question before retrieving.

If you want to improve quality after launch (and you will), you need observability from day one. At minimum, log each request with a trace ID and store: user query, rewritten query (if any), selected filters, retrieved chunk IDs with scores, reranked order, final prompt size, model name, tokens in/out, latency, and the final answer with citations. This makes every failure reproducible.

Design logs for humans and machines. Humans need a “debug view” that shows the retrieved chunks and why they were chosen. Machines need structured fields so you can aggregate: top failure categories, most expensive queries, documents that cause confusion, and questions with frequent “no answer” responses. A common mistake is logging only the final answer; that hides the real cause (often retrieval).

Add lightweight feedback capture in the UI: a thumbs up/down plus an optional “what was wrong?” tag (wrong source, outdated, missing detail, should refuse, too long). When a user flags an answer, store the exact retrieved chunk IDs. That turns feedback into actionable fixes: update docs, adjust filters, repair chunking, or add a prompt rule.

Be mindful of privacy and compliance. Redact sensitive fields, avoid storing secrets, and establish retention rules. If your domain is regulated, log access to sources and show citations so auditors can trace claims back to documents.

Quality is not the only ship criterion. A RAG app that is accurate but slow or expensive will not survive. Performance comes down to three practical budgets: latency (seconds), tokens (context size), and cost (per request and monthly).

Latency: Measure time spent in (1) retrieval, (2) reranking, and (3) generation. Retrieval is usually fast; reranking and generation dominate. If latency is high, reduce reranked chunks, use smaller reranker models, or cache frequent queries and their retrieval results. Also consider streaming responses so users see progress quickly.

Tokens: The biggest hidden cost is stuffing too many chunks into the prompt. More context is not always better; irrelevant context can degrade answer quality and increases tokens. Use a fixed “context budget” (e.g., max 6 chunks or max 3,000 tokens of context). Prefer fewer, higher-quality chunks via reranking. Remove repeated boilerplate during ingestion (headers, footers, navigation).

Cost control: Log tokens in/out and compute cost per request. Then set guardrails: maximum top-k, maximum reranked chunks, maximum answer length, and refusal when the question is out of scope. For launch, define a “good enough” bar that includes both quality and performance: for example, “80% of test questions retrieve an expected source in top-10, 70% produce a correct grounded answer with citations, p95 latency under 6 seconds, and average cost under $0.02 per query.”

When you can hit that bar consistently, you are ready to ship. After launch, keep running your test set weekly, add new real-user questions, and treat improvements as a disciplined loop: measure, diagnose, change one lever, and re-measure.

For a first public deployment, choose the option that minimizes “infrastructure homework” while still letting you control secrets and logs. Your goal is repeatable demos, not a perfectly optimized cloud architecture.

Managed platforms (recommended for beginners) include services like Render, Railway, Fly.io, Vercel (frontend) + a small API host, or Streamlit Community Cloud for simple demos. The benefits: push-to-deploy from GitHub, built-in HTTPS, simple environment variable UI, and basic logs. The tradeoff: limited customization, occasional cold starts, and sometimes constraints around background jobs (useful if your ingestion pipeline runs server-side).

DIY cloud usually means a VM (DigitalOcean/AWS Lightsail), a reverse proxy (Nginx/Caddy), and a process manager (systemd, PM2, Docker Compose). The benefit: full control and predictable behavior. The tradeoff: you own patching, firewall rules, TLS, and uptime. It’s a great learning path, but it can delay shipping.

• Good beginner default: Host your API (FastAPI/Flask/Express) on Render/Railway/Fly.io and your UI on Vercel, calling the API via HTTPS.
• If your app is one repo: Host a single container that serves the UI and API together.
• If ingestion is heavy: Keep ingestion as a local/admin-only command (run it from your laptop) or as a separate “job” service so your demo app stays fast.

Common mistake: deploying with “debug mode” enabled or relying on local files for your vector database. If the host restarts, your index disappears. For demos, prefer a managed vector store (or a persisted disk volume) and treat ingestion as a repeatable pipeline you can rerun.

Your deployed app will need secrets: LLM API keys, embedding model keys, vector database credentials, and sometimes a basic auth password. Never hardcode these in code or commit them to Git. In production-minded projects, secrets live outside the repo and are injected at runtime.

Use environment variables for anything sensitive or environment-specific. A typical set includes: OPENAI_API_KEY (or your provider), VECTOR_DB_URL, VECTOR_DB_API_KEY, APP_ENV, and BASIC_AUTH_USER/BASIC_AUTH_PASS. For local development, store them in a .env file that is excluded via .gitignore. For deployment, set them in your hosting platform’s “Environment Variables/Secrets” UI.

• Rule of thumb: if you’d be unhappy seeing it in a screenshot, it’s a secret.
• Separate dev vs prod: use different keys so you can revoke one without breaking everything.
• Rotate keys: if you accidentally expose a key (even briefly), revoke and reissue it immediately.

Engineering judgment: keep your configuration surface small. Beginners often add too many toggles and forget which ones matter. Start with the essentials: model name, max tokens, retrieval top-k, and the API keys. Document these in your README as a short “Configuration” section without revealing secret values.

Common mistake: printing secrets in logs during debugging. Treat logs as potentially public to teammates (and sometimes to third-party logging tools). Never log request headers that may contain tokens, and be careful when dumping “full request objects.”

A shareable RAG app should have a minimum security posture. You don’t need enterprise SSO to demonstrate good judgment; you need basic protections that prevent obvious abuse and accidental data leaks.

Add basic auth if your app is for demos. A simple username/password gate (or a one-time “demo token”) prevents random visitors from burning your API quota. If you have a small set of trusted testers, basic auth is enough. If you’re building a more serious product, consider OAuth (Google/GitHub) or a hosted auth provider.

Rate limit requests at the API layer to prevent cost spikes and denial-of-wallet scenarios. Even a lightweight limit (e.g., 30 requests/minute per IP) is a strong signal to recruiters that you understand operational risk. Combine it with request size limits so someone can’t paste a 2 MB prompt.

Content safeguards should cover two things: (1) prompt injection and (2) sensitive data. For prompt injection, treat retrieved documents as untrusted input and instruct the model to ignore instructions inside sources. For sensitive data, implement basic PII hygiene: don’t ingest private documents you’re not allowed to share, redact obvious identifiers in sample datasets, and avoid logging user prompts if they might contain personal information.

• Permissions: if you support multiple knowledge bases, ensure users can only query what they are allowed to access (separate indexes or filter by metadata).
• Citations: show sources so users can verify answers; this also reduces the risk of “authoritative-sounding” hallucinations.
• Default-deny mindset: if unsure whether data should be exposed, don’t expose it.

Common mistake: storing uploaded files in a public bucket without access controls. If you support uploads, store them privately and expire access links.

Reliability is where a demo becomes a product-like artifact. LLM APIs time out, vector databases have brief hiccups, and users ask questions your knowledge base cannot answer. Your app should fail in ways that protect trust.

Retries: add short, bounded retries for transient failures (timeouts, 429 rate limits, occasional 5xx). Use exponential backoff and stop after a small number of attempts (e.g., 2–3) to avoid cascading delays. If you retry on 429, respect Retry-After when provided.

Fallbacks: decide what happens when retrieval fails or returns low-quality results. Practical fallbacks include: (1) return a “not enough evidence” response, (2) ask a clarifying question, or (3) run a minimal answer mode that does not claim to use sources. The important part is that the UI makes this explicit so users understand the system state.

Graceful errors: users should see actionable messages (“The document index is updating—try again in a minute”) instead of stack traces. Internally, log correlation IDs so you can trace a failure across components. In your case-study README, mention that you implemented structured errors and basic observability (request timing, retrieval latency, and model latency).

• Timeout budgets: set maximum time for retrieval and generation so the app doesn’t hang.
• Validation: reject empty queries, overly long queries, and unsupported file types early.
• RAG-specific check: if top retrieved chunks have low similarity or fail reranking thresholds, refuse to answer confidently and cite that limitation.

Common mistake: treating every failure as “model is down.” Often the vector store is empty (ingestion didn’t run), metadata filters exclude everything, or chunking produced garbage. Your error messages should help you diagnose which layer failed.

Recruiters and hiring managers scan quickly. Your portfolio packaging should let them understand your project in under two minutes, then give them optional depth if they want it. Think of this as “product marketing,” but for your engineering judgment.

Portfolio page: include a short description (“RAG app that answers questions over X corpus with citations”), 2–3 screenshots (upload flow, answer with citations, failure mode), and a link to the live demo. Add a clear note about access (basic auth credentials shared on request) and what data is included (sanitized public docs).

2-minute demo video outline: keep it tight and scripted. Suggested structure: (1) problem statement (15s), (2) show ingestion/knowledge base (20s), (3) ask 2 questions: one easy, one tricky (45s), (4) point out citations and guardrails (20s), (5) reliability/security highlights (10s), (6) close with what you’d build next (10s). Record with your mic and a clean cursor; clarity beats production value.

Case-study README: write for skimmers. Use headings like “What it does,” “Architecture,” “Data pipeline,” “Safety & reliability,” “Evaluation,” “Tradeoffs,” and “How to run.” Include 2–3 concrete metrics you measured (even simple ones): retrieval hit rate on test questions, average response latency, and a small failure analysis count (“3/30 questions needed better chunking”).

• Narrative tip: highlight decisions: why RAG vs fine-tuning, why your chunk size, why your reranker, why you added rate limiting.
• Show constraints: cost limits, time limits, and privacy decisions make your work feel real.

Common mistake: a README that is a wall of setup steps. Put the story first; put setup later.

Your project is only as valuable as your ability to translate it into the language of roles. The same RAG app can support multiple career paths depending on how you frame the skills: product thinking, data pipeline work, QA/evaluation, or platform reliability.

Start by mapping what you built to common job titles:

• AI Product Analyst / Product Ops: requirements → guardrails, failure modes, evaluation plan, user-centric error messages, cost awareness.
• Junior ML/AI Engineer: embeddings + vector DB + retrieval/reranking + prompt orchestration + deployment with secrets.
• Data/Analytics Engineer (AI-adjacent): ingestion pipelines, document cleaning, chunking strategy, metadata design, reproducibility.
• Solutions Engineer: demo quality, deployment, authentication, explaining tradeoffs to stakeholders, clear documentation.

Prepare three interview stories: (1) a technical decision (e.g., how you chose chunking and reranking), (2) a safety/reliability decision (auth, rate limits, PII hygiene, graceful failures), and (3) an iteration story (what failed in evaluation and how you fixed it). Each story should include the constraint, the action, and a measurable result.

Next steps should be concrete and incremental: add multi-tenant metadata filters, support web page ingestion, add a small evaluation harness you can run in CI, or instrument latency and cost per request. These upgrades show you can iterate like a professional.

Common mistake: applying broadly without tailoring. For each role, adjust the top of your README and portfolio page to emphasize the parts they care about most (evaluation for analysts, deployment and reliability for solutions, pipeline and metadata for data roles).