HELP

+40 722 606 166

messenger@eduailast.com

Hands-On AI Risk Assessments for Beginners: Map Impacts First

AI Ethics, Safety & Governance — Beginner

Hands-On AI Risk Assessments for Beginners: Map Impacts First

Hands-On AI Risk Assessments for Beginners: Map Impacts First

Confidently map AI harms, controls, and decisions before you deploy.

Beginner ai-risk-assessment · ai-governance · ai-ethics · model-risk

Map AI impacts before you deploy—without needing technical skills

This course is a short, book-style guide that teaches you how to do an AI risk assessment from the ground up. You don’t need to code, build models, or understand complex math. You’ll learn a practical way to describe an AI system, identify who it affects, spot common harms, and document decisions clearly—before the system reaches real users.

AI risk assessments are not only for large organizations or legal teams. If you’re experimenting with chatbots, automated screening tools, content generators, or “AI-powered” features, you can use the same simple logic: define what the AI does, map how its outputs become real-world outcomes, and put safeguards in place where the risks are highest.

What you will build in this course

By the end, you will have a beginner-friendly toolkit you can reuse:

  • A plain-language description of an AI system (purpose, users, inputs, outputs, and dependencies)
  • A stakeholder and impact pathway map (who could benefit, who could be harmed, and how)
  • A checklist-based risk discovery process covering privacy, fairness, safety, security, and transparency
  • A simple scoring method (likelihood × impact) that avoids false precision
  • A set of controls and an owner/timeline plan for each major risk
  • A one-page AI risk assessment summary you can share with a manager, client, or review group

How the course is structured (like a short technical book)

The course has exactly six chapters that build on each other. First, you learn what “risk” means in everyday terms. Next, you practice describing an AI system so other people can review it. Then you map stakeholders and impact pathways, which helps you see hidden effects. After that, you use checklists to find common AI risks. You then score and treat risks with practical controls. Finally, you produce a one-page assessment and make a clear deployment decision, including what to monitor after launch.

Who this is for

This course is designed for absolute beginners: students, founders, product managers, policy staff, procurement teams, analysts, and anyone asked to “review AI” without being a data scientist. It also works well for small organizations that need a lightweight, repeatable approach before using third-party AI tools.

How to get the most value

Pick one real or realistic AI use case (for example: a customer support chatbot, an AI résumé screener, a classroom tutoring assistant, or an AI content generator). As you move through the chapters, you will fill in the same risk log and template. The goal is not perfect prediction—it’s clear thinking, documentation, and safer decisions.

When you’re ready, you can Register free to start learning, or browse all courses to build a complete governance skill path.

Outcome

You will finish with a practical, shareable AI risk assessment you can adapt to many projects—helping you reduce harm, improve trust, and avoid preventable deployment mistakes.

What You Will Learn

  • Explain what an AI risk assessment is and why it matters before deployment
  • Describe an AI system in plain language (purpose, users, data, outputs, decisions)
  • Identify stakeholders and map who could be helped or harmed
  • List common AI harms (privacy, bias, safety, misinformation, security) using simple checklists
  • Rate risks using a beginner-friendly likelihood × impact method
  • Choose practical controls (human review, data limits, testing, monitoring) matched to risks
  • Write a one-page risk assessment summary you can share with others
  • Decide whether to proceed, pause, or change the AI use case based on evidence

Requirements

  • No prior AI or coding experience required
  • Basic ability to use a web browser and edit a document
  • Willingness to think through real-world scenarios and trade-offs

Chapter 1: What an AI Risk Assessment Really Is

  • Milestone: Understand AI as an input-to-output system (no math required)
  • Milestone: Learn what “risk” means: harm, likelihood, and impact
  • Milestone: Know when to run a risk assessment in a project timeline
  • Milestone: Set your scope: one use case, one decision, one context
  • Milestone: Start a simple risk log you will build throughout the course

Chapter 2: Describe the AI System So Anyone Can Review It

  • Milestone: Write a plain-language system summary (purpose and output)
  • Milestone: Map the user journey from prompt/input to action/decision
  • Milestone: Identify what data is used and where it comes from
  • Milestone: List dependencies (tools, vendors, APIs, human reviewers)
  • Milestone: Define “success” and “failure” for the system

Chapter 3: Map Stakeholders and Impact Pathways

  • Milestone: Identify direct and indirect stakeholders
  • Milestone: Create an impact pathway map (who is affected and how)
  • Milestone: Spot hidden “second-order” effects
  • Milestone: Capture context: environment, power, and constraints
  • Milestone: Convert impacts into clear risk statements

Chapter 4: Find Risks with Beginner-Friendly Checklists

  • Milestone: Run a privacy and data-protection check
  • Milestone: Run a fairness and accessibility check
  • Milestone: Run a safety and misinformation check
  • Milestone: Run a security and misuse check
  • Milestone: Prioritize risks worth addressing first

Chapter 5: Score and Treat Risks (Likelihood × Impact)

  • Milestone: Build a simple scoring scale you can defend
  • Milestone: Score top risks consistently and document reasoning
  • Milestone: Choose controls that reduce likelihood or impact
  • Milestone: Assign owners and timelines for each control
  • Milestone: Define “residual risk” and decide what’s acceptable

Chapter 6: Produce a One-Page Assessment and Deployment Decision

  • Milestone: Write a one-page AI risk assessment summary
  • Milestone: Create a go/no-go decision checklist for stakeholders
  • Milestone: Prepare a basic communication plan (users + internal teams)
  • Milestone: Plan post-launch reviews and incident response basics
  • Milestone: Practice with a realistic case study and finalize your template

Sofia Chen

AI Governance Lead and Risk Assessment Specialist

Sofia Chen designs practical AI governance programs for product teams and public-sector services. She specializes in beginner-friendly risk assessments, documentation, and safeguards that can be applied before launch. Her work focuses on turning ethics goals into clear decisions and measurable controls.

Chapter 1: What an AI Risk Assessment Really Is

An AI risk assessment is a practical way to think ahead: what could go wrong, who could be affected, how bad it could be, how likely it is, and what you can do about it before people rely on the system. It is not a legal memo and not a philosophical debate. It is an engineering and product habit: describe the system in plain language, map impacts first, rate risks in a repeatable way, and choose controls that match the risks.

Beginners often assume risk assessment requires math, specialized jargon, or perfect knowledge of the model. You do not need any of those. You need a clear description of the input-to-output pathway (what goes in, what the system does, what comes out, what decision follows), plus a disciplined way to consider harms such as privacy loss, bias, safety failures, misinformation, and security abuse.

This chapter sets up your workflow for the whole course. You will learn to: treat AI as an input-to-output system (no math required), define “risk” as harm × likelihood × impact, run the assessment early in the project timeline, set scope tightly (one use case, one decision, one context), and start a simple risk log you will build throughout the course.

Keep one idea in mind as you read: the goal is not to eliminate all risk. The goal is to make risk visible early enough that you can reduce it cheaply, explain it honestly, and monitor what remains.

Practice note for Milestone: Understand AI as an input-to-output system (no math required): document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Learn what “risk” means: harm, likelihood, and impact: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Know when to run a risk assessment in a project timeline: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Set your scope: one use case, one decision, one context: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Start a simple risk log you will build throughout the course: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Understand AI as an input-to-output system (no math required): document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Learn what “risk” means: harm, likelihood, and impact: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Know when to run a risk assessment in a project timeline: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Sections in this chapter
Section 1.1: AI in plain words: models, rules, and automation

For risk work, you don’t need to start with algorithms. Start with a plain-language system sketch: inputs → transformation → outputs → decision/action. Inputs might be text prompts, images, customer records, sensor data, or clicks. The transformation might be a model (machine learning), a rules engine, or a hybrid workflow. Outputs might be a score, a label, a summary, a recommendation, or generated content. Finally, there is almost always a decision or action: approve a loan, flag a transaction, route a support ticket, draft an email, or show a piece of content to a user.

Risk hides in the handoffs. A model may only output “suggested priority,” but the organization may treat it as “truth” and auto-route urgent cases away from humans. That’s why your assessment must include the automation boundary: what is automated, what is reviewed, and what is merely suggested.

Use a “no-math” description that anyone on the team can understand. Write three sentences: (1) Purpose: why the system exists. (2) Users: who interacts with it (operators, end users, affected non-users). (3) Data and outputs: what data is used and what is produced. Then add one sentence describing the decision that follows the output.

  • Purpose: “Help call-center agents draft responses faster.”
  • Users: “Agents use it; customers receive the messages; supervisors audit samples.”
  • Data: “Customer chat history and account notes; outputs are suggested reply drafts.”
  • Decision/action: “Agent sends, edits, or discards the draft.”

Common mistake: describing the model but not the system. A risk assessment is about the full socio-technical workflow: data collection, UI design, incentives, time pressure, and how outputs are used. When you map it as a pipeline, you can see where harms can enter: bad input data, ambiguous prompts, brittle edge cases, or over-automation.

Section 1.2: What counts as harm (to people, orgs, and society)

In this course, “harm” means a negative outcome that matters to someone. Start by naming stakeholders, then asking: Who could be helped? Who could be harmed? How? Stakeholders include direct users, people affected by decisions, people whose data is used, internal staff, vendors, regulators, and communities impacted at scale.

Use a simple checklist to avoid missing obvious harms. You are not trying to be exhaustive; you are trying to be systematic enough for beginners to catch the common failure modes.

  • Privacy: exposure of personal data; over-collection; unintended retention; sensitive inference (e.g., health status); leaking training data through outputs.
  • Bias and unfairness: unequal error rates across groups; harmful stereotypes in generated content; exclusion of dialects or accessibility needs; proxy discrimination through correlated features.
  • Safety: physical harm (robotics, medical advice), psychological harm (harassment), or unsafe instructions; escalation failures in crisis contexts.
  • Misinformation and integrity: confident but wrong answers; fabricated citations; misleading summaries; content that looks official; automation that amplifies errors quickly.
  • Security and abuse: prompt injection, data exfiltration, adversarial inputs, model theft, account takeover via social engineering, or the system being repurposed for fraud.

Also consider organizational harms: legal exposure, brand damage, operational disruption, and loss of trust. And societal harms: scaling discrimination, enabling spam, shifting power asymmetrically, or eroding information quality.

Practical technique: for each harm category, write one “impact story” in plain words: “If the system does X, then Y could happen to Z.” Example: “If the assistant drafts a payment-link email using the wrong account number, a customer could send money to the wrong place, causing financial loss and support burden.” These short stories become entries in your risk log.

Section 1.3: The difference between risk, issue, and uncertainty

Teams get stuck when they mix up three different concepts: risk, issue, and uncertainty. Separating them makes your assessment actionable.

Risk is a potential future harm: it has a cause, an outcome, a likelihood, and an impact. Example: “Support agent may accidentally paste sensitive customer data into a public channel because the UI encourages copy/paste from the AI draft.” That is a risk because it might happen, and you can reduce it with controls.

Issue is a harm that is already happening or a defect you have already observed. Example: “In testing, the model included credit card numbers in its summary.” Issues require remediation now (bug fix, rollback, data purge), not just scoring.

Uncertainty is what you do not yet know that affects your confidence in the risk picture. Example: “We don’t know whether the vendor retains prompts for training.” Uncertainties are not “fine”; they are work items. Convert uncertainty into a plan: ask the vendor, run a test, measure, or document an assumption.

This distinction helps you build a beginner-friendly likelihood × impact method. Likelihood is your best judgement about how often the harm could occur in the real setting, not in a perfect lab. Impact is how severe the harm would be if it occurred. Use a simple 1–5 scale for each, then multiply to get a score for prioritization.

  • Likelihood (1–5): rare, unlikely, possible, likely, frequent.
  • Impact (1–5): negligible, minor, moderate, major, severe.

Common mistake: treating the score as truth. The score is a conversation tool to rank what to address first. Document your reasoning in one sentence. If you cannot justify the rating, you probably have an uncertainty to resolve.

Section 1.4: Why “before deployment” changes everything

Running an AI risk assessment before deployment is not bureaucracy; it is a cost-saving move. Before deployment, you can change the product shape: remove a risky feature, narrow the use case, add friction, redesign the human review step, or choose different data. After deployment, you are often stuck with users, integrations, contractual promises, and reputational expectations.

“Before deployment” also changes what evidence you can collect. Early on, you can do controlled testing, simulate edge cases, and run red-team exercises without harming real people. You can instrument logging and monitoring from day one instead of trying to retrofit it later. You can decide up front what good performance means and for whom, rather than discovering late that your metrics ignore a harmed group.

Place the assessment at multiple points in a simple timeline:

  • Idea stage: quick impact map and scope decision. If the risk is inherently high, you may stop or redesign immediately.
  • Data and design stage: confirm data sources, privacy constraints, and human-in-the-loop workflow.
  • Pre-release: test for harms (bias checks, prompt injection tests, hallucination rates, failure-mode reviews), finalize controls, and define monitoring.
  • Post-release: update the risk log with incidents, near-misses, and measured drift; adjust controls.

Common mistake: waiting for “model readiness” before thinking about risk. Many harms come from product choices (auto-send vs. draft, who can access outputs, default sharing settings) and can be prevented long before the model is finalized.

Practical outcome: by the time you deploy, you should already know (1) what risks remain, (2) who accepted them, (3) what controls reduce them, and (4) how you will detect if reality diverges from your assumptions.

Section 1.5: Scoping basics: boundaries, assumptions, and exclusions

Beginners try to assess “the AI” in general and get overwhelmed. Scope is your safety rail. In this course, your default scope is: one use case, one decision, one context. That means one concrete workflow (use case), one specific decision/action that follows the output, and one environment (who uses it, where, under what constraints).

Define boundaries explicitly:

  • In scope: the user group, data sources, output types, decision points, and operational setting (e.g., weekdays, staffed hours, regulated region).
  • Out of scope (for now): future features, other regions, and unrelated use cases. Record them as “later assessments,” not as forgotten items.

Write assumptions so readers know what your ratings depend on. Example assumptions: “Agents must approve drafts before sending,” “No medical advice is allowed,” “The model cannot access raw payment data,” “All logs are retained for 30 days.” Assumptions are powerful because if any assumption is false in production, your likelihood scores may be wrong.

Also write exclusions honestly. Example: “We are not assessing third-party downstream reuse of exported reports.” Exclusions are not excuses; they are prompts to schedule the next assessment or add contractual controls.

Common mistake: scoping only the model and excluding the decision. If the decision is “auto-reject applicants,” the risk profile is radically different than “surface a recommendation for a trained reviewer.” The same output, used differently, creates different harms. Your scope statement should always name the decision that matters.

Section 1.6: Your starter toolkit: checklists, a risk log, and a one-page template

You will use three tools throughout this course: lightweight checklists, a living risk log, and a one-page assessment template. Together, they keep you practical and consistent without slowing the project down.

1) Checklists (to generate risks): Use the harm categories from Section 1.2 as your baseline, then add “operational” prompts: What happens when the model is wrong? When it refuses? When inputs are malicious? When the user is rushed? When the system is down? This is engineering judgement: imagine realistic failure modes, not just worst-case fantasies.

2) The risk log (to track and prioritize): Start a simple table you can maintain in a spreadsheet. Each row is one risk story. Recommended columns:

  • ID (R-001, R-002…)
  • Use case + decision (tight scope label)
  • Risk statement (“If… then… could…”)
  • Stakeholders affected
  • Harm type (privacy, bias, safety, misinformation, security)
  • Likelihood (1–5) and Impact (1–5) + brief rationale
  • Current controls (what already reduces it)
  • Planned controls (what you will add)
  • Owner and due date
  • Status (open, mitigated, accepted, monitoring)

3) The one-page template (to communicate): Put the essentials on one page: system description (inputs/outputs/decision), scope, top stakeholders, top 5 risks with scores, and the controls you will implement before launch. This is what aligns product, engineering, legal, and operations.

Choosing controls should be matched to the risk, not chosen from a generic list. Examples of practical controls you will revisit in later chapters include: human review steps for high-impact decisions, data minimization and retention limits for privacy risk, targeted testing for bias and performance gaps, guardrails against prompt injection, output filtering, rate limits, and monitoring dashboards plus an incident response path.

Common mistake: writing a risk log that never changes. Treat it as a living artifact. When you learn something (a test result, a near-miss, a new user behavior), update likelihood, impact, and controls. That habit—mapping impacts first, then updating with evidence—is what makes an AI risk assessment real.

Chapter milestones
  • Milestone: Understand AI as an input-to-output system (no math required)
  • Milestone: Learn what “risk” means: harm, likelihood, and impact
  • Milestone: Know when to run a risk assessment in a project timeline
  • Milestone: Set your scope: one use case, one decision, one context
  • Milestone: Start a simple risk log you will build throughout the course
Chapter quiz

1. Which description best matches what an AI risk assessment is in this chapter?

Show answer
Correct answer: A practical habit to anticipate what could go wrong, who could be affected, how bad and how likely it is, and what to do before people rely on the system
The chapter frames risk assessment as an engineering/product workflow to think ahead and choose controls—not a legal or philosophical exercise.

2. To describe an AI system for risk assessment, what is the most important structure to capture first?

Show answer
Correct answer: The input-to-output pathway: what goes in, what the system does, what comes out, and what decision follows
Beginners do not need math or perfect model knowledge; they need a clear plain-language input→output→decision description.

3. In this chapter, “risk” is defined using which combination of factors?

Show answer
Correct answer: Harm × likelihood × impact
The chapter defines risk in terms of the harm and how likely and severe it could be.

4. When does the chapter recommend running an AI risk assessment in a project timeline?

Show answer
Correct answer: Early, so risks become visible when they are cheaper to reduce and easier to explain honestly
The goal is to surface and reduce risks before people rely on the system.

5. Which scoping choice best follows the chapter’s guidance for beginners?

Show answer
Correct answer: Pick one use case, one decision, and one context to assess
The chapter emphasizes tight scope so the assessment stays concrete and repeatable.

Chapter 2: Describe the AI System So Anyone Can Review It

A risk assessment can only be as good as the system description behind it. Beginners often jump straight to “Is it biased?” or “Could it leak data?” without first pinning down what the AI actually does, who uses it, and what happens next. Reviewers can’t evaluate risks they can’t see, and teams can’t mitigate risks they haven’t named.

This chapter gives you a practical, plain-language way to describe an AI system so a non-expert reviewer (a manager, legal counsel, security engineer, or an impacted stakeholder) can understand it in minutes. You will write a concise system summary, map the user journey end-to-end, list what data is involved and where it comes from, document dependencies, and define what “success” and “failure” look like. These milestones are the backbone of the rest of the course: once you can describe the system clearly, you can map impacts and choose controls with confidence.

A useful description is not marketing copy and not an architecture diagram. It is a reviewer’s map: purpose, users, inputs, outputs, decisions, data, dependencies, and boundaries. You’ll also notice an important theme: many “AI risks” are actually process risks around the AI—handoffs, escalation, and what humans do with outputs.

Practice note for Milestone: Write a plain-language system summary (purpose and output): document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Map the user journey from prompt/input to action/decision: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Identify what data is used and where it comes from: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: List dependencies (tools, vendors, APIs, human reviewers): document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Define “success” and “failure” for the system: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Write a plain-language system summary (purpose and output): document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Map the user journey from prompt/input to action/decision: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Identify what data is used and where it comes from: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: List dependencies (tools, vendors, APIs, human reviewers): document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Sections in this chapter
Section 2.1: Purpose, users, and the job the AI is doing

Start with the milestone that unlocks everything else: a plain-language system summary. Your goal is to make the system reviewable by someone who has never seen it. Keep it short, specific, and operational—avoid vague claims like “improves productivity” or “optimizes decisions.”

Write two sentences:

  • Purpose: What problem is the AI meant to solve, in the real world (not in model terms)?
  • Output: What does it produce that a user can see or act on (text, score, ranking, alert, recommendation, image)?

Then name the users and the job the AI is doing for them. “Users” is broader than “people typing prompts.” Include (1) direct users (operators), (2) indirect users (people who receive the output), and (3) affected parties (people the decision is about). A customer support summarizer has different stakeholders than a loan pre-screening model, even if both output text.

Engineering judgment: be explicit about whether the AI is advisory (suggests) or determinative (decides). Beginners often describe an AI as “just a tool,” but if the organization treats the output as authoritative, then functionally it is part of a decision system. Capture that reality in your purpose statement.

Common mistake: writing the description from the builder’s viewpoint (“uses a transformer model with RAG”) instead of the reviewer’s viewpoint (“answers employee policy questions using HR documents; provides citations”). Technical details can be added later, but you need a clear job-to-be-done first.

Section 2.2: Inputs and outputs: prompts, files, scores, and recommendations

Next, map the user journey from prompt/input to output. List every input channel the system accepts, not just the main one. Inputs can include typed prompts, uploaded files, pasted text, form fields, clickstream/context signals, microphone audio, images, or API calls from other systems.

For each input, answer three reviewer-friendly questions:

  • Who provides it? (employee, customer, vendor system, automated scraper)
  • What format is it? (free text, PDF, CSV, database row, image)
  • What might be inside? (personal data, confidential business data, third-party content)

Then describe outputs just as concretely. Outputs are not always “a response.” They may be a risk score, a ranking of candidates, a label (spam/not spam), a summary with citations, a recommended next action, or a generated artifact (email draft, code, medical note). Note where the output appears: in a dashboard, in a ticketing system, in a customer chat, or sent via webhook to another application.

Practical tip: include examples with placeholders, not real data. For instance: “Input: ‘Customer reports late delivery, order #12345’ → Output: ‘Suggested reply and refund policy excerpt.’” This makes the system legible without exposing sensitive content.

Common mistakes include forgetting “hidden inputs” (system prompts, retrieval queries, default context like user profile fields) and ignoring “secondary outputs” (logs, analytics events, stored embeddings). Reviewers care about these because they can become privacy, security, or IP risks even if users never see them.

Section 2.3: Decisions and actions: when output becomes real-world impact

Risk is created when an output becomes an action. A model can generate an incorrect answer, but harm happens when someone acts on it—denies a service, shares misinformation, triggers an investigation, or discloses private data. This milestone is about identifying the decision points.

Take your user journey and add a simple “so what?” layer:

  • Decision: What choice is made using the output? (approve/deny, prioritize, message, investigate, dispatch, escalate)
  • Decision owner: Who is accountable for the choice? (agent, manager, automated workflow)
  • Consequence: What changes in the real world? (money, access, reputation, time, safety)

Be explicit about automation level. Many systems start “human-in-the-loop” but drift into “human-on-the-loop,” where people rubber-stamp suggestions to keep up with volume. If the AI output is copied into customer communications, that is effectively a publication pathway. If the AI writes code that is deployed, that is effectively an engineering control pathway. Your description should reflect how the system is actually used, not how you wish it were used.

Define “success” and “failure” in terms of outcomes. Success is not “high accuracy” in the abstract; it is “reduces average handle time without increasing complaint rate,” or “identifies truly high-risk transactions with acceptable false positives.” Failure should include harmful modes: “hallucinates policy,” “recommends unsafe medical action,” “incorrectly flags legitimate users,” “reveals private details,” or “creates misleading confidence.”

Common mistake: describing decisions as if they happen only at the final step. In reality, impacts can occur earlier—during triage, prioritization, or what the system chooses to show or hide. Capture each decision point, even if it feels minor.

Section 2.4: Data basics: personal data, sensitive data, and provenance

Now identify what data is used and where it comes from. For risk assessment, “data” includes anything the system consumes, retrieves, stores, or generates—training data (if you trained), prompts, uploaded documents, retrieved knowledge base content, conversation history, labels, and logs.

Start with a data inventory table in plain language (you can keep it in a doc): dataset/source, what it contains, who it’s about, who owns it, where it is stored, retention period, and who can access it. Then mark whether the data may include:

  • Personal data: names, emails, IDs, device identifiers, voice recordings, location, IP addresses
  • Sensitive data: health, financial info, government IDs, precise location, minors’ data, biometrics, union membership, or other special-category traits (by jurisdiction)
  • Confidential business data: trade secrets, strategy docs, source code, unreleased product plans

Provenance matters: describe where the data originated and how it was collected. Was it provided by the user knowingly? Imported from another system under a contract? Scraped from the web? Generated by another model? Each source carries different consent, quality, and IP implications.

Engineering judgment: distinguish between data used at runtime (e.g., retrieval-augmented generation pulling internal documents) and data used for improvement (e.g., storing prompts to fine-tune or evaluate). These are different risk surfaces. If prompts are stored, say so. If they are not, say how you ensure that (configuration, logging controls, vendor settings).

Common mistakes include assuming “we don’t store data” because the product UI doesn’t show storage, or overlooking that vendors may retain data by default. Another frequent gap is ignoring derived data such as embeddings, summaries, or labels—these can still encode personal or sensitive information.

Section 2.5: Human roles: oversight, escalation, and accountability

AI systems are socio-technical: people are always part of the system, even when the model runs automatically. This milestone documents the human roles and the accountability chain so reviewers can see who can intervene when something goes wrong.

List the humans involved across the lifecycle:

  • Operators: who uses the tool day-to-day
  • Reviewers/approvers: who signs off on outputs or decisions (if anyone)
  • Escalation owners: who handles edge cases, complaints, or suspected harm
  • System owners: product owner, engineering owner, data owner, security owner

Then define the oversight mechanism in behaviorally specific terms. “Human review” is not a control unless you specify: what is reviewed, when, by whom, with what checklist, and what authority they have to stop or override the AI. A practical description might be: “If the model’s confidence is below X or the content contains restricted topics, the draft cannot be sent until a supervisor approves.”

Include failure handling: how a user reports an issue, expected response time, and what triggers an escalation (e.g., suspected privacy breach, discrimination complaint, safety risk). Reviewers will look for a clear path from detection to correction.

Common mistake: treating accountability as “the user’s responsibility.” If your workflow encourages fast copying/pasting of outputs, you need explicit guardrails and training. Also note any human dependencies outside your team, such as contracted moderators, clinical reviewers, or customer support quality teams—these affect reliability and fairness.

Section 2.6: System boundaries: where the AI stops and the process continues

Finally, draw the system boundary. Boundaries prevent two classic assessment failures: (1) ignoring key dependencies that introduce risk, and (2) trying to assess “AI” as if it exists alone, without the surrounding process.

Document dependencies (tools, vendors, APIs, and humans). Examples include: the foundation model provider, embedding/retrieval service, vector database, content filter, identity provider, ticketing system, cloud storage, analytics platform, and any third-party data feeds. For each dependency, capture what it does, what data it receives, and what happens if it fails.

Then define “where the AI stops.” Does the AI only generate a suggestion, after which a separate system sends the message? Does an orchestration layer transform outputs into API calls? Does a human take the output and perform an offline action? Your boundary statement should clarify what is in scope for your assessment now and what is a downstream process that still needs risk consideration.

Also specify environment and access boundaries: internal-only vs public-facing, authenticated vs anonymous, and what user permissions exist (who can upload files, who can view logs, who can export outputs). Seemingly small boundary details often dominate risk—public exposure changes abuse likelihood; broad internal access increases privacy and IP leakage risk.

Practical outcome: at the end of this chapter you should have a one-page “review packet” consisting of (1) purpose/output summary, (2) user journey map from input to action, (3) data inventory with provenance, (4) dependency list, and (5) success/failure definitions. That packet becomes the input to stakeholder mapping and harm checklists in the next chapters.

Chapter milestones
  • Milestone: Write a plain-language system summary (purpose and output)
  • Milestone: Map the user journey from prompt/input to action/decision
  • Milestone: Identify what data is used and where it comes from
  • Milestone: List dependencies (tools, vendors, APIs, human reviewers)
  • Milestone: Define “success” and “failure” for the system
Chapter quiz

1. Why does the chapter say you should describe the AI system before evaluating risks like bias or data leakage?

Show answer
Correct answer: Because reviewers can’t evaluate risks they can’t see, and teams can’t mitigate risks they haven’t named
The chapter emphasizes that risk assessment quality depends on a clear system description that makes the system visible to reviewers.

2. Which best matches the chapter’s definition of a “useful” system description?

Show answer
Correct answer: A reviewer’s map covering purpose, users, inputs, outputs, decisions, data, dependencies, and boundaries
The chapter contrasts a useful description with marketing copy and architecture diagrams, calling it a reviewer’s map.

3. What is the goal of writing the plain-language system summary milestone?

Show answer
Correct answer: Explain the system’s purpose and output so a non-expert can understand it in minutes
The chapter focuses on quick comprehension by non-expert reviewers through plain-language purpose and output.

4. What does mapping the user journey help you capture that affects risk?

Show answer
Correct answer: The end-to-end path from prompt/input to action/decision, including what happens next
The chapter highlights that risks often appear in what happens after the AI output—handoffs, escalation, and downstream actions.

5. According to the chapter, many “AI risks” are actually what kind of risks?

Show answer
Correct answer: Process risks around the AI, such as handoffs, escalation, and what humans do with outputs
A key theme is that the surrounding process and human use of outputs often drives the real risk.

Chapter 3: Map Stakeholders and Impact Pathways

In Chapter 2 you learned to describe an AI system in plain language: what it does, who uses it, what data it touches, and what outputs it produces. In this chapter you turn that description into an “impact map” that shows who could be helped or harmed and through what pathway. This step is the heart of a beginner-friendly AI risk assessment: before you can rate risk, you must know where risk can land.

Many teams jump straight to checklists (bias, privacy, security) without first identifying the full set of stakeholders. That creates blind spots. For example, an AI scheduling tool may have “users” (managers) and “subjects” (workers), but also “bystanders” (customers affected by service quality) and “operators” (admins maintaining access). Each group can experience different impacts, and the same output can become a benefit for one group and a harm for another.

This chapter follows a practical workflow: (1) list direct and indirect stakeholders, (2) map impact pathways from model output to real-world outcomes, (3) look for second-order effects that emerge over time, (4) capture context—environment, power, and constraints—so you understand who is vulnerable and why, and (5) convert what you learned into clear risk statements you can rate and control in later chapters.

Your deliverable is a one-page artifact: a stakeholder list plus an impact pathway map. It does not need to be perfect. It needs to be legible, testable, and connected to concrete decisions your system influences.

Practice note for Milestone: Identify direct and indirect stakeholders: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Create an impact pathway map (who is affected and how): document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Spot hidden “second-order” effects: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Capture context: environment, power, and constraints: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Convert impacts into clear risk statements: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Identify direct and indirect stakeholders: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Create an impact pathway map (who is affected and how): document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Spot hidden “second-order” effects: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Capture context: environment, power, and constraints: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Sections in this chapter
Section 3.1: Stakeholder types: users, subjects, bystanders, operators

Start by listing stakeholders in four simple buckets. This milestone—identifying direct and indirect stakeholders—prevents the most common beginner mistake: treating “the user” as the only person who matters.

Users are people who directly interact with the AI output to do a task. Examples: a recruiter viewing candidate rankings, a teacher using an AI tutor dashboard, a customer asking a chatbot for support. Users experience impacts like productivity gains, over-trust, confusion, or increased workload from false alarms.

Subjects are people the AI is about. They may never touch the system. Examples: job applicants whose resumes are scored, patients whose notes are summarized, drivers whose images are captured for fraud detection. Subjects can experience privacy intrusion, unfair treatment, or denial of opportunities.

Bystanders are people indirectly affected by decisions made with AI. Examples: family members impacted by a lending decision, coworkers who inherit work when someone is mis-scheduled, community members affected by misinformation amplified by AI-generated posts. Bystanders are where hidden “second-order” effects often live.

Operators are people who deploy, monitor, and maintain the system: IT admins, customer support, model owners, compliance staff. They are exposed to operational harms (alert fatigue, unclear accountability, security incidents) and are crucial to implementing controls.

Practical step: create a table with columns Stakeholder, Role, What they want, How they might be harmed, How they might benefit. Do not stop at internal roles. Include vendors, contractors, and downstream partners who may reuse outputs.

Section 3.2: Impact pathways: from output to outcome to harm/benefit

Once you know who is involved, map the pathway from the AI’s output to real-world impact. The milestone here is creating an impact pathway map—who is affected and how. A useful pattern is: Output → Decision/Action → Outcome → Harm/Benefit.

Example: an AI tool outputs a “fraud risk score.” That output becomes an action (freeze account, require additional verification). The action causes an outcome (customer cannot pay rent on time). The outcome creates a harm (financial stress, fees, reputational damage) or a benefit (reduced fraud losses, safer marketplace).

Write pathways as short chains. Keep them concrete and observable. Avoid vague nodes like “bad experience.” Instead use measurable outcomes like “delayed service,” “denied eligibility,” “inaccurate public statement,” or “unauthorized data exposure.”

Engineering judgment matters here: identify where humans interpret outputs. Many harms are mediated by how people use the output, not the output itself. If your model provides a summary, the pathway includes whether a user verifies it, whether the summary is copied into a record, and whether it triggers a decision. Mark these handoffs explicitly.

  • Tip: Draw two to five pathways per major output. Too many pathways becomes noise; too few misses key impacts.
  • Common mistake: Mapping only the “happy path.” Always include error modes: false positives, false negatives, and “unknown/uncertain” outputs.

Your map should make it easy to point at a box and say, “If this step goes wrong, this stakeholder experiences this outcome.” That traceability is what turns ethics concerns into actionable risk management.

Section 3.3: Vulnerable groups and why context matters

Stakeholders are not affected equally. The milestone here is spotting hidden second-order effects and capturing context: environment, power, and constraints. “Vulnerable groups” are not defined by a label alone; vulnerability often comes from context—limited resources, reduced ability to recover from errors, or higher exposure to monitoring or enforcement.

Context questions to ask: Where is the system used (hospital, school, border crossing, workplace)? What happens if the model is wrong (minor inconvenience vs. life-changing denial)? What constraints do people face (language barriers, disability access, limited internet, fear of retaliation)?

Second-order effects often emerge when an AI system changes incentives over time. A call-center assistant that tracks “agent quality” may initially help coaching, but later be used to justify layoffs, leading agents to avoid complex cases. A content-moderation model may reduce visible toxicity but also push harmful communities to private channels where reporting is harder.

Practical step: add a “context layer” to each pathway. For each stakeholder, note: (1) what resources they have to respond, (2) what they risk by complaining, and (3) how frequently they are exposed. Then identify groups who are disproportionately represented among subjects (e.g., certain neighborhoods in surveillance footage) or who face higher stakes (e.g., patients with rare conditions where errors are more likely).

Common mistake: treating fairness as only “demographic parity” without considering harm severity. Two groups can have similar error rates, but one group can be harmed more because they have fewer alternatives or face compounding disadvantages.

Section 3.4: Power dynamics: who can contest or opt out

Power dynamics determine whether an impact becomes a durable harm. This section connects to capturing context and constraints, and it directly influences which controls will work later (human review, appeals, monitoring). Ask: who can contest decisions, who can opt out, and who is locked in?

Map the “contestability path” next to your impact pathway. If the AI output leads to a decision, can a subject see that a model was involved? Can they request an explanation in plain language? Is there a human who can override the output, and do they have time and authority to do so?

Power is also about information. Users often get dashboards and confidence scores; subjects may get a one-line denial. That gap increases harm because people cannot correct errors. Similarly, operators may have logs but no clear policy for when to roll back a model, turning known issues into prolonged exposure.

Practical checklist:

  • Notice: Are people informed AI is used, and what it affects?
  • Choice: Is there an opt-out or alternative channel?
  • Appeal: Is there a review process with timelines?
  • Override: Can a human reverse the decision without punishment?
  • Record: Are decisions and evidence retained for auditing?

Common mistake: assuming “human in the loop” automatically fixes risk. If humans are incentivized to follow the model (speed targets, liability fears), the loop becomes rubber-stamping. Your map should reflect real authority and real constraints, not idealized policy.

Section 3.5: Benefit mapping: positive outcomes and who receives them

Risk assessment is not only about harms. Mapping benefits clarifies trade-offs and prevents a narrow focus on compliance. This milestone—benefit mapping—asks you to specify positive outcomes and who receives them, using the same pathway discipline: Output → Action → Outcome → Benefit.

Benefits should be explicit and distributed. If the benefit is “efficiency,” identify whose time is saved. If the benefit is “accuracy,” identify which decisions improve and whether improvements reach subjects or only the organization.

Example: an AI triage assistant outputs a prioritized queue. Benefits may include faster response for urgent cases (subjects), reduced cognitive load for clinicians (users), and lower operating costs (organization). But benefit mapping should also reveal benefit concentration: perhaps clinicians save time while patients experience more automation and less personal interaction.

Practical step: for each stakeholder group, write at least one intended benefit and one potential unintended benefit (e.g., a tool designed for support might also generate training data that improves future models). Then note what evidence would demonstrate that benefit: reduced wait time, fewer escalations, higher task completion, fewer incidents.

Common mistake: treating benefit as a justification to ignore harm (“net positive”). In governance work, benefits help you choose proportionate controls. If benefits are large and widely shared, investing in strong safeguards is easier to justify; if benefits are narrow and harms fall on others, you may need redesign, not just mitigations.

Section 3.6: Writing risk statements: “If X happens, then Y harm may occur”

Your final milestone is converting impacts into clear risk statements. This is the bridge from mapping to rating. A good risk statement is specific, testable, and connected to a pathway. Use the pattern: If X happens, then Y harm may occur to Z stakeholder, because …

Start with “X” as a failure mode or misuse, not a general fear. Examples of X: “the model produces a false positive fraud score,” “the chatbot fabricates a policy,” “training data includes sensitive attributes,” “the model is prompt-injected to reveal private information,” “users rely on the score without verification.”

Then define “Y harm” in real-world terms: denial of service, reputational damage, unequal access, unsafe medical advice, privacy breach, or financial loss. Include “Z stakeholder” so ownership is clear. The “because” clause should point to the pathway step where the harm is realized (e.g., automatic enforcement, lack of appeal, high-stakes setting).

Practical examples:

  • If the resume screening model under-scores candidates with non-standard job histories, then qualified applicants may be unfairly rejected, because recruiters use the ranking as a cutoff and applicants cannot appeal.
  • If the support chatbot provides incorrect refund instructions, then customers may miss deadlines and lose money, because the chat transcript is treated as authoritative and policy language is complex.
  • If operator dashboards expose raw user prompts to broad internal roles, then sensitive personal data may be accessed without need, because access controls and logging are incomplete.

Common mistake: writing risks as vague nouns (“bias,” “privacy”). Those labels belong in categories later. In this chapter your goal is crisp statements that you can later rate for likelihood and impact and match to controls like human review, data minimization, targeted testing, and monitoring.

Chapter milestones
  • Milestone: Identify direct and indirect stakeholders
  • Milestone: Create an impact pathway map (who is affected and how)
  • Milestone: Spot hidden “second-order” effects
  • Milestone: Capture context: environment, power, and constraints
  • Milestone: Convert impacts into clear risk statements
Chapter quiz

1. Why does Chapter 3 emphasize mapping stakeholders and impact pathways before using common AI risk checklists (e.g., bias, privacy, security)?

Show answer
Correct answer: Because checklists can miss who risk can land on if stakeholders and pathways aren’t identified first
The chapter argues that jumping to checklists creates blind spots unless you first map who is affected and how.

2. In the chapter’s example of an AI scheduling tool, which pairing best reflects different stakeholder roles beyond just “users” and “subjects”?

Show answer
Correct answer: Bystanders (customers affected by service quality) and operators (admins maintaining access)
The chapter explicitly names customers as bystanders and admins as operators in the scheduling example.

3. What is an “impact pathway map” in Chapter 3?

Show answer
Correct answer: A map that traces how model outputs lead to real-world outcomes for specific stakeholders
Impact mapping connects outputs to outcomes and shows who could be helped or harmed through those pathways.

4. What is the purpose of looking for “second-order effects” when mapping impacts?

Show answer
Correct answer: To identify indirect impacts that emerge over time beyond the immediate, direct outcome
Second-order effects are hidden, longer-term or indirect consequences that can be missed if you only consider immediate impacts.

5. Which sequence best matches the chapter’s practical workflow for creating the Chapter 3 deliverable?

Show answer
Correct answer: List direct and indirect stakeholders → map impact pathways → look for second-order effects → capture context (environment, power, constraints) → convert impacts into clear risk statements
The chapter provides this five-step workflow in order and uses it to produce a legible, testable one-page artifact.

Chapter 4: Find Risks with Beginner-Friendly Checklists

In Chapter 3 you mapped impacts by describing the system, its stakeholders, and who could be helped or harmed. Now you turn that map into a risk assessment you can act on. A beginner-friendly risk assessment is not a legal memo or a spreadsheet marathon. It is a structured way to (1) spot common harms using checklists, (2) capture them as clear risk statements, and (3) prioritize what to fix first using a simple likelihood × impact rating.

This chapter is built around “milestones” you can run in under an hour for a small prototype, or in a day for a more complex system. Each milestone is a focused pass: privacy/data protection, fairness/accessibility, safety/misinformation, security/misuse, and finally prioritization. You will notice that the same issue can show up in multiple milestones (for example, personal data leakage is both a privacy issue and a security issue). That is normal; your goal is not to perfectly categorize risks, but to ensure you find them and assign practical controls.

A workable workflow looks like this: start with your plain-language system description (purpose, users, data, outputs, decisions). Then, for each checklist, write risks in the same format: “If , then , affecting , because .” After you list risks, rate each one on likelihood and impact (e.g., 1–5 each) and multiply them for a priority score. Finally, select controls matched to the risk’s cause—human review for judgement calls, data limits for privacy exposure, testing for reliability, monitoring for drift and abuse.

Common mistakes at this stage are (a) writing vague risks (“bias is possible”), (b) listing controls without verifying they are feasible (“we will perfectly filter everything”), and (c) skipping the “who is harmed” part, which leads to missing accessibility and downstream misuse. Use the sections below as a practical set of checklists you can reuse on every project.

Practice note for Milestone: Run a privacy and data-protection check: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Run a fairness and accessibility check: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Run a safety and misinformation check: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Run a security and misuse check: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Prioritize risks worth addressing first: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Run a privacy and data-protection check: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Run a fairness and accessibility check: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Run a safety and misinformation check: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Sections in this chapter
Section 4.1: Privacy risks: collection, retention, leakage, and consent

Milestone: Run a privacy and data-protection check. Privacy risks show up wherever your system touches data: what you collect, how long you keep it, where it flows, and who can see it. Start by listing each data type the system uses (user inputs, uploaded files, logs, model outputs, analytics events, support tickets). Then ask four questions: collection, retention, leakage, and consent.

  • Collection: Are you collecting personal data you do not need? Are you collecting sensitive data (health, biometrics, precise location, children’s data) unintentionally via free-text inputs?
  • Retention: How long are prompts, files, and outputs stored? Are there separate retention policies for logs vs. primary records? Can users delete their data?
  • Leakage: Could outputs reveal private information from the prompt, from training data, or from other users? Could internal tools expose raw prompts to staff who do not need them?
  • Consent and notice: Do users understand what is collected and why? Is there a clear opt-in/opt-out for data used to improve models? Are there special rules for minors or regulated contexts?

Turn findings into concrete risks. Example: “If customer support agents paste full customer records into the chatbot, then personal data may be stored in vendor logs, affecting customers, because the tool retains prompts by default.” Controls should match the cause: input guidance (“do not paste IDs”), data minimization (masking, redaction), retention limits, and access controls. Another practical control is purpose limitation: only use data for the feature the user expects, and keep it no longer than needed.

Engineering judgement matters in tradeoffs. Debug logs are valuable, but raw prompts are often too sensitive to keep. A common beginner error is assuming “we don’t store data” because the application doesn’t save it—while the model provider, monitoring stack, or error reporting tool might. Trace the full pipeline, including third parties, and write down defaults you must change.

Section 4.2: Bias and unfairness: where it comes from and how it shows up

Milestone: Run a fairness and accessibility check. Fairness risks are easiest to find when you connect them to user groups and decision points. Ask: does the system recommend, rank, screen, approve, deny, or prioritize anything that affects people’s opportunities? Even “assistive” tools can create unfair outcomes if humans over-trust them or if they work better for some groups than others.

Bias can come from multiple sources: training data that under-represents a group, labels that reflect historical discrimination, proxies (zip code acting like race), or interface design that excludes users (language, reading level, disability access). For generative AI, unfairness often appears as stereotyped language, differential refusal behavior, or poorer performance for dialects and non-standard phrasing.

  • Representation check: Who are typical vs. edge-case users? Are there groups with different language, accessibility needs, or contexts of use?
  • Outcome check: What does “good” look like for each group? Could the system’s errors systematically burden one group more?
  • Accessibility check: Can users with screen readers, low vision, or limited motor control use the UI? Are captions, contrast, and keyboard navigation supported?
  • Human workflow check: Are reviewers trained to spot biased outputs? Are they incentivized to rubber-stamp model suggestions?

Write risks as observable behaviors. Example: “If the résumé screener ranks candidates based on wording similarity, then non-native speakers may be ranked lower, affecting candidates, because the embedding model penalizes atypical phrasing.” Controls might include balanced evaluation sets, constraints (no protected-class inference), calibration tests by subgroup where appropriate, and user experience changes (collect structured skills rather than free-form text). A common mistake is treating fairness as a single metric. For beginners, focus on practical checks: compare error rates and user complaints across groups, and ensure accessibility is not an afterthought.

Section 4.3: Reliability: errors, hallucinations, and brittle behavior

Reliability risks are about whether the system behaves consistently and correctly in real conditions. With AI, failures are often non-obvious: hallucinated facts, confident wrong answers, brittle behavior on slightly different inputs, or performance collapse when the data distribution shifts. Reliability matters even when the tool is “only advisory,” because users may still act on it.

Start with your system’s required level of correctness. A writing assistant can tolerate stylistic variance; a medication advice tool cannot. Then run a simple reliability checklist: (1) define what the model must never do, (2) define what it must always do, and (3) test representative and adversarial inputs.

  • Error modes: Does it invent citations, prices, policies, or medical/legal claims? Does it misread units, dates, or names?
  • Brittleness: Does output change dramatically with minor prompt edits? Does it fail on long inputs, unusual formatting, or mixed languages?
  • Boundary conditions: What happens when data is missing, ambiguous, or contradictory? Does it ask clarifying questions or guess?
  • Monitoring: Do you track quality regressions after model/provider updates and prompt changes?

Convert reliability issues into risks tied to decisions. Example: “If the assistant fabricates return-policy details, then customers may be misled into missing deadlines, affecting customers and support teams, because the model is not grounded in the latest policy text.” Controls include retrieval grounding (use approved sources), forced citations to known documents, refusal when confidence is low, and human review for high-stakes outputs. A frequent beginner mistake is testing only “happy path” prompts. Your checklist should include messy real user input and the top 10 failure cases from support tickets or stakeholder interviews.

Section 4.4: Safety: physical, financial, and emotional harms

Milestone: Run a safety and misinformation check. Safety risks are the harms that occur when the system’s outputs or actions lead to injury, loss, or distress. For beginners, it helps to separate safety into physical, financial, and emotional domains, and to explicitly include misinformation as a safety accelerator—false claims can spread quickly and cause real-world harm.

  • Physical: Could advice lead to injury (workplace instructions, medical suggestions, DIY repairs)? Could the system control devices or trigger actions?
  • Financial: Could it recommend harmful transactions, misstate fees, or enable scams? Could errors cause chargebacks or penalties?
  • Emotional/social: Could it produce harassment, self-harm encouragement, or manipulative content? Could it shame or discriminate?
  • Misinformation: Could it generate plausible but false news, fake endorsements, or altered images that mislead users?

Write risks with the user journey in mind: where does the model speak with authority, and where might users stop verifying? Example: “If the tool summarizes medical symptoms without a clear disclaimer and routes, then users may delay seeking care, affecting patients, because the output sounds definitive.” Controls include clear scope limits, safety policies with refusal behavior, escalation to humans (or to emergency resources) for crisis topics, and UI friction for high-risk actions (confirmations, warnings, ‘not a substitute’ disclosures).

Engineering judgement is crucial: over-blocking can harm users who need legitimate information; under-blocking can enable dangerous guidance. A practical approach is to define “high-stakes topics” for your domain and require stronger controls there: grounding, citations, human review, and audit logs. Measure what matters: incident rates, near-misses, and user reports—not just offline accuracy.

Section 4.5: Security and abuse: prompt injection, data exfiltration, fraud

Milestone: Run a security and misuse check. Security risks cover both traditional threats (unauthorized access) and AI-specific abuse patterns (prompt injection, indirect prompt injection via documents, and data exfiltration). The key idea for beginners: treat the model as an untrusted component that can be manipulated, and design the surrounding system to keep secrets and enforce permissions.

  • Prompt injection: Can a user instruct the model to ignore rules, reveal system prompts, or take disallowed actions?
  • Indirect injection: If the model reads emails, web pages, or PDFs, could hidden text trick it into leaking data or changing behavior?
  • Data exfiltration: Can the model be coerced into outputting sensitive data from tools, memory, logs, or connected databases?
  • Fraud and abuse: Could attackers use the system to generate phishing, impersonation, or fake documents at scale?

Translate into risks tied to assets. Example: “If the assistant can call internal APIs based only on natural-language instructions, then an attacker can retrieve another user’s account details, affecting customers, because the tool layer lacks authorization checks.” Controls are usually outside the model: strong authZ at the API layer, allowlists for tools, least-privilege tokens, output filtering for secrets, rate limits, and abuse monitoring. For retrieval systems, sanitize untrusted content and separate instructions from data (e.g., treat retrieved text as quoted context, not executable commands).

A common mistake is relying on “the model won’t do that” as a security control. Assume it will, eventually. Your checklist outcome should be a short list of hard requirements (permission checks, logging, incident response contacts) and a plan to test them with red-team prompts.

Section 4.6: Transparency: explainability, disclosures, and user expectations

Transparency is how you keep user expectations aligned with what the system actually does. It includes explainability (what factors drove an output), disclosures (AI involvement, limitations, data use), and operational clarity (how to contest decisions and report issues). Transparency reduces harm by preventing over-reliance and by enabling accountability when things go wrong.

Run a practical transparency checklist: What should users know before using the tool, what should they see at the moment they receive an output, and what can they access after an outcome affects them?

  • Disclosures: Is it clear when content is AI-generated? Are limitations stated in plain language (freshness, potential errors, scope boundaries)?
  • Explainability: For decisions (approvals, rankings), can you provide reason codes or the main contributing factors? Can a user correct wrong inputs?
  • Recourse: Can users appeal, request human review, or opt out? Is there a clear support path?
  • Data expectations: Do users understand what is stored, for how long, and whether it is used to improve models?

Milestone: Prioritize risks worth addressing first. Transparency also helps you prioritize: if a risk can’t be fully eliminated quickly, you may still reduce impact by warning users, gating features, or requiring confirmation. Use your likelihood × impact scores to pick the top items, then choose controls that are feasible before deployment. A helpful rule: address high-impact risks even if likelihood feels “low,” because AI systems can scale failures fast. Capture outcomes in a simple register: risk statement, score, owner, control, and a date for re-review. That document is your bridge from “we noticed risks” to “we managed them.”

Chapter milestones
  • Milestone: Run a privacy and data-protection check
  • Milestone: Run a fairness and accessibility check
  • Milestone: Run a safety and misinformation check
  • Milestone: Run a security and misuse check
  • Milestone: Prioritize risks worth addressing first
Chapter quiz

1. What is the main purpose of the beginner-friendly risk assessment approach in Chapter 4?

Show answer
Correct answer: Turn the impact map into actionable risks by spotting common harms, writing clear risk statements, and prioritizing fixes
The chapter emphasizes an actionable, structured workflow: checklists → clear risk statements → likelihood × impact prioritization.

2. Which workflow best matches the chapter’s recommended process for running the milestones?

Show answer
Correct answer: Start with a plain-language system description, run each checklist, write risks in a standard format, then rate likelihood and impact to prioritize controls
The chapter’s sequence starts with system description, then checklists and structured risk statements, then scoring and control selection.

3. Why does the chapter say it’s normal for the same issue (e.g., personal data leakage) to appear in multiple milestones?

Show answer
Correct answer: Because the goal is to find and control risks, not to perfectly categorize them
Overlaps are expected; what matters is capturing the risk and assigning practical controls, not perfect classification.

4. Which risk statement best follows the chapter’s recommended format?

Show answer
Correct answer: If the model is trained on unrepresentative data, then it may underperform for some user groups, affecting those users, because the training set does not reflect their language patterns
The chapter recommends: “If , then , affecting , because .” The other options are vague or are controls, not risks.

5. How does Chapter 4 recommend prioritizing which risks to address first?

Show answer
Correct answer: Rate each risk on likelihood and impact (e.g., 1–5) and multiply them to get a priority score
The chapter uses a simple likelihood × impact rating to produce a priority score for deciding what to fix first.

Chapter 5: Score and Treat Risks (Likelihood × Impact)

By now you have a list of plausible harms and a map of who might be affected. This chapter turns that list into decisions. You will (1) score risks using a simple likelihood × impact method, (2) choose controls that reduce either likelihood or impact, (3) assign owners and timelines so the plan is real, and (4) decide what “residual risk” remains after controls—and whether it is acceptable.

A beginner mistake is to treat risk scoring like math homework: pick numbers, multiply, and declare victory. In practice, the scoring scale is a communication tool. Your goal is not precision; it is consistency and defendability. If your team can explain why one risk is “High” and another is “Medium,” and can show the evidence used, you are already doing professional-grade governance.

Another common mistake is to score everything as high. That feels cautious, but it prevents prioritization and makes treatment plans impossible to execute. A good assessment forces trade-offs: which few risks do we treat immediately, which do we monitor, and which do we accept with documentation and sign-off?

  • Likelihood: How probable is the harm in the real world (given your users, data, and operating context)?
  • Impact: If it happens, how severe is the harm (to people, the organization, and the broader ecosystem)?
  • Risk score: A simple combination (often a 3×3 matrix) that helps rank items and choose controls.
  • Residual risk: The risk that remains after controls are implemented and verified.

Use the milestones in this chapter as your workflow: build a simple scale you can defend, score the top risks consistently with documented reasoning, choose controls matched to each risk, assign owners and timelines, then decide what residual risk is acceptable for launch.

Practice note for Milestone: Build a simple scoring scale you can defend: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Score top risks consistently and document reasoning: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Choose controls that reduce likelihood or impact: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Assign owners and timelines for each control: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Define “residual risk” and decide what’s acceptable: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Build a simple scoring scale you can defend: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Score top risks consistently and document reasoning: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Choose controls that reduce likelihood or impact: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Sections in this chapter
Section 5.1: Simple scales: low/medium/high without false precision

Section 5.1: Simple scales: low/medium/high without false precision

Start with a 3-level scale for both likelihood and impact: Low / Medium / High. Avoid 1–10 scales early on; they encourage fake precision and arguments about whether a risk is a “6” or a “7.” Your scoring system should be simple enough that two people can apply it independently and end up close.

Define the scale in words, not numbers. For example:

  • Likelihood—Low: rare edge cases; requires unusual inputs or misuse; limited exposure.
  • Likelihood—Medium: plausible in normal use; could occur monthly/weekly depending on volume.
  • Likelihood—High: expected in normal use; you have seen it in testing or similar systems; could occur frequently.
  • Impact—Low: minor inconvenience; reversible; limited to a small number of users; no sensitive data.
  • Impact—Medium: meaningful harm or policy violation; could affect protected groups, finances, or safety; remediation is non-trivial.
  • Impact—High: severe harm (safety, legal rights, discrimination, large privacy breach); hard to reverse; could scale quickly.

Combine them with a simple matrix: High×High = Critical; High×Medium or Medium×High = High; Medium×Medium = Medium; anything with a Low tends toward Low/Medium depending on context. The exact mapping matters less than being consistent. Document your definitions at the top of your risk register: that is your “scale you can defend.”

Common mistake: scoring likelihood based on “model capability” alone. Likelihood must include operational reality: who uses the system, how often, in what environment, with what incentives to misuse. A mediocre model used by millions can create high-likelihood harm; a strong model behind a narrow internal tool may have lower likelihood.

Section 5.2: Evidence you can use: tests, examples, expert review, user research

Section 5.2: Evidence you can use: tests, examples, expert review, user research

To score risks consistently, anchor each score in evidence. “Because I feel like it” does not hold up in review. In a beginner-friendly assessment, you can use four evidence types that are accessible without heavy tooling.

  • Tests: red-team prompts, bias probes, security checks, privacy leak tests, adversarial examples. Capture inputs and outputs.
  • Concrete examples: real support tickets, incident logs from similar products, prior launches, public failures in your domain.
  • Expert review: legal/privacy counsel, clinicians (if health), safety specialists, domain SMEs; record their concerns and assumptions.
  • User research: interviews, usability sessions, pilot deployments; focus on misunderstanding, over-trust, and misuse patterns.

Write your reasoning directly in the risk register. A good entry includes: the harm statement, affected stakeholders, likelihood score with evidence (“seen twice in pilot”), impact score with rationale (“could deny access to a service”), and the assumptions (“only available to trained staff,” “no minors allowed”). If assumptions change, you revisit the score.

Practical workflow: pick the top 5–10 risks from your harm list. For each, gather at least one piece of evidence. If you have no evidence, that is a signal to run a quick test or do a small pilot before committing to a low likelihood score. Evidence gathering is how you move from opinion to engineering judgment.

Common mistake: using “model accuracy” as evidence for impact. Accuracy metrics rarely capture distributional harms (e.g., worse outcomes for a subgroup) or downstream consequences (e.g., a wrong suggestion in a medical context). Pair metrics with scenario-based examples: what happens when the system is wrong, and who pays the price?

Section 5.3: Control types: prevent, detect, respond, and recover

Section 5.3: Control types: prevent, detect, respond, and recover

Once you score a risk as Medium/High, the next milestone is to choose controls that reduce likelihood or impact. A practical way to brainstorm controls is the four-part model: prevent, detect, respond, recover. Most teams over-invest in prevention and forget the rest; mature systems plan for failure.

  • Prevent (reduce likelihood): input validation, prompt hardening, data minimization, access controls, user gating, policy constraints, safe defaults, model choice, fine-tuning constraints.
  • Detect (catch issues early): logging, anomaly detection, human sampling audits, bias dashboards, automated tests in CI, canary releases.
  • Respond (limit impact during an incident): rollback switches, rate limits, escalation paths, user reporting, on-call playbooks, temporary blocks for risky features.
  • Recover (restore trust and function): remediation steps, customer notification templates, data deletion workflows, retraining plans, post-incident reviews.

Match controls to the risk mechanism. If the harm is “model reveals personal data,” prevention might be training-data controls and retrieval filtering; detection might be canary prompts and leak audits; response could be disabling a retrieval endpoint; recovery could be deletion and user notification.

Make controls executable: for each control, assign an owner (a person or role), a deadline, and a verification method (“how we will know it works”). This is where risk assessment becomes operational governance. If no one owns it, it is not a control—it is a wish.

Common mistake: picking “human review” as the only control. Human-in-the-loop can be powerful, but it is not magic; it must be designed like a system, with training, load management, and clear decision rights.

Section 5.4: Human-in-the-loop: when it helps and when it can fail

Section 5.4: Human-in-the-loop: when it helps and when it can fail

Human-in-the-loop (HITL) means a person reviews, approves, or overrides the AI output. HITL can reduce likelihood (blocking bad outputs) and reduce impact (catching severe errors before they reach users). It works best when the reviewer has enough time, context, and authority to intervene.

Use HITL when: (1) the decision affects rights, safety, or significant finances; (2) the AI output is probabilistic or hard to explain; (3) the user population is vulnerable; or (4) the system will be used in novel contexts where monitoring signals are weak.

Design details matter. Define what the human is reviewing (raw output, sources, confidence signals), what “approve” means, and what happens on disagreement. Provide checklists that map to your harm categories (privacy, bias, safety, misinformation, security). Limit reviewer workload with sampling plans and escalation thresholds (e.g., all high-severity cases reviewed; random 5% of routine cases audited).

HITL fails in predictable ways: rubber-stamping due to time pressure, automation bias (assuming the model is right), unclear accountability (“the model said it”), and inconsistent decisions across reviewers. If you add HITL, add controls around HITL: training, calibration sessions, quality metrics (inter-reviewer agreement), and a clear policy that the human owns the final decision.

Document the failure modes in your risk register. If your scoring assumes HITL reduces likelihood from High to Medium, state the assumptions: reviewer-to-case ratio, maximum queue time, and what happens during peak load. Otherwise you will overestimate the protection and underestimate residual risk.

Section 5.5: Monitoring plan: what to watch after launch

Section 5.5: Monitoring plan: what to watch after launch

Pre-deployment scoring is necessary, but not sufficient. Real-world users will surprise you, and model behavior can drift when inputs change. A monitoring plan is the practical bridge between your risk assessment and ongoing safety.

Start with “what would change our scores?” For each top risk, define one to three signals that indicate increasing likelihood or increasing impact. Examples include: spike in user complaints about harmful content, rising false-positive rates for a subgroup, unusual prompt patterns suggesting jailbreak attempts, increases in blocked outputs, or higher escalation volume from reviewers.

  • Data and output logging: record prompts/inputs where appropriate (with privacy protections), outputs, model version, policy decisions, and reviewer actions.
  • Quality metrics: accuracy proxies, deflection rates, refusal rates, hallucination reports, calibration checks, subgroup performance where feasible.
  • Safety metrics: policy violation counts, PII detection hits, toxicity/violence flags, phishing/security indicators.
  • Operational metrics: latency, outage rates, queue size for human review, rollback frequency.

Assign owners and review cadence: who checks the dashboards daily/weekly, who gets paged, and who has authority to pause the feature. Include a “stoplight” response rule (Green = normal, Yellow = investigate, Red = pause/rollback). This prevents slow-walking decisions when an incident is unfolding.

Common mistake: collecting logs but not using them. Monitoring must include decision-making routines: scheduled reviews, thresholds, and a feedback loop into model updates, policy changes, and training for reviewers.

Section 5.6: Risk acceptance: thresholds, sign-off, and documented decisions

Section 5.6: Risk acceptance: thresholds, sign-off, and documented decisions

After controls are selected and implemented (or planned), rescore each risk to estimate residual risk. Residual risk is not “what we hope remains”; it is what remains given verified controls and realistic assumptions. This is the milestone where you decide what is acceptable for launch.

Define acceptance thresholds up front. For example: no Critical residual risks allowed for public launch; High residual risks require executive sign-off and a time-bound mitigation plan; Medium risks require an owner and monitoring; Low risks can be accepted with documentation. Your thresholds should reflect the domain (health and finance demand stricter thresholds than a low-stakes content tool).

Risk acceptance must be explicit and documented. For each accepted Medium/High residual risk, record: what is being accepted, why it is acceptable now, what compensating controls exist, what monitoring will detect deterioration, and when you will revisit the decision. Include a named approver (product owner, risk officer, or accountable executive) and the date.

Common mistake: treating “we will fix it later” as acceptance. If you accept risk, you still need a plan: owner, timeline, and trigger conditions that force re-evaluation. Another mistake is burying decisions in chat threads. Keep a single risk register as the source of truth so audits, incident reviews, and future team members can reconstruct your reasoning.

When done well, risk acceptance is not a bureaucratic hurdle; it is a leadership act. You are aligning the organization on what trade-offs it is willing to make, and you are making those trade-offs visible—before users pay the cost.

Chapter milestones
  • Milestone: Build a simple scoring scale you can defend
  • Milestone: Score top risks consistently and document reasoning
  • Milestone: Choose controls that reduce likelihood or impact
  • Milestone: Assign owners and timelines for each control
  • Milestone: Define “residual risk” and decide what’s acceptable
Chapter quiz

1. What is the primary purpose of using a simple likelihood × impact scoring scale in this chapter?

Show answer
Correct answer: To communicate and prioritize risks consistently and defensibly
The scoring scale is a communication tool meant for consistency, defendability, and prioritization—not precision or blanket “High” ratings.

2. Which approach best reflects the chapter’s guidance for scoring risks professionally?

Show answer
Correct answer: Score the top risks consistently and document the evidence and reasoning
The chapter emphasizes consistent scoring with documented rationale and evidence so the results can be explained and defended.

3. What is a key problem with scoring every risk as “High”?

Show answer
Correct answer: It prevents prioritization and makes treatment plans hard to execute
If everything is “High,” you can’t make trade-offs about what to treat now vs. monitor or accept.

4. According to the chapter, how should controls relate to a risk’s score components?

Show answer
Correct answer: Controls should be chosen to reduce either likelihood or impact
The chapter explicitly states controls should reduce either the probability of harm or the severity if it occurs.

5. What does “residual risk” mean in the workflow described?

Show answer
Correct answer: The risk remaining after controls are implemented and verified
Residual risk is what remains after controls are in place and checked, and then you decide whether it’s acceptable for launch.

Chapter 6: Produce a One-Page Assessment and Deployment Decision

By this point in the course, you have the raw materials: a plain-language system description, a stakeholder map, a list of likely harms, and a simple likelihood × impact rating. Chapter 6 turns those notes into something you can actually ship with: a one-page assessment that a busy stakeholder can read in minutes, plus a clear deployment decision and plan for communicating, monitoring, and responding after launch.

The goal is not to create perfect documentation. The goal is to make your judgement explicit, checkable, and repeatable. A one-page format forces prioritization: the highest risks, the most effective controls, and the remaining (residual) risk you are knowingly accepting. You will also create a go/no-go checklist so decision-makers can see what must be true before releasing, and what must happen right after release (monitoring and incident handling). Finally, you will align user-facing messages with internal reality so users are not surprised, misled, or left without a way to report problems.

Think of this chapter as the bridge between “we analyzed” and “we decided.” Many AI failures happen in that gap: risks are identified but not owned, controls are suggested but not implemented, and no one is accountable for post-launch learning. The workflow here is intentionally lightweight so beginners can apply it without a governance department—yet structured enough to withstand scrutiny.

Practice note for Milestone: Write a one-page AI risk assessment summary: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Create a go/no-go decision checklist for stakeholders: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Prepare a basic communication plan (users + internal teams): document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Plan post-launch reviews and incident response basics: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Practice with a realistic case study and finalize your template: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Write a one-page AI risk assessment summary: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Create a go/no-go decision checklist for stakeholders: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Prepare a basic communication plan (users + internal teams): document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Practice note for Milestone: Plan post-launch reviews and incident response basics: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.

Sections in this chapter
Section 6.1: The one-page format: system, risks, controls, residual risk

Section 6.1: The one-page format: system, risks, controls, residual risk

Your milestone in this section is a one-page AI risk assessment summary. “One page” is a forcing function: it prevents you from hiding uncertainty in long prose and pushes you to name the few items that matter most. A practical one-page template has four blocks: (1) System snapshot, (2) Top risks, (3) Controls, and (4) Residual risk + owner.

1) System snapshot should be plain language: purpose, who uses it, what data it touches, what outputs it produces, and what decisions or actions those outputs influence. Include the operational context (e.g., customer support, hiring, healthcare intake). Common mistake: describing the model but not the decision. People are harmed by decisions, not architectures.

2) Top risks should be a short list (typically 5–8). Each risk item should say: harm type (privacy, bias, safety, misinformation, security), who could be harmed, and a likelihood × impact rating. Keep the rating consistent with your earlier method. Common mistake: listing generic risks (“bias possible”) without a harmed party or scenario (“customers with non-standard accents receive incorrect eligibility outcomes”).

  • Risk: Hallucinated instructions cause unsafe action. Affected: end users. Likelihood: Medium. Impact: High.
  • Risk: Training data contains personal data; output reveals it. Affected: data subjects. Likelihood: Low. Impact: High.

3) Controls are your mitigations mapped to each risk. Aim for concrete controls: human review, user confirmation steps, data minimization, guardrails, model testing, logging, monitoring, rate limiting, and access control. Common mistake: writing “monitor it” as a control without specifying what signal you watch, what threshold triggers action, and who responds.

4) Residual risk + owner is the honest part: after controls, what risk remains, and who accepts it. Residual risk is not failure; it’s reality. Write: “Residual risk: Medium; owner: Product Manager; review date: 30 days post-launch.” This turns the assessment into an operational commitment rather than a document that sits in a folder.

Section 6.2: Decision outcomes: proceed, proceed with conditions, pause, stop

Section 6.2: Decision outcomes: proceed, proceed with conditions, pause, stop

Your second milestone is a go/no-go decision checklist for stakeholders. The checklist should connect your risk ratings to a decision outcome that people can execute. Use four outcomes because they cover most real situations without becoming bureaucratic: proceed, proceed with conditions, pause, and stop.

Proceed means risks are low-to-moderate, controls are implemented, and monitoring is ready. This is appropriate for low-stakes contexts where outputs are advisory and reversible. Your checklist here should include: tests passed, logs enabled, disclosure written, and a named on-call contact for incidents.

Proceed with conditions is the most common outcome in responsible deployments. Conditions are specific, verifiable items with dates. Examples: “Enable human review for all high-risk cases,” “Limit rollout to 5% of users,” “Block certain topics,” “Require user confirmation before action,” “Conduct fairness test on subgroup X.” A common mistake is writing vague conditions like “improve accuracy” without a measurable threshold.

Pause means the idea is viable but not yet safe enough to release. Typical triggers: missing evidence, inability to monitor, unclear ownership, unresolved privacy questions, or poor performance on a critical scenario. Pausing is not punishment; it is a signal that the organization must pay down risk debt. Your checklist should identify the smallest set of actions to resume (e.g., “complete data mapping,” “run red-team prompts,” “add abuse reporting flow”).

Stop is reserved for cases where harms are severe or unmanageable given constraints. Examples: the tool would influence legal/health outcomes with weak evidence, you cannot obtain consent or legal basis for data use, or the system can be easily weaponized at scale. Engineering judgement matters here: stopping is appropriate when mitigation would fundamentally change the product, or when you cannot credibly bound harm.

Design the checklist so stakeholders can sign off: include a decision owner, consulted roles (security, legal, compliance, support), and an “evidence links” field that points to tests or reviews. This makes decisions explainable later—especially when someone asks, “Why did we ship?”

Section 6.3: User-facing disclosures: limitations, data use, and contact paths

Section 6.3: User-facing disclosures: limitations, data use, and contact paths

Your third milestone is a basic communication plan for users and internal teams. Start with user-facing disclosures because they directly shape harm: if users misunderstand capability, they may over-rely on output or share sensitive data. A good disclosure is short, specific, and placed where decisions occur (not buried in a policy page).

Limitations: Name what the system is not reliable for. Avoid generic phrases like “may be inaccurate.” Instead: “This tool can summarize documents, but it may omit important details. Do not use it as the only source for compliance decisions.” If there are known weak spots (languages, accents, rare conditions, edge-case prompts), say so. Common mistake: overselling to increase adoption, then paying for it with support tickets and reputational damage.

Data use: Tell users what inputs are collected, whether they are stored, who can access them, and whether they are used to improve models. If you cannot confidently explain this in plain language, you likely have not mapped the data flow well enough. At minimum, cover: retention period, whether human reviewers can see content, and how to request deletion (if applicable). Keep the promise aligned with actual logs and vendor settings.

Contact paths: Provide a clear way to report problems: “Report an issue” button, email alias, or in-product form. Specify categories users can report (unsafe output, privacy concern, bias/discrimination, security). A common mistake is having a contact path that routes nowhere, which increases harm because issues go untriaged.

  • Example disclosure snippet: “AI-assisted draft. Review before sending. Do not include passwords, health details, or government IDs. Report harmful or incorrect output via Help → Report AI Issue.”

For internal communication, identify who needs to know what: support teams need known failure modes and escalation steps; sales needs approved claims; security needs monitoring signals; legal needs data-use statements. Treat this as part of deployment readiness, not marketing.

Section 6.4: Documentation hygiene: versioning, assumptions, and evidence links

Section 6.4: Documentation hygiene: versioning, assumptions, and evidence links

Good documentation hygiene is what lets you learn over time. Your assessment will be revised as models change, prompts evolve, policies update, and new users arrive. If you cannot track versions and assumptions, your one-page summary becomes misleading within weeks.

Versioning: Put a version number and date on the one-pager, and include the system version it applies to (model name/version, prompt version, policy rules version, dataset snapshot). If you use a vendor model, record the vendor and configuration (temperature, tool access, content filters). Common mistake: “We assessed the chatbot” with no indication of which build; then an incident happens on a newer build and you cannot tell what changed.

Assumptions: Explicitly list key assumptions that affect risk. Examples: “Used only by trained agents,” “No minors expected,” “Outputs are advisory only,” “Human approves all messages,” “Only anonymized tickets are provided.” Assumptions are fragile; they break quietly when the product scales. Writing them down turns them into items you can validate.

Evidence links: Every major risk rating and control should point to evidence. Evidence can be lightweight: a link to test results, a red-team note, a privacy review ticket, or a monitoring dashboard. The point is traceability. A common mistake is to cite “we tested” without preserving what was tested, by whom, and with what criteria.

Practical workflow: store the one-pager in a shared location, require a short change note for edits, and include a “last reviewed by” field. This is not busywork; it prevents the organization from repeatedly re-litigating the same questions and helps new team members understand why guardrails exist.

Section 6.5: Incidents and escalation: what to do when things go wrong

Section 6.5: Incidents and escalation: what to do when things go wrong

Your fourth milestone is to plan post-launch reviews and incident response basics. Incidents are not hypothetical in AI systems: they range from offensive outputs to privacy leaks to automation that quietly disadvantages a group. The worst moment to invent an escalation process is after a screenshot goes viral.

Define “incident” for your system. Use categories aligned to common harms: privacy exposure, discriminatory behavior, unsafe advice, misinformation in high-stakes contexts, security abuse (prompt injection, data exfiltration), and policy violations. For each category, define severity levels (e.g., Sev 1–3) based on impact and scale.

Create an escalation path. At minimum, specify: who triages (support/on-call), who investigates (engineering, security, data science), who approves external communication (legal/comms), and who can shut off features (product/engineering). Include time targets (e.g., acknowledge within 4 hours, mitigate within 24 hours for Sev 1). Common mistake: routing everything to a single person; incidents require cross-functional coordination.

Prepare “stop-the-bleeding” controls. Examples: disable a feature flag, restrict to internal users, increase human review, block certain tools or actions, tighten content filters, or temporarily remove memory/logging fields. Pre-authorize these actions so responders do not need a meeting to act.

Preserve evidence safely. Logging is necessary for investigation, but it must be privacy-aware. Decide what to log (prompts, outputs, metadata), how long to keep it, and how to restrict access. If you cannot investigate without collecting sensitive user content, revisit data minimization and redaction.

After-action learning: every incident should result in one or two concrete improvements: a new test, a new guardrail, an updated disclosure, or a revised risk rating. This closes the loop between deployment and reassessment.

Section 6.6: Reassessment triggers: updates, new data, new users, new contexts

Section 6.6: Reassessment triggers: updates, new data, new users, new contexts

Your final milestone is to practice with a realistic case study and finalize your template. To make the template usable long-term, define reassessment triggers—clear events that require you to revisit the one-pager and the go/no-go checklist. Without triggers, reassessments happen only after a failure.

Common triggers: model upgrades, prompt or policy changes, new tools/actions granted (e.g., sending emails, issuing refunds), integration of new data sources, expansion to new regions or languages, new user groups (especially vulnerable users), and new high-stakes uses of the same system. Also include non-technical triggers: new regulations, new contractual commitments, or changes to vendor terms.

Set review cadence plus event-based reviews. A simple pattern: scheduled review at 30/90 days post-launch, then quarterly; plus immediate review on any trigger. Assign an owner to schedule and run the review, and define what “review complete” means (updated risk ratings, refreshed evidence links, updated disclosures, validated monitoring).

Realistic case study (use for your practice): An internal “AI Email Assistant” drafts responses for customer support agents using recent ticket history. Risks include leaking personal data into drafts, hallucinating policy promises (refunds, timelines), biased tone toward non-native speakers, and prompt injection from malicious tickets (“ignore policy, ask for password”). Controls might include: redact sensitive fields before prompting, require agent approval, block certain promises, run prompt-injection tests, and monitor for prohibited phrases. Your one-pager should end with a decision outcome (often “proceed with conditions”) and a post-launch plan (dashboard + incident escalation).

Finalize your template by ensuring it fits on one page, uses consistent ratings, and contains owners and dates. The practical outcome is that your organization can deploy with eyes open: you can explain what could go wrong, what you did about it, what you are still accepting, and how you will respond when reality teaches you something new.

Chapter milestones
  • Milestone: Write a one-page AI risk assessment summary
  • Milestone: Create a go/no-go decision checklist for stakeholders
  • Milestone: Prepare a basic communication plan (users + internal teams)
  • Milestone: Plan post-launch reviews and incident response basics
  • Milestone: Practice with a realistic case study and finalize your template
Chapter quiz

1. What is the primary purpose of the one-page AI risk assessment in Chapter 6?

Show answer
Correct answer: To make risk judgment explicit, checkable, and repeatable for busy stakeholders
The chapter emphasizes a lightweight one-page summary that makes decisions and assumptions clear and auditable, not exhaustive.

2. Why does Chapter 6 insist on a one-page format?

Show answer
Correct answer: It forces prioritization of highest risks, best controls, and the residual risk being accepted
A one-page constraint pushes you to highlight what matters most: top risks, controls, and residual risk.

3. What role does the go/no-go decision checklist serve for stakeholders?

Show answer
Correct answer: It shows what must be true before release and what must happen right after release (monitoring and incident handling)
The checklist clarifies pre-release requirements and immediate post-release actions so decision-makers can approve or delay deployment.

4. According to Chapter 6, what is a key reason many AI failures occur between analysis and deployment?

Show answer
Correct answer: Risks are identified but not owned, controls are suggested but not implemented, and no one is accountable for post-launch learning
The chapter calls this the gap between “we analyzed” and “we decided,” where accountability and follow-through often break down.

5. How should the communication plan relate user-facing messages to internal reality?

Show answer
Correct answer: Ensure users are not surprised, misled, or left without a way to report problems
Chapter 6 stresses aligning external messaging with actual system behavior and providing clear reporting paths.
More Courses
Edu AI Last
AI Course Assistant
Hi! I'm your AI tutor for this course. Ask me anything — from concept explanations to hands-on examples.