AI Ethics, Safety & Governance — Beginner
Confidently map AI harms, controls, and decisions before you deploy.
This course is a short, book-style guide that teaches you how to do an AI risk assessment from the ground up. You don’t need to code, build models, or understand complex math. You’ll learn a practical way to describe an AI system, identify who it affects, spot common harms, and document decisions clearly—before the system reaches real users.
AI risk assessments are not only for large organizations or legal teams. If you’re experimenting with chatbots, automated screening tools, content generators, or “AI-powered” features, you can use the same simple logic: define what the AI does, map how its outputs become real-world outcomes, and put safeguards in place where the risks are highest.
By the end, you will have a beginner-friendly toolkit you can reuse:
The course has exactly six chapters that build on each other. First, you learn what “risk” means in everyday terms. Next, you practice describing an AI system so other people can review it. Then you map stakeholders and impact pathways, which helps you see hidden effects. After that, you use checklists to find common AI risks. You then score and treat risks with practical controls. Finally, you produce a one-page assessment and make a clear deployment decision, including what to monitor after launch.
This course is designed for absolute beginners: students, founders, product managers, policy staff, procurement teams, analysts, and anyone asked to “review AI” without being a data scientist. It also works well for small organizations that need a lightweight, repeatable approach before using third-party AI tools.
Pick one real or realistic AI use case (for example: a customer support chatbot, an AI résumé screener, a classroom tutoring assistant, or an AI content generator). As you move through the chapters, you will fill in the same risk log and template. The goal is not perfect prediction—it’s clear thinking, documentation, and safer decisions.
When you’re ready, you can Register free to start learning, or browse all courses to build a complete governance skill path.
You will finish with a practical, shareable AI risk assessment you can adapt to many projects—helping you reduce harm, improve trust, and avoid preventable deployment mistakes.
AI Governance Lead and Risk Assessment Specialist
Sofia Chen designs practical AI governance programs for product teams and public-sector services. She specializes in beginner-friendly risk assessments, documentation, and safeguards that can be applied before launch. Her work focuses on turning ethics goals into clear decisions and measurable controls.
An AI risk assessment is a practical way to think ahead: what could go wrong, who could be affected, how bad it could be, how likely it is, and what you can do about it before people rely on the system. It is not a legal memo and not a philosophical debate. It is an engineering and product habit: describe the system in plain language, map impacts first, rate risks in a repeatable way, and choose controls that match the risks.
Beginners often assume risk assessment requires math, specialized jargon, or perfect knowledge of the model. You do not need any of those. You need a clear description of the input-to-output pathway (what goes in, what the system does, what comes out, what decision follows), plus a disciplined way to consider harms such as privacy loss, bias, safety failures, misinformation, and security abuse.
This chapter sets up your workflow for the whole course. You will learn to: treat AI as an input-to-output system (no math required), define “risk” as harm × likelihood × impact, run the assessment early in the project timeline, set scope tightly (one use case, one decision, one context), and start a simple risk log you will build throughout the course.
Keep one idea in mind as you read: the goal is not to eliminate all risk. The goal is to make risk visible early enough that you can reduce it cheaply, explain it honestly, and monitor what remains.
Practice note for Milestone: Understand AI as an input-to-output system (no math required): document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Learn what “risk” means: harm, likelihood, and impact: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Know when to run a risk assessment in a project timeline: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Set your scope: one use case, one decision, one context: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Start a simple risk log you will build throughout the course: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Understand AI as an input-to-output system (no math required): document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Learn what “risk” means: harm, likelihood, and impact: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Know when to run a risk assessment in a project timeline: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
For risk work, you don’t need to start with algorithms. Start with a plain-language system sketch: inputs → transformation → outputs → decision/action. Inputs might be text prompts, images, customer records, sensor data, or clicks. The transformation might be a model (machine learning), a rules engine, or a hybrid workflow. Outputs might be a score, a label, a summary, a recommendation, or generated content. Finally, there is almost always a decision or action: approve a loan, flag a transaction, route a support ticket, draft an email, or show a piece of content to a user.
Risk hides in the handoffs. A model may only output “suggested priority,” but the organization may treat it as “truth” and auto-route urgent cases away from humans. That’s why your assessment must include the automation boundary: what is automated, what is reviewed, and what is merely suggested.
Use a “no-math” description that anyone on the team can understand. Write three sentences: (1) Purpose: why the system exists. (2) Users: who interacts with it (operators, end users, affected non-users). (3) Data and outputs: what data is used and what is produced. Then add one sentence describing the decision that follows the output.
Common mistake: describing the model but not the system. A risk assessment is about the full socio-technical workflow: data collection, UI design, incentives, time pressure, and how outputs are used. When you map it as a pipeline, you can see where harms can enter: bad input data, ambiguous prompts, brittle edge cases, or over-automation.
In this course, “harm” means a negative outcome that matters to someone. Start by naming stakeholders, then asking: Who could be helped? Who could be harmed? How? Stakeholders include direct users, people affected by decisions, people whose data is used, internal staff, vendors, regulators, and communities impacted at scale.
Use a simple checklist to avoid missing obvious harms. You are not trying to be exhaustive; you are trying to be systematic enough for beginners to catch the common failure modes.
Also consider organizational harms: legal exposure, brand damage, operational disruption, and loss of trust. And societal harms: scaling discrimination, enabling spam, shifting power asymmetrically, or eroding information quality.
Practical technique: for each harm category, write one “impact story” in plain words: “If the system does X, then Y could happen to Z.” Example: “If the assistant drafts a payment-link email using the wrong account number, a customer could send money to the wrong place, causing financial loss and support burden.” These short stories become entries in your risk log.
Teams get stuck when they mix up three different concepts: risk, issue, and uncertainty. Separating them makes your assessment actionable.
Risk is a potential future harm: it has a cause, an outcome, a likelihood, and an impact. Example: “Support agent may accidentally paste sensitive customer data into a public channel because the UI encourages copy/paste from the AI draft.” That is a risk because it might happen, and you can reduce it with controls.
Issue is a harm that is already happening or a defect you have already observed. Example: “In testing, the model included credit card numbers in its summary.” Issues require remediation now (bug fix, rollback, data purge), not just scoring.
Uncertainty is what you do not yet know that affects your confidence in the risk picture. Example: “We don’t know whether the vendor retains prompts for training.” Uncertainties are not “fine”; they are work items. Convert uncertainty into a plan: ask the vendor, run a test, measure, or document an assumption.
This distinction helps you build a beginner-friendly likelihood × impact method. Likelihood is your best judgement about how often the harm could occur in the real setting, not in a perfect lab. Impact is how severe the harm would be if it occurred. Use a simple 1–5 scale for each, then multiply to get a score for prioritization.
Common mistake: treating the score as truth. The score is a conversation tool to rank what to address first. Document your reasoning in one sentence. If you cannot justify the rating, you probably have an uncertainty to resolve.
Running an AI risk assessment before deployment is not bureaucracy; it is a cost-saving move. Before deployment, you can change the product shape: remove a risky feature, narrow the use case, add friction, redesign the human review step, or choose different data. After deployment, you are often stuck with users, integrations, contractual promises, and reputational expectations.
“Before deployment” also changes what evidence you can collect. Early on, you can do controlled testing, simulate edge cases, and run red-team exercises without harming real people. You can instrument logging and monitoring from day one instead of trying to retrofit it later. You can decide up front what good performance means and for whom, rather than discovering late that your metrics ignore a harmed group.
Place the assessment at multiple points in a simple timeline:
Common mistake: waiting for “model readiness” before thinking about risk. Many harms come from product choices (auto-send vs. draft, who can access outputs, default sharing settings) and can be prevented long before the model is finalized.
Practical outcome: by the time you deploy, you should already know (1) what risks remain, (2) who accepted them, (3) what controls reduce them, and (4) how you will detect if reality diverges from your assumptions.
Beginners try to assess “the AI” in general and get overwhelmed. Scope is your safety rail. In this course, your default scope is: one use case, one decision, one context. That means one concrete workflow (use case), one specific decision/action that follows the output, and one environment (who uses it, where, under what constraints).
Define boundaries explicitly:
Write assumptions so readers know what your ratings depend on. Example assumptions: “Agents must approve drafts before sending,” “No medical advice is allowed,” “The model cannot access raw payment data,” “All logs are retained for 30 days.” Assumptions are powerful because if any assumption is false in production, your likelihood scores may be wrong.
Also write exclusions honestly. Example: “We are not assessing third-party downstream reuse of exported reports.” Exclusions are not excuses; they are prompts to schedule the next assessment or add contractual controls.
Common mistake: scoping only the model and excluding the decision. If the decision is “auto-reject applicants,” the risk profile is radically different than “surface a recommendation for a trained reviewer.” The same output, used differently, creates different harms. Your scope statement should always name the decision that matters.
You will use three tools throughout this course: lightweight checklists, a living risk log, and a one-page assessment template. Together, they keep you practical and consistent without slowing the project down.
1) Checklists (to generate risks): Use the harm categories from Section 1.2 as your baseline, then add “operational” prompts: What happens when the model is wrong? When it refuses? When inputs are malicious? When the user is rushed? When the system is down? This is engineering judgement: imagine realistic failure modes, not just worst-case fantasies.
2) The risk log (to track and prioritize): Start a simple table you can maintain in a spreadsheet. Each row is one risk story. Recommended columns:
3) The one-page template (to communicate): Put the essentials on one page: system description (inputs/outputs/decision), scope, top stakeholders, top 5 risks with scores, and the controls you will implement before launch. This is what aligns product, engineering, legal, and operations.
Choosing controls should be matched to the risk, not chosen from a generic list. Examples of practical controls you will revisit in later chapters include: human review steps for high-impact decisions, data minimization and retention limits for privacy risk, targeted testing for bias and performance gaps, guardrails against prompt injection, output filtering, rate limits, and monitoring dashboards plus an incident response path.
Common mistake: writing a risk log that never changes. Treat it as a living artifact. When you learn something (a test result, a near-miss, a new user behavior), update likelihood, impact, and controls. That habit—mapping impacts first, then updating with evidence—is what makes an AI risk assessment real.
1. Which description best matches what an AI risk assessment is in this chapter?
2. To describe an AI system for risk assessment, what is the most important structure to capture first?
3. In this chapter, “risk” is defined using which combination of factors?
4. When does the chapter recommend running an AI risk assessment in a project timeline?
5. Which scoping choice best follows the chapter’s guidance for beginners?
A risk assessment can only be as good as the system description behind it. Beginners often jump straight to “Is it biased?” or “Could it leak data?” without first pinning down what the AI actually does, who uses it, and what happens next. Reviewers can’t evaluate risks they can’t see, and teams can’t mitigate risks they haven’t named.
This chapter gives you a practical, plain-language way to describe an AI system so a non-expert reviewer (a manager, legal counsel, security engineer, or an impacted stakeholder) can understand it in minutes. You will write a concise system summary, map the user journey end-to-end, list what data is involved and where it comes from, document dependencies, and define what “success” and “failure” look like. These milestones are the backbone of the rest of the course: once you can describe the system clearly, you can map impacts and choose controls with confidence.
A useful description is not marketing copy and not an architecture diagram. It is a reviewer’s map: purpose, users, inputs, outputs, decisions, data, dependencies, and boundaries. You’ll also notice an important theme: many “AI risks” are actually process risks around the AI—handoffs, escalation, and what humans do with outputs.
Practice note for Milestone: Write a plain-language system summary (purpose and output): document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Map the user journey from prompt/input to action/decision: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Identify what data is used and where it comes from: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: List dependencies (tools, vendors, APIs, human reviewers): document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Define “success” and “failure” for the system: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Write a plain-language system summary (purpose and output): document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Map the user journey from prompt/input to action/decision: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Identify what data is used and where it comes from: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: List dependencies (tools, vendors, APIs, human reviewers): document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Start with the milestone that unlocks everything else: a plain-language system summary. Your goal is to make the system reviewable by someone who has never seen it. Keep it short, specific, and operational—avoid vague claims like “improves productivity” or “optimizes decisions.”
Write two sentences:
Then name the users and the job the AI is doing for them. “Users” is broader than “people typing prompts.” Include (1) direct users (operators), (2) indirect users (people who receive the output), and (3) affected parties (people the decision is about). A customer support summarizer has different stakeholders than a loan pre-screening model, even if both output text.
Engineering judgment: be explicit about whether the AI is advisory (suggests) or determinative (decides). Beginners often describe an AI as “just a tool,” but if the organization treats the output as authoritative, then functionally it is part of a decision system. Capture that reality in your purpose statement.
Common mistake: writing the description from the builder’s viewpoint (“uses a transformer model with RAG”) instead of the reviewer’s viewpoint (“answers employee policy questions using HR documents; provides citations”). Technical details can be added later, but you need a clear job-to-be-done first.
Next, map the user journey from prompt/input to output. List every input channel the system accepts, not just the main one. Inputs can include typed prompts, uploaded files, pasted text, form fields, clickstream/context signals, microphone audio, images, or API calls from other systems.
For each input, answer three reviewer-friendly questions:
Then describe outputs just as concretely. Outputs are not always “a response.” They may be a risk score, a ranking of candidates, a label (spam/not spam), a summary with citations, a recommended next action, or a generated artifact (email draft, code, medical note). Note where the output appears: in a dashboard, in a ticketing system, in a customer chat, or sent via webhook to another application.
Practical tip: include examples with placeholders, not real data. For instance: “Input: ‘Customer reports late delivery, order #12345’ → Output: ‘Suggested reply and refund policy excerpt.’” This makes the system legible without exposing sensitive content.
Common mistakes include forgetting “hidden inputs” (system prompts, retrieval queries, default context like user profile fields) and ignoring “secondary outputs” (logs, analytics events, stored embeddings). Reviewers care about these because they can become privacy, security, or IP risks even if users never see them.
Risk is created when an output becomes an action. A model can generate an incorrect answer, but harm happens when someone acts on it—denies a service, shares misinformation, triggers an investigation, or discloses private data. This milestone is about identifying the decision points.
Take your user journey and add a simple “so what?” layer:
Be explicit about automation level. Many systems start “human-in-the-loop” but drift into “human-on-the-loop,” where people rubber-stamp suggestions to keep up with volume. If the AI output is copied into customer communications, that is effectively a publication pathway. If the AI writes code that is deployed, that is effectively an engineering control pathway. Your description should reflect how the system is actually used, not how you wish it were used.
Define “success” and “failure” in terms of outcomes. Success is not “high accuracy” in the abstract; it is “reduces average handle time without increasing complaint rate,” or “identifies truly high-risk transactions with acceptable false positives.” Failure should include harmful modes: “hallucinates policy,” “recommends unsafe medical action,” “incorrectly flags legitimate users,” “reveals private details,” or “creates misleading confidence.”
Common mistake: describing decisions as if they happen only at the final step. In reality, impacts can occur earlier—during triage, prioritization, or what the system chooses to show or hide. Capture each decision point, even if it feels minor.
Now identify what data is used and where it comes from. For risk assessment, “data” includes anything the system consumes, retrieves, stores, or generates—training data (if you trained), prompts, uploaded documents, retrieved knowledge base content, conversation history, labels, and logs.
Start with a data inventory table in plain language (you can keep it in a doc): dataset/source, what it contains, who it’s about, who owns it, where it is stored, retention period, and who can access it. Then mark whether the data may include:
Provenance matters: describe where the data originated and how it was collected. Was it provided by the user knowingly? Imported from another system under a contract? Scraped from the web? Generated by another model? Each source carries different consent, quality, and IP implications.
Engineering judgment: distinguish between data used at runtime (e.g., retrieval-augmented generation pulling internal documents) and data used for improvement (e.g., storing prompts to fine-tune or evaluate). These are different risk surfaces. If prompts are stored, say so. If they are not, say how you ensure that (configuration, logging controls, vendor settings).
Common mistakes include assuming “we don’t store data” because the product UI doesn’t show storage, or overlooking that vendors may retain data by default. Another frequent gap is ignoring derived data such as embeddings, summaries, or labels—these can still encode personal or sensitive information.
AI systems are socio-technical: people are always part of the system, even when the model runs automatically. This milestone documents the human roles and the accountability chain so reviewers can see who can intervene when something goes wrong.
List the humans involved across the lifecycle:
Then define the oversight mechanism in behaviorally specific terms. “Human review” is not a control unless you specify: what is reviewed, when, by whom, with what checklist, and what authority they have to stop or override the AI. A practical description might be: “If the model’s confidence is below X or the content contains restricted topics, the draft cannot be sent until a supervisor approves.”
Include failure handling: how a user reports an issue, expected response time, and what triggers an escalation (e.g., suspected privacy breach, discrimination complaint, safety risk). Reviewers will look for a clear path from detection to correction.
Common mistake: treating accountability as “the user’s responsibility.” If your workflow encourages fast copying/pasting of outputs, you need explicit guardrails and training. Also note any human dependencies outside your team, such as contracted moderators, clinical reviewers, or customer support quality teams—these affect reliability and fairness.
Finally, draw the system boundary. Boundaries prevent two classic assessment failures: (1) ignoring key dependencies that introduce risk, and (2) trying to assess “AI” as if it exists alone, without the surrounding process.
Document dependencies (tools, vendors, APIs, and humans). Examples include: the foundation model provider, embedding/retrieval service, vector database, content filter, identity provider, ticketing system, cloud storage, analytics platform, and any third-party data feeds. For each dependency, capture what it does, what data it receives, and what happens if it fails.
Then define “where the AI stops.” Does the AI only generate a suggestion, after which a separate system sends the message? Does an orchestration layer transform outputs into API calls? Does a human take the output and perform an offline action? Your boundary statement should clarify what is in scope for your assessment now and what is a downstream process that still needs risk consideration.
Also specify environment and access boundaries: internal-only vs public-facing, authenticated vs anonymous, and what user permissions exist (who can upload files, who can view logs, who can export outputs). Seemingly small boundary details often dominate risk—public exposure changes abuse likelihood; broad internal access increases privacy and IP leakage risk.
Practical outcome: at the end of this chapter you should have a one-page “review packet” consisting of (1) purpose/output summary, (2) user journey map from input to action, (3) data inventory with provenance, (4) dependency list, and (5) success/failure definitions. That packet becomes the input to stakeholder mapping and harm checklists in the next chapters.
1. Why does the chapter say you should describe the AI system before evaluating risks like bias or data leakage?
2. Which best matches the chapter’s definition of a “useful” system description?
3. What is the goal of writing the plain-language system summary milestone?
4. What does mapping the user journey help you capture that affects risk?
5. According to the chapter, many “AI risks” are actually what kind of risks?
In Chapter 2 you learned to describe an AI system in plain language: what it does, who uses it, what data it touches, and what outputs it produces. In this chapter you turn that description into an “impact map” that shows who could be helped or harmed and through what pathway. This step is the heart of a beginner-friendly AI risk assessment: before you can rate risk, you must know where risk can land.
Many teams jump straight to checklists (bias, privacy, security) without first identifying the full set of stakeholders. That creates blind spots. For example, an AI scheduling tool may have “users” (managers) and “subjects” (workers), but also “bystanders” (customers affected by service quality) and “operators” (admins maintaining access). Each group can experience different impacts, and the same output can become a benefit for one group and a harm for another.
This chapter follows a practical workflow: (1) list direct and indirect stakeholders, (2) map impact pathways from model output to real-world outcomes, (3) look for second-order effects that emerge over time, (4) capture context—environment, power, and constraints—so you understand who is vulnerable and why, and (5) convert what you learned into clear risk statements you can rate and control in later chapters.
Your deliverable is a one-page artifact: a stakeholder list plus an impact pathway map. It does not need to be perfect. It needs to be legible, testable, and connected to concrete decisions your system influences.
Practice note for Milestone: Identify direct and indirect stakeholders: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Create an impact pathway map (who is affected and how): document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Spot hidden “second-order” effects: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Capture context: environment, power, and constraints: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Convert impacts into clear risk statements: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Identify direct and indirect stakeholders: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Create an impact pathway map (who is affected and how): document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Spot hidden “second-order” effects: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Capture context: environment, power, and constraints: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Start by listing stakeholders in four simple buckets. This milestone—identifying direct and indirect stakeholders—prevents the most common beginner mistake: treating “the user” as the only person who matters.
Users are people who directly interact with the AI output to do a task. Examples: a recruiter viewing candidate rankings, a teacher using an AI tutor dashboard, a customer asking a chatbot for support. Users experience impacts like productivity gains, over-trust, confusion, or increased workload from false alarms.
Subjects are people the AI is about. They may never touch the system. Examples: job applicants whose resumes are scored, patients whose notes are summarized, drivers whose images are captured for fraud detection. Subjects can experience privacy intrusion, unfair treatment, or denial of opportunities.
Bystanders are people indirectly affected by decisions made with AI. Examples: family members impacted by a lending decision, coworkers who inherit work when someone is mis-scheduled, community members affected by misinformation amplified by AI-generated posts. Bystanders are where hidden “second-order” effects often live.
Operators are people who deploy, monitor, and maintain the system: IT admins, customer support, model owners, compliance staff. They are exposed to operational harms (alert fatigue, unclear accountability, security incidents) and are crucial to implementing controls.
Practical step: create a table with columns Stakeholder, Role, What they want, How they might be harmed, How they might benefit. Do not stop at internal roles. Include vendors, contractors, and downstream partners who may reuse outputs.
Once you know who is involved, map the pathway from the AI’s output to real-world impact. The milestone here is creating an impact pathway map—who is affected and how. A useful pattern is: Output → Decision/Action → Outcome → Harm/Benefit.
Example: an AI tool outputs a “fraud risk score.” That output becomes an action (freeze account, require additional verification). The action causes an outcome (customer cannot pay rent on time). The outcome creates a harm (financial stress, fees, reputational damage) or a benefit (reduced fraud losses, safer marketplace).
Write pathways as short chains. Keep them concrete and observable. Avoid vague nodes like “bad experience.” Instead use measurable outcomes like “delayed service,” “denied eligibility,” “inaccurate public statement,” or “unauthorized data exposure.”
Engineering judgment matters here: identify where humans interpret outputs. Many harms are mediated by how people use the output, not the output itself. If your model provides a summary, the pathway includes whether a user verifies it, whether the summary is copied into a record, and whether it triggers a decision. Mark these handoffs explicitly.
Your map should make it easy to point at a box and say, “If this step goes wrong, this stakeholder experiences this outcome.” That traceability is what turns ethics concerns into actionable risk management.
Stakeholders are not affected equally. The milestone here is spotting hidden second-order effects and capturing context: environment, power, and constraints. “Vulnerable groups” are not defined by a label alone; vulnerability often comes from context—limited resources, reduced ability to recover from errors, or higher exposure to monitoring or enforcement.
Context questions to ask: Where is the system used (hospital, school, border crossing, workplace)? What happens if the model is wrong (minor inconvenience vs. life-changing denial)? What constraints do people face (language barriers, disability access, limited internet, fear of retaliation)?
Second-order effects often emerge when an AI system changes incentives over time. A call-center assistant that tracks “agent quality” may initially help coaching, but later be used to justify layoffs, leading agents to avoid complex cases. A content-moderation model may reduce visible toxicity but also push harmful communities to private channels where reporting is harder.
Practical step: add a “context layer” to each pathway. For each stakeholder, note: (1) what resources they have to respond, (2) what they risk by complaining, and (3) how frequently they are exposed. Then identify groups who are disproportionately represented among subjects (e.g., certain neighborhoods in surveillance footage) or who face higher stakes (e.g., patients with rare conditions where errors are more likely).
Common mistake: treating fairness as only “demographic parity” without considering harm severity. Two groups can have similar error rates, but one group can be harmed more because they have fewer alternatives or face compounding disadvantages.
Power dynamics determine whether an impact becomes a durable harm. This section connects to capturing context and constraints, and it directly influences which controls will work later (human review, appeals, monitoring). Ask: who can contest decisions, who can opt out, and who is locked in?
Map the “contestability path” next to your impact pathway. If the AI output leads to a decision, can a subject see that a model was involved? Can they request an explanation in plain language? Is there a human who can override the output, and do they have time and authority to do so?
Power is also about information. Users often get dashboards and confidence scores; subjects may get a one-line denial. That gap increases harm because people cannot correct errors. Similarly, operators may have logs but no clear policy for when to roll back a model, turning known issues into prolonged exposure.
Practical checklist:
Common mistake: assuming “human in the loop” automatically fixes risk. If humans are incentivized to follow the model (speed targets, liability fears), the loop becomes rubber-stamping. Your map should reflect real authority and real constraints, not idealized policy.
Risk assessment is not only about harms. Mapping benefits clarifies trade-offs and prevents a narrow focus on compliance. This milestone—benefit mapping—asks you to specify positive outcomes and who receives them, using the same pathway discipline: Output → Action → Outcome → Benefit.
Benefits should be explicit and distributed. If the benefit is “efficiency,” identify whose time is saved. If the benefit is “accuracy,” identify which decisions improve and whether improvements reach subjects or only the organization.
Example: an AI triage assistant outputs a prioritized queue. Benefits may include faster response for urgent cases (subjects), reduced cognitive load for clinicians (users), and lower operating costs (organization). But benefit mapping should also reveal benefit concentration: perhaps clinicians save time while patients experience more automation and less personal interaction.
Practical step: for each stakeholder group, write at least one intended benefit and one potential unintended benefit (e.g., a tool designed for support might also generate training data that improves future models). Then note what evidence would demonstrate that benefit: reduced wait time, fewer escalations, higher task completion, fewer incidents.
Common mistake: treating benefit as a justification to ignore harm (“net positive”). In governance work, benefits help you choose proportionate controls. If benefits are large and widely shared, investing in strong safeguards is easier to justify; if benefits are narrow and harms fall on others, you may need redesign, not just mitigations.
Your final milestone is converting impacts into clear risk statements. This is the bridge from mapping to rating. A good risk statement is specific, testable, and connected to a pathway. Use the pattern: If X happens, then Y harm may occur to Z stakeholder, because …
Start with “X” as a failure mode or misuse, not a general fear. Examples of X: “the model produces a false positive fraud score,” “the chatbot fabricates a policy,” “training data includes sensitive attributes,” “the model is prompt-injected to reveal private information,” “users rely on the score without verification.”
Then define “Y harm” in real-world terms: denial of service, reputational damage, unequal access, unsafe medical advice, privacy breach, or financial loss. Include “Z stakeholder” so ownership is clear. The “because” clause should point to the pathway step where the harm is realized (e.g., automatic enforcement, lack of appeal, high-stakes setting).
Practical examples:
Common mistake: writing risks as vague nouns (“bias,” “privacy”). Those labels belong in categories later. In this chapter your goal is crisp statements that you can later rate for likelihood and impact and match to controls like human review, data minimization, targeted testing, and monitoring.
1. Why does Chapter 3 emphasize mapping stakeholders and impact pathways before using common AI risk checklists (e.g., bias, privacy, security)?
2. In the chapter’s example of an AI scheduling tool, which pairing best reflects different stakeholder roles beyond just “users” and “subjects”?
3. What is an “impact pathway map” in Chapter 3?
4. What is the purpose of looking for “second-order effects” when mapping impacts?
5. Which sequence best matches the chapter’s practical workflow for creating the Chapter 3 deliverable?
In Chapter 3 you mapped impacts by describing the system, its stakeholders, and who could be helped or harmed. Now you turn that map into a risk assessment you can act on. A beginner-friendly risk assessment is not a legal memo or a spreadsheet marathon. It is a structured way to (1) spot common harms using checklists, (2) capture them as clear risk statements, and (3) prioritize what to fix first using a simple likelihood × impact rating.
This chapter is built around “milestones” you can run in under an hour for a small prototype, or in a day for a more complex system. Each milestone is a focused pass: privacy/data protection, fairness/accessibility, safety/misinformation, security/misuse, and finally prioritization. You will notice that the same issue can show up in multiple milestones (for example, personal data leakage is both a privacy issue and a security issue). That is normal; your goal is not to perfectly categorize risks, but to ensure you find them and assign practical controls.
A workable workflow looks like this: start with your plain-language system description (purpose, users, data, outputs, decisions). Then, for each checklist, write risks in the same format: “If
Common mistakes at this stage are (a) writing vague risks (“bias is possible”), (b) listing controls without verifying they are feasible (“we will perfectly filter everything”), and (c) skipping the “who is harmed” part, which leads to missing accessibility and downstream misuse. Use the sections below as a practical set of checklists you can reuse on every project.
Practice note for Milestone: Run a privacy and data-protection check: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Run a fairness and accessibility check: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Run a safety and misinformation check: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Run a security and misuse check: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Prioritize risks worth addressing first: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Run a privacy and data-protection check: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Run a fairness and accessibility check: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Run a safety and misinformation check: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Milestone: Run a privacy and data-protection check. Privacy risks show up wherever your system touches data: what you collect, how long you keep it, where it flows, and who can see it. Start by listing each data type the system uses (user inputs, uploaded files, logs, model outputs, analytics events, support tickets). Then ask four questions: collection, retention, leakage, and consent.
Turn findings into concrete risks. Example: “If customer support agents paste full customer records into the chatbot, then personal data may be stored in vendor logs, affecting customers, because the tool retains prompts by default.” Controls should match the cause: input guidance (“do not paste IDs”), data minimization (masking, redaction), retention limits, and access controls. Another practical control is purpose limitation: only use data for the feature the user expects, and keep it no longer than needed.
Engineering judgement matters in tradeoffs. Debug logs are valuable, but raw prompts are often too sensitive to keep. A common beginner error is assuming “we don’t store data” because the application doesn’t save it—while the model provider, monitoring stack, or error reporting tool might. Trace the full pipeline, including third parties, and write down defaults you must change.
Milestone: Run a fairness and accessibility check. Fairness risks are easiest to find when you connect them to user groups and decision points. Ask: does the system recommend, rank, screen, approve, deny, or prioritize anything that affects people’s opportunities? Even “assistive” tools can create unfair outcomes if humans over-trust them or if they work better for some groups than others.
Bias can come from multiple sources: training data that under-represents a group, labels that reflect historical discrimination, proxies (zip code acting like race), or interface design that excludes users (language, reading level, disability access). For generative AI, unfairness often appears as stereotyped language, differential refusal behavior, or poorer performance for dialects and non-standard phrasing.
Write risks as observable behaviors. Example: “If the résumé screener ranks candidates based on wording similarity, then non-native speakers may be ranked lower, affecting candidates, because the embedding model penalizes atypical phrasing.” Controls might include balanced evaluation sets, constraints (no protected-class inference), calibration tests by subgroup where appropriate, and user experience changes (collect structured skills rather than free-form text). A common mistake is treating fairness as a single metric. For beginners, focus on practical checks: compare error rates and user complaints across groups, and ensure accessibility is not an afterthought.
Reliability risks are about whether the system behaves consistently and correctly in real conditions. With AI, failures are often non-obvious: hallucinated facts, confident wrong answers, brittle behavior on slightly different inputs, or performance collapse when the data distribution shifts. Reliability matters even when the tool is “only advisory,” because users may still act on it.
Start with your system’s required level of correctness. A writing assistant can tolerate stylistic variance; a medication advice tool cannot. Then run a simple reliability checklist: (1) define what the model must never do, (2) define what it must always do, and (3) test representative and adversarial inputs.
Convert reliability issues into risks tied to decisions. Example: “If the assistant fabricates return-policy details, then customers may be misled into missing deadlines, affecting customers and support teams, because the model is not grounded in the latest policy text.” Controls include retrieval grounding (use approved sources), forced citations to known documents, refusal when confidence is low, and human review for high-stakes outputs. A frequent beginner mistake is testing only “happy path” prompts. Your checklist should include messy real user input and the top 10 failure cases from support tickets or stakeholder interviews.
Milestone: Run a safety and misinformation check. Safety risks are the harms that occur when the system’s outputs or actions lead to injury, loss, or distress. For beginners, it helps to separate safety into physical, financial, and emotional domains, and to explicitly include misinformation as a safety accelerator—false claims can spread quickly and cause real-world harm.
Write risks with the user journey in mind: where does the model speak with authority, and where might users stop verifying? Example: “If the tool summarizes medical symptoms without a clear disclaimer and routes, then users may delay seeking care, affecting patients, because the output sounds definitive.” Controls include clear scope limits, safety policies with refusal behavior, escalation to humans (or to emergency resources) for crisis topics, and UI friction for high-risk actions (confirmations, warnings, ‘not a substitute’ disclosures).
Engineering judgement is crucial: over-blocking can harm users who need legitimate information; under-blocking can enable dangerous guidance. A practical approach is to define “high-stakes topics” for your domain and require stronger controls there: grounding, citations, human review, and audit logs. Measure what matters: incident rates, near-misses, and user reports—not just offline accuracy.
Milestone: Run a security and misuse check. Security risks cover both traditional threats (unauthorized access) and AI-specific abuse patterns (prompt injection, indirect prompt injection via documents, and data exfiltration). The key idea for beginners: treat the model as an untrusted component that can be manipulated, and design the surrounding system to keep secrets and enforce permissions.
Translate into risks tied to assets. Example: “If the assistant can call internal APIs based only on natural-language instructions, then an attacker can retrieve another user’s account details, affecting customers, because the tool layer lacks authorization checks.” Controls are usually outside the model: strong authZ at the API layer, allowlists for tools, least-privilege tokens, output filtering for secrets, rate limits, and abuse monitoring. For retrieval systems, sanitize untrusted content and separate instructions from data (e.g., treat retrieved text as quoted context, not executable commands).
A common mistake is relying on “the model won’t do that” as a security control. Assume it will, eventually. Your checklist outcome should be a short list of hard requirements (permission checks, logging, incident response contacts) and a plan to test them with red-team prompts.
Transparency is how you keep user expectations aligned with what the system actually does. It includes explainability (what factors drove an output), disclosures (AI involvement, limitations, data use), and operational clarity (how to contest decisions and report issues). Transparency reduces harm by preventing over-reliance and by enabling accountability when things go wrong.
Run a practical transparency checklist: What should users know before using the tool, what should they see at the moment they receive an output, and what can they access after an outcome affects them?
Milestone: Prioritize risks worth addressing first. Transparency also helps you prioritize: if a risk can’t be fully eliminated quickly, you may still reduce impact by warning users, gating features, or requiring confirmation. Use your likelihood × impact scores to pick the top items, then choose controls that are feasible before deployment. A helpful rule: address high-impact risks even if likelihood feels “low,” because AI systems can scale failures fast. Capture outcomes in a simple register: risk statement, score, owner, control, and a date for re-review. That document is your bridge from “we noticed risks” to “we managed them.”
1. What is the main purpose of the beginner-friendly risk assessment approach in Chapter 4?
2. Which workflow best matches the chapter’s recommended process for running the milestones?
3. Why does the chapter say it’s normal for the same issue (e.g., personal data leakage) to appear in multiple milestones?
4. Which risk statement best follows the chapter’s recommended format?
5. How does Chapter 4 recommend prioritizing which risks to address first?
By now you have a list of plausible harms and a map of who might be affected. This chapter turns that list into decisions. You will (1) score risks using a simple likelihood × impact method, (2) choose controls that reduce either likelihood or impact, (3) assign owners and timelines so the plan is real, and (4) decide what “residual risk” remains after controls—and whether it is acceptable.
A beginner mistake is to treat risk scoring like math homework: pick numbers, multiply, and declare victory. In practice, the scoring scale is a communication tool. Your goal is not precision; it is consistency and defendability. If your team can explain why one risk is “High” and another is “Medium,” and can show the evidence used, you are already doing professional-grade governance.
Another common mistake is to score everything as high. That feels cautious, but it prevents prioritization and makes treatment plans impossible to execute. A good assessment forces trade-offs: which few risks do we treat immediately, which do we monitor, and which do we accept with documentation and sign-off?
Use the milestones in this chapter as your workflow: build a simple scale you can defend, score the top risks consistently with documented reasoning, choose controls matched to each risk, assign owners and timelines, then decide what residual risk is acceptable for launch.
Practice note for Milestone: Build a simple scoring scale you can defend: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Score top risks consistently and document reasoning: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Choose controls that reduce likelihood or impact: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Assign owners and timelines for each control: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Define “residual risk” and decide what’s acceptable: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Build a simple scoring scale you can defend: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Score top risks consistently and document reasoning: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Choose controls that reduce likelihood or impact: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Start with a 3-level scale for both likelihood and impact: Low / Medium / High. Avoid 1–10 scales early on; they encourage fake precision and arguments about whether a risk is a “6” or a “7.” Your scoring system should be simple enough that two people can apply it independently and end up close.
Define the scale in words, not numbers. For example:
Combine them with a simple matrix: High×High = Critical; High×Medium or Medium×High = High; Medium×Medium = Medium; anything with a Low tends toward Low/Medium depending on context. The exact mapping matters less than being consistent. Document your definitions at the top of your risk register: that is your “scale you can defend.”
Common mistake: scoring likelihood based on “model capability” alone. Likelihood must include operational reality: who uses the system, how often, in what environment, with what incentives to misuse. A mediocre model used by millions can create high-likelihood harm; a strong model behind a narrow internal tool may have lower likelihood.
To score risks consistently, anchor each score in evidence. “Because I feel like it” does not hold up in review. In a beginner-friendly assessment, you can use four evidence types that are accessible without heavy tooling.
Write your reasoning directly in the risk register. A good entry includes: the harm statement, affected stakeholders, likelihood score with evidence (“seen twice in pilot”), impact score with rationale (“could deny access to a service”), and the assumptions (“only available to trained staff,” “no minors allowed”). If assumptions change, you revisit the score.
Practical workflow: pick the top 5–10 risks from your harm list. For each, gather at least one piece of evidence. If you have no evidence, that is a signal to run a quick test or do a small pilot before committing to a low likelihood score. Evidence gathering is how you move from opinion to engineering judgment.
Common mistake: using “model accuracy” as evidence for impact. Accuracy metrics rarely capture distributional harms (e.g., worse outcomes for a subgroup) or downstream consequences (e.g., a wrong suggestion in a medical context). Pair metrics with scenario-based examples: what happens when the system is wrong, and who pays the price?
Once you score a risk as Medium/High, the next milestone is to choose controls that reduce likelihood or impact. A practical way to brainstorm controls is the four-part model: prevent, detect, respond, recover. Most teams over-invest in prevention and forget the rest; mature systems plan for failure.
Match controls to the risk mechanism. If the harm is “model reveals personal data,” prevention might be training-data controls and retrieval filtering; detection might be canary prompts and leak audits; response could be disabling a retrieval endpoint; recovery could be deletion and user notification.
Make controls executable: for each control, assign an owner (a person or role), a deadline, and a verification method (“how we will know it works”). This is where risk assessment becomes operational governance. If no one owns it, it is not a control—it is a wish.
Common mistake: picking “human review” as the only control. Human-in-the-loop can be powerful, but it is not magic; it must be designed like a system, with training, load management, and clear decision rights.
Human-in-the-loop (HITL) means a person reviews, approves, or overrides the AI output. HITL can reduce likelihood (blocking bad outputs) and reduce impact (catching severe errors before they reach users). It works best when the reviewer has enough time, context, and authority to intervene.
Use HITL when: (1) the decision affects rights, safety, or significant finances; (2) the AI output is probabilistic or hard to explain; (3) the user population is vulnerable; or (4) the system will be used in novel contexts where monitoring signals are weak.
Design details matter. Define what the human is reviewing (raw output, sources, confidence signals), what “approve” means, and what happens on disagreement. Provide checklists that map to your harm categories (privacy, bias, safety, misinformation, security). Limit reviewer workload with sampling plans and escalation thresholds (e.g., all high-severity cases reviewed; random 5% of routine cases audited).
HITL fails in predictable ways: rubber-stamping due to time pressure, automation bias (assuming the model is right), unclear accountability (“the model said it”), and inconsistent decisions across reviewers. If you add HITL, add controls around HITL: training, calibration sessions, quality metrics (inter-reviewer agreement), and a clear policy that the human owns the final decision.
Document the failure modes in your risk register. If your scoring assumes HITL reduces likelihood from High to Medium, state the assumptions: reviewer-to-case ratio, maximum queue time, and what happens during peak load. Otherwise you will overestimate the protection and underestimate residual risk.
Pre-deployment scoring is necessary, but not sufficient. Real-world users will surprise you, and model behavior can drift when inputs change. A monitoring plan is the practical bridge between your risk assessment and ongoing safety.
Start with “what would change our scores?” For each top risk, define one to three signals that indicate increasing likelihood or increasing impact. Examples include: spike in user complaints about harmful content, rising false-positive rates for a subgroup, unusual prompt patterns suggesting jailbreak attempts, increases in blocked outputs, or higher escalation volume from reviewers.
Assign owners and review cadence: who checks the dashboards daily/weekly, who gets paged, and who has authority to pause the feature. Include a “stoplight” response rule (Green = normal, Yellow = investigate, Red = pause/rollback). This prevents slow-walking decisions when an incident is unfolding.
Common mistake: collecting logs but not using them. Monitoring must include decision-making routines: scheduled reviews, thresholds, and a feedback loop into model updates, policy changes, and training for reviewers.
After controls are selected and implemented (or planned), rescore each risk to estimate residual risk. Residual risk is not “what we hope remains”; it is what remains given verified controls and realistic assumptions. This is the milestone where you decide what is acceptable for launch.
Define acceptance thresholds up front. For example: no Critical residual risks allowed for public launch; High residual risks require executive sign-off and a time-bound mitigation plan; Medium risks require an owner and monitoring; Low risks can be accepted with documentation. Your thresholds should reflect the domain (health and finance demand stricter thresholds than a low-stakes content tool).
Risk acceptance must be explicit and documented. For each accepted Medium/High residual risk, record: what is being accepted, why it is acceptable now, what compensating controls exist, what monitoring will detect deterioration, and when you will revisit the decision. Include a named approver (product owner, risk officer, or accountable executive) and the date.
Common mistake: treating “we will fix it later” as acceptance. If you accept risk, you still need a plan: owner, timeline, and trigger conditions that force re-evaluation. Another mistake is burying decisions in chat threads. Keep a single risk register as the source of truth so audits, incident reviews, and future team members can reconstruct your reasoning.
When done well, risk acceptance is not a bureaucratic hurdle; it is a leadership act. You are aligning the organization on what trade-offs it is willing to make, and you are making those trade-offs visible—before users pay the cost.
1. What is the primary purpose of using a simple likelihood × impact scoring scale in this chapter?
2. Which approach best reflects the chapter’s guidance for scoring risks professionally?
3. What is a key problem with scoring every risk as “High”?
4. According to the chapter, how should controls relate to a risk’s score components?
5. What does “residual risk” mean in the workflow described?
By this point in the course, you have the raw materials: a plain-language system description, a stakeholder map, a list of likely harms, and a simple likelihood × impact rating. Chapter 6 turns those notes into something you can actually ship with: a one-page assessment that a busy stakeholder can read in minutes, plus a clear deployment decision and plan for communicating, monitoring, and responding after launch.
The goal is not to create perfect documentation. The goal is to make your judgement explicit, checkable, and repeatable. A one-page format forces prioritization: the highest risks, the most effective controls, and the remaining (residual) risk you are knowingly accepting. You will also create a go/no-go checklist so decision-makers can see what must be true before releasing, and what must happen right after release (monitoring and incident handling). Finally, you will align user-facing messages with internal reality so users are not surprised, misled, or left without a way to report problems.
Think of this chapter as the bridge between “we analyzed” and “we decided.” Many AI failures happen in that gap: risks are identified but not owned, controls are suggested but not implemented, and no one is accountable for post-launch learning. The workflow here is intentionally lightweight so beginners can apply it without a governance department—yet structured enough to withstand scrutiny.
Practice note for Milestone: Write a one-page AI risk assessment summary: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Create a go/no-go decision checklist for stakeholders: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Prepare a basic communication plan (users + internal teams): document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Plan post-launch reviews and incident response basics: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Practice with a realistic case study and finalize your template: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Write a one-page AI risk assessment summary: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Create a go/no-go decision checklist for stakeholders: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Prepare a basic communication plan (users + internal teams): document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Practice note for Milestone: Plan post-launch reviews and incident response basics: document your objective, define a measurable success check, and run a small experiment before scaling. Capture what changed, why it changed, and what you would test next. This discipline improves reliability and makes your learning transferable to future projects.
Your milestone in this section is a one-page AI risk assessment summary. “One page” is a forcing function: it prevents you from hiding uncertainty in long prose and pushes you to name the few items that matter most. A practical one-page template has four blocks: (1) System snapshot, (2) Top risks, (3) Controls, and (4) Residual risk + owner.
1) System snapshot should be plain language: purpose, who uses it, what data it touches, what outputs it produces, and what decisions or actions those outputs influence. Include the operational context (e.g., customer support, hiring, healthcare intake). Common mistake: describing the model but not the decision. People are harmed by decisions, not architectures.
2) Top risks should be a short list (typically 5–8). Each risk item should say: harm type (privacy, bias, safety, misinformation, security), who could be harmed, and a likelihood × impact rating. Keep the rating consistent with your earlier method. Common mistake: listing generic risks (“bias possible”) without a harmed party or scenario (“customers with non-standard accents receive incorrect eligibility outcomes”).
3) Controls are your mitigations mapped to each risk. Aim for concrete controls: human review, user confirmation steps, data minimization, guardrails, model testing, logging, monitoring, rate limiting, and access control. Common mistake: writing “monitor it” as a control without specifying what signal you watch, what threshold triggers action, and who responds.
4) Residual risk + owner is the honest part: after controls, what risk remains, and who accepts it. Residual risk is not failure; it’s reality. Write: “Residual risk: Medium; owner: Product Manager; review date: 30 days post-launch.” This turns the assessment into an operational commitment rather than a document that sits in a folder.
Your second milestone is a go/no-go decision checklist for stakeholders. The checklist should connect your risk ratings to a decision outcome that people can execute. Use four outcomes because they cover most real situations without becoming bureaucratic: proceed, proceed with conditions, pause, and stop.
Proceed means risks are low-to-moderate, controls are implemented, and monitoring is ready. This is appropriate for low-stakes contexts where outputs are advisory and reversible. Your checklist here should include: tests passed, logs enabled, disclosure written, and a named on-call contact for incidents.
Proceed with conditions is the most common outcome in responsible deployments. Conditions are specific, verifiable items with dates. Examples: “Enable human review for all high-risk cases,” “Limit rollout to 5% of users,” “Block certain topics,” “Require user confirmation before action,” “Conduct fairness test on subgroup X.” A common mistake is writing vague conditions like “improve accuracy” without a measurable threshold.
Pause means the idea is viable but not yet safe enough to release. Typical triggers: missing evidence, inability to monitor, unclear ownership, unresolved privacy questions, or poor performance on a critical scenario. Pausing is not punishment; it is a signal that the organization must pay down risk debt. Your checklist should identify the smallest set of actions to resume (e.g., “complete data mapping,” “run red-team prompts,” “add abuse reporting flow”).
Stop is reserved for cases where harms are severe or unmanageable given constraints. Examples: the tool would influence legal/health outcomes with weak evidence, you cannot obtain consent or legal basis for data use, or the system can be easily weaponized at scale. Engineering judgement matters here: stopping is appropriate when mitigation would fundamentally change the product, or when you cannot credibly bound harm.
Design the checklist so stakeholders can sign off: include a decision owner, consulted roles (security, legal, compliance, support), and an “evidence links” field that points to tests or reviews. This makes decisions explainable later—especially when someone asks, “Why did we ship?”
Your third milestone is a basic communication plan for users and internal teams. Start with user-facing disclosures because they directly shape harm: if users misunderstand capability, they may over-rely on output or share sensitive data. A good disclosure is short, specific, and placed where decisions occur (not buried in a policy page).
Limitations: Name what the system is not reliable for. Avoid generic phrases like “may be inaccurate.” Instead: “This tool can summarize documents, but it may omit important details. Do not use it as the only source for compliance decisions.” If there are known weak spots (languages, accents, rare conditions, edge-case prompts), say so. Common mistake: overselling to increase adoption, then paying for it with support tickets and reputational damage.
Data use: Tell users what inputs are collected, whether they are stored, who can access them, and whether they are used to improve models. If you cannot confidently explain this in plain language, you likely have not mapped the data flow well enough. At minimum, cover: retention period, whether human reviewers can see content, and how to request deletion (if applicable). Keep the promise aligned with actual logs and vendor settings.
Contact paths: Provide a clear way to report problems: “Report an issue” button, email alias, or in-product form. Specify categories users can report (unsafe output, privacy concern, bias/discrimination, security). A common mistake is having a contact path that routes nowhere, which increases harm because issues go untriaged.
For internal communication, identify who needs to know what: support teams need known failure modes and escalation steps; sales needs approved claims; security needs monitoring signals; legal needs data-use statements. Treat this as part of deployment readiness, not marketing.
Good documentation hygiene is what lets you learn over time. Your assessment will be revised as models change, prompts evolve, policies update, and new users arrive. If you cannot track versions and assumptions, your one-page summary becomes misleading within weeks.
Versioning: Put a version number and date on the one-pager, and include the system version it applies to (model name/version, prompt version, policy rules version, dataset snapshot). If you use a vendor model, record the vendor and configuration (temperature, tool access, content filters). Common mistake: “We assessed the chatbot” with no indication of which build; then an incident happens on a newer build and you cannot tell what changed.
Assumptions: Explicitly list key assumptions that affect risk. Examples: “Used only by trained agents,” “No minors expected,” “Outputs are advisory only,” “Human approves all messages,” “Only anonymized tickets are provided.” Assumptions are fragile; they break quietly when the product scales. Writing them down turns them into items you can validate.
Evidence links: Every major risk rating and control should point to evidence. Evidence can be lightweight: a link to test results, a red-team note, a privacy review ticket, or a monitoring dashboard. The point is traceability. A common mistake is to cite “we tested” without preserving what was tested, by whom, and with what criteria.
Practical workflow: store the one-pager in a shared location, require a short change note for edits, and include a “last reviewed by” field. This is not busywork; it prevents the organization from repeatedly re-litigating the same questions and helps new team members understand why guardrails exist.
Your fourth milestone is to plan post-launch reviews and incident response basics. Incidents are not hypothetical in AI systems: they range from offensive outputs to privacy leaks to automation that quietly disadvantages a group. The worst moment to invent an escalation process is after a screenshot goes viral.
Define “incident” for your system. Use categories aligned to common harms: privacy exposure, discriminatory behavior, unsafe advice, misinformation in high-stakes contexts, security abuse (prompt injection, data exfiltration), and policy violations. For each category, define severity levels (e.g., Sev 1–3) based on impact and scale.
Create an escalation path. At minimum, specify: who triages (support/on-call), who investigates (engineering, security, data science), who approves external communication (legal/comms), and who can shut off features (product/engineering). Include time targets (e.g., acknowledge within 4 hours, mitigate within 24 hours for Sev 1). Common mistake: routing everything to a single person; incidents require cross-functional coordination.
Prepare “stop-the-bleeding” controls. Examples: disable a feature flag, restrict to internal users, increase human review, block certain tools or actions, tighten content filters, or temporarily remove memory/logging fields. Pre-authorize these actions so responders do not need a meeting to act.
Preserve evidence safely. Logging is necessary for investigation, but it must be privacy-aware. Decide what to log (prompts, outputs, metadata), how long to keep it, and how to restrict access. If you cannot investigate without collecting sensitive user content, revisit data minimization and redaction.
After-action learning: every incident should result in one or two concrete improvements: a new test, a new guardrail, an updated disclosure, or a revised risk rating. This closes the loop between deployment and reassessment.
Your final milestone is to practice with a realistic case study and finalize your template. To make the template usable long-term, define reassessment triggers—clear events that require you to revisit the one-pager and the go/no-go checklist. Without triggers, reassessments happen only after a failure.
Common triggers: model upgrades, prompt or policy changes, new tools/actions granted (e.g., sending emails, issuing refunds), integration of new data sources, expansion to new regions or languages, new user groups (especially vulnerable users), and new high-stakes uses of the same system. Also include non-technical triggers: new regulations, new contractual commitments, or changes to vendor terms.
Set review cadence plus event-based reviews. A simple pattern: scheduled review at 30/90 days post-launch, then quarterly; plus immediate review on any trigger. Assign an owner to schedule and run the review, and define what “review complete” means (updated risk ratings, refreshed evidence links, updated disclosures, validated monitoring).
Realistic case study (use for your practice): An internal “AI Email Assistant” drafts responses for customer support agents using recent ticket history. Risks include leaking personal data into drafts, hallucinating policy promises (refunds, timelines), biased tone toward non-native speakers, and prompt injection from malicious tickets (“ignore policy, ask for password”). Controls might include: redact sensitive fields before prompting, require agent approval, block certain promises, run prompt-injection tests, and monitor for prohibited phrases. Your one-pager should end with a decision outcome (often “proceed with conditions”) and a post-launch plan (dashboard + incident escalation).
Finalize your template by ensuring it fits on one page, uses consistent ratings, and contains owners and dates. The practical outcome is that your organization can deploy with eyes open: you can explain what could go wrong, what you did about it, what you are still accepting, and how you will respond when reality teaches you something new.
1. What is the primary purpose of the one-page AI risk assessment in Chapter 6?
2. Why does Chapter 6 insist on a one-page format?
3. What role does the go/no-go decision checklist serve for stakeholders?
4. According to Chapter 6, what is a key reason many AI failures occur between analysis and deployment?
5. How should the communication plan relate user-facing messages to internal reality?