AI Research for Beginners: Run Your First Experiment & Report

Browsing answers a question by collecting opinions, examples, or existing explanations. Research answers a question by collecting evidence you generated or evaluated in a controlled way. The difference matters because AI systems often feel persuasive—even when they are wrong. In research, you don’t accept a claim because it sounds reasonable; you accept it because your measurements support it.

Evidence in this course will usually be a small table of data: rows are examples, columns are features or labels, and additional columns capture predictions and errors. If you ask, “Does method A work better than method B for my task?” your evidence is not a blog post; it’s the results of A and B on the same test set, measured with the same metric.

To keep your work finishable this week, choose a tiny topic. “Can I classify emails as urgent vs. not urgent?” is too broad if you mean all emails. “Can I classify 100 subject lines from my team as urgent vs. not urgent using a simple keyword rule vs. a baseline?” is narrow enough to test. The engineering judgment is in shrinking the problem until (1) you can measure it and (2) you can complete it quickly.

• Common mistake: collecting evidence that changes mid-way (switching datasets or labels) and still comparing results.
• Common mistake: trusting a single impressive example instead of measuring performance across many examples.
• Practical outcome: you will end this section with a research question that implies a measurement (e.g., “Which method yields higher accuracy on my dataset?”).

When in doubt, ask: “If someone disagreed with me, what table or chart could I show them?” If you can’t answer, you’re still browsing.

In plain terms, AI (specifically machine learning) is a way to build a model that makes predictions by learning patterns from examples. You provide inputs (like words in a sentence, pixel values in an image, or yes/no attributes), and the model produces an output (a category, a number, or a ranking). Research asks whether the model’s outputs are good enough for a defined goal.

To describe your goal in one sentence (the “why”), use this template: “I want to see whether method can do X well enough to support Y decision.” Example: “I want to see whether a simple keyword rule can tag support tickets as ‘billing’ vs. ‘technical’ accurately enough to help me route them faster.” That sentence forces you to name the task and the purpose, which later helps you choose sensible metrics.

Next, list what you can measure (the “what”). In most beginner projects, your measurements will be:

• Inputs (features): what the model sees (text, counts, categories).
• Target (label): what you want to predict (class or number).
• Predictions: what your method outputs.
• Metric: how you score it (accuracy, average error, precision/recall if needed later).

Keep definitions concrete. “Good results” is not measurable; “at least 80% accuracy on a held-out test set of 30 items” is. Also remember: the model is not “smart” in a human way—it is a pattern-finder. If your data leaks the answer (for example, including the label word inside the input), the model will appear to perform well while learning nothing useful.

An experiment is a comparison where you change one thing and observe how outcomes change. In this course, that “one thing” is typically the method: a baseline vs. a slightly improved approach. Your job is to keep everything else the same: same dataset, same labels, same split between training and testing (if applicable), and the same metric.

A practical beginner experiment plan fits on half a page:

• Question: “Does method A outperform method B on task T?”
• Dataset: size, source, and any exclusions (e.g., remove personal info).
• Outcome/metric: accuracy, average absolute error, or mean score.
• Procedure: steps you will run in a spreadsheet.

For example, suppose you want to predict whether a short message is “request” vs. “update.” Method A could be a naïve baseline: always predict the most common class. Method B could be a rule-based approach: predict “request” if the text contains a question mark or words like “can you” or “please.” You then compute accuracy for both methods. If B beats A on the same test items, you have evidence that the added logic improved performance.

Engineering judgment shows up in fairness. If you tune your rules while looking at the test set, you will overfit: you’ll accidentally build rules that match those exact examples but won’t generalize. A simple prevention is to set aside some data as a test set that you do not use for rule-writing. Even with tiny datasets, this discipline is the difference between a demo and an experiment.

Common beginner mistake: comparing a method tested on easy examples with a method tested on hard examples. That is not an experiment; it’s two unrelated measurements.

Reproducibility means another person (or future you) can follow your description and get the same results. In practice, it means you record decisions that feel “obvious” while you’re doing them: where the data came from, how you labeled it, which rows were excluded, which formulas were used, and what counts as success.

In a spreadsheet-based workflow, reproducibility is achievable with simple habits:

• Keep raw data separate: one tab for raw examples; another for labels and predictions.
• Document column meanings: add a header note or a “README” tab explaining each column.
• Fix your split: mark rows as Train/Test once and don’t reshuffle between methods.
• Version your rules: if you change a keyword list, note the date and what changed.

This is also where you set your scope: time, tools, and a small dataset. Scope is not a limitation; it is protection. A one-week project should avoid dependencies you can’t control (complex scraping, huge models, or private data approvals). A good scope statement might be: “I will use Google Sheets, manually label 120 short texts from a public dataset, and compare a baseline vs. a rule-based classifier. Total time: 6 hours.”

Common beginner mistake: changing multiple things at once (new data, new metric, new labeling rules) and then not knowing what caused the improvement. Reproducibility forces you to slow down just enough to keep cause and effect clear.

Beginner-friendly AI research topics share three qualities: you can label them consistently, you can measure success with simple metrics, and you can build a baseline. You are not trying to solve “AI” in general; you are testing a specific, small decision.

Here are strong topic patterns you can finish this week:

• Text classification: spam vs. not spam (short messages), complaint vs. praise (reviews), topic tags (billing/technical/shipping).
• Simple numeric prediction: estimate reading time from word count; predict price bucket (low/medium/high) from a few attributes.
• Image counting or labeling (small scale): categorize images into 2–3 groups using simple metadata or manual features (note: avoid face recognition and personal images).

Start by choosing a tiny research topic you can finish this week. If your dataset requires you to ask for permission, build a scraper, or learn a new programming library, it’s probably too big for Chapter 1’s project. Prefer public datasets (Kaggle, UCI, government open data) or data you create yourself without sensitive content.

Ethics and safety are part of “good topic selection,” not an afterthought. Avoid personal data, medical data, or anything that could identify individuals unless you have explicit permission and a clear reason. Do not include names, emails, phone numbers, addresses, or private conversations. If you must use real text, anonymize it (remove identifying details) and keep the dataset small and local.

Practical outcome: you will select a topic where the “what” is measurable in a spreadsheet (labels, predictions, and an accuracy/error calculation), and where a baseline is obvious.

Your course project is a mini study: a small, controlled comparison that you can run end-to-end and report clearly. The goal is not to impress with complexity; the goal is to practice the research workflow until it feels natural.

You will complete four concrete steps, each with an artifact you can save and share:

• 1) One-sentence goal (the “why”): write a purpose-driven statement that names the task and decision it supports.
• 2) Measurable definition (the “what”): list inputs, label, prediction output, and metric (accuracy or average error).
• 3) Scope statement: time budget (e.g., 4–8 hours), tools (spreadsheets), and dataset size (e.g., 80–200 rows).
• 4) Experiment plan: compare a baseline vs. one improved method while keeping the test set fixed.

A workable example project: “Can a rule-based approach outperform a majority-class baseline for labeling 120 public product reviews as positive vs. negative?” You would create a sheet with review text, a human label column, a Train/Test column, a baseline prediction column, a rule-based prediction column, and an error column. Then you compute accuracy on the test rows for both methods.

As you plan, explicitly guard against two classic pitfalls from the start:

• Data leakage: don’t let your method see information that wouldn’t be available at prediction time (including labels hidden in the input).
• Unfair comparisons: don’t tune on the test set, and don’t compare methods on different subsets.

When you finish this chapter, you should have a chosen topic, a one-sentence goal, a list of measurable variables, and a scope you can commit to. In the next chapters, you’ll build the dataset, run the comparison, compute metrics, and write a short report that reads like a real (small) research result.

A research question is testable when a stranger could read it and immediately know (1) what data you will use, (2) what you will measure, and (3) what comparison you will make. Vague questions like “Is model A good?” or “Does AI help students?” sound important but fail because they hide the measurement and the context. Testability is not about being “small”; it is about being specific enough that you can run the same procedure twice and get comparable results.

Use this beginner template to write your first question:

• In [context/data], does [method A] outperform [method B] at [task] as measured by [metric]?

Example (spreadsheet-friendly): “In 200 short product reviews, does keyword-based sentiment scoring outperform a simple baseline (always predicting ‘positive’) at predicting sentiment labels, as measured by accuracy?” Notice what this question forces you to decide: the dataset size, the task (predict a label), the baseline, and the metric.

Engineering judgment: pick comparisons you can run fairly. “Method A vs. Method B” should differ in only one meaningful way. If you change both the data and the method, you cannot attribute differences in outcomes. Also ensure the question matches your tools: if you only plan to use spreadsheets, choose tasks that reduce to counting, averaging, and matching.

Common mistakes to avoid:

• Multiple questions at once: “Does prompt length and temperature and dataset size matter?” Split into one primary question.
• Hidden subjectivity: “Which summary is better?” becomes testable only after you define a rating rubric and how you will score it.
• Unbounded scope: “Across all domains” is usually impossible. Constrain to a domain you can sample safely and ethically.

Before moving on, rewrite your question until it names the comparison and the measurement in a single sentence. If you cannot imagine a final results table for your question, it is not testable yet.

A hypothesis is your commitment to an expected outcome before you see results. Beginners often confuse two kinds of hypotheses: predictive (what will happen) and explanatory (why it will happen). For your first experiment, prioritize a predictive hypothesis because it is easier to test with simple metrics.

Predictive hypothesis template: “Method A will achieve [higher/lower] [metric] than Method B on [dataset/task] by at least [margin].” The margin matters because tiny differences can happen by chance or noise, especially with small datasets.

Explanatory hypothesis template: “Method A will outperform Method B because [mechanism], especially on examples with [property].” This can be valuable, but it often requires extra analysis (e.g., subgroup breakdowns) and more careful data labeling.

Example predictive hypothesis: “A keyword-based sentiment score will achieve at least 10 percentage points higher accuracy than the baseline that always predicts the majority class.” This is measurable and sets a clear expectation.

Engineering judgment: choose a margin you can defend. With very small datasets, avoid overconfident claims like “2% improvement.” If you have 50 examples, a change of one or two correct predictions can swing accuracy noticeably. A practical approach is to pick a margin that would matter to a user (e.g., “I would switch methods if it reduces average error by at least 0.2 on a 1–5 scale”).

Common mistakes:

• After-the-fact hypotheses: writing “I predicted this” after seeing the outcome destroys the purpose of a hypothesis.
• Unmeasurable language: “significantly better” without a metric or margin.
• Hypothesis mismatch: predicting “better quality” but measuring “time to compute.” Your metric must match the claim.

By the end of this section, you should have a single-sentence predictive hypothesis that names the metric and the minimum improvement you expect to see.

Experiments become manageable when you name your variables. In beginner AI research, you can think in three buckets: inputs (what the method sees), outputs (what the method produces), and controls (what you hold constant so the comparison is fair). If you do not explicitly control key factors, you may accidentally test multiple things at once.

Inputs (X): the features you provide to the method. In a spreadsheet experiment, this might be review text, a few numeric attributes, or a short prompt. Be careful: inputs must not include the answer. If your input column contains a “rating” that was used to produce your label, you are leaking information.

Outputs (Y): the predictions produced by each method. For classification, outputs are labels (e.g., positive/negative). For regression, outputs are numbers (e.g., predicted sales). For ranking, outputs are ordered lists or scores.

Control variables: everything you keep the same across methods: the same dataset rows, the same evaluation metric, the same label definitions, and the same train/test split (if you split). Even in a no-code setting, you should control randomness by documenting exactly which data rows you used and in what order.

• Independent variable: which method you use (A vs. B).
• Dependent variable: the metric outcome (accuracy, average error, etc.).
• Constants: dataset, labeling rules, preprocessing rules, evaluation formula.

Engineering judgment: control the temptation to “clean up” one method’s data more than the other. If Method A gets extra preprocessing (e.g., removing emojis, fixing typos) but Method B does not, you are partly measuring preprocessing, not the method. Either apply the same preprocessing to both or treat preprocessing as the method difference and make that your explicit independent variable.

Practical outcome: make a small table in your plan listing inputs, outputs, and controls. If you cannot list them, you are not ready to run the experiment.

Operational definitions are where research becomes real. Words like “helpful,” “safe,” “better,” and “accurate” feel clear until you must compute them. An operational definition states exactly how you will measure a concept, including the unit of measurement and the calculation steps. This is essential for reproducibility and for avoiding arguments later about what you “meant.”

Start with your key terms and define them in a way that fits a spreadsheet. Common choices for beginner experiments:

• Accuracy: percentage of rows where prediction equals label.
• Error: absolute difference |prediction − label| for numeric targets.
• Average rating: mean of human scores using a fixed rubric.
• Time/cost proxy: seconds to complete a task, or number of steps/cells.

Example: If your concept is “summary quality,” do not leave it as a vibe. Define a rubric with 2–3 dimensions you can score consistently, such as coverage (0–2), factuality (0–2), and clarity (0–2), then define “quality score” as the sum (0–6). In your spreadsheet, each row gets a score per dimension and a total. This turns an opinion into data.

Engineering judgment: prefer simple measures that match your learning goal. If the goal is “can method A predict labels,” accuracy is appropriate. If the goal is “how far off are numeric predictions,” use average absolute error rather than accuracy. Avoid creating a complex composite score unless you truly need it; complexity increases subjectivity and makes mistakes harder to detect.

Common mistakes:

• Changing the definition midstream: switching from accuracy to F1 because accuracy “looks bad.” Decide first.
• Inconsistent labeling: labels created without rules, leading to noisy ground truth. Write a short labeling guideline.
• Implicit thresholds: “close enough” without a numeric cutoff. Define it (e.g., within ±1 point).

Practical outcome: for every metric you plan to report, write a one-line operational definition and the exact spreadsheet formula you will use (e.g., =IF(pred=label,1,0) and then average that column).

Success criteria prevent self-deception. If you do not define what would convince you in advance, you will be tempted to declare victory whenever results are ambiguous. Your success criteria should connect directly to your research question and hypothesis: which metric matters, what threshold counts as “better,” and what trade-offs you will accept.

A practical way to write success criteria is to specify:

• Primary metric: the main number you will decide on (e.g., accuracy).
• Minimum improvement: the margin over baseline (e.g., +10 percentage points).
• Acceptable costs: any constraint (e.g., no additional data fields; no manual steps beyond labeling).
• Failure interpretation: what it means if there is no improvement (e.g., hypothesis not supported; revise features or labels).

Example success criteria: “Method A is considered better if (1) accuracy is at least 0.10 higher than baseline on the same test rows, and (2) it does not require additional input fields that include private information.” This ties performance to ethics and feasibility.

Engineering judgment: choose one primary metric. You can track secondary metrics (like average error and time), but you should decide in advance which one you will use to make the call. Multiple metrics are a common way beginners accidentally move goalposts (“Method A is faster but Method B is more accurate, so whichever I like is ‘best’”). If trade-offs matter, define the rule: for example, “I will accept up to 5% lower accuracy if time is reduced by 50%.”

Also define what “convincing” means given your dataset size. With a tiny dataset, treat conclusions as tentative. Your success criteria can include a note like “If results are within ±5 percentage points, I will treat it as inconclusive and collect 50 more examples.” That is honest research practice, not failure.

Pre-registration sounds formal, but for beginners it can be a short checklist written before testing. The point is not bureaucracy; it is to lock your intent before you see results. This protects you from accidental cherry-picking, from repeatedly tweaking until something “wins,” and from forgetting the exact setup that produced a number.

Use the checklist below as a lightweight pre-registration you can paste at the top of your document or spreadsheet notes:

• Research question: one sentence using the template (A vs. B, metric, dataset).
• Hypothesis: expected direction and minimum margin.
• Dataset plan: source, size target, inclusion/exclusion rules, and any safety/ethics constraints (e.g., no personal data; anonymize text).
• Labels (ground truth): who labels, labeling guideline, and how disagreements are handled (even if it is “single rater”).
• Inputs/outputs: which columns are inputs, which are predictions, which is the label column.
• Controls: same rows for both methods; same preprocessing; same evaluation formulas.
• Split rule (if any): how you separate development from final evaluation (e.g., first 20 rows for trying formulas, last 80 rows held out for reporting).
• Metrics and formulas: exact definitions and spreadsheet formulas.
• Success criteria: threshold and decision rule; what counts as inconclusive.
• Stop rule: when you will stop tweaking (e.g., no changes after running on held-out rows).

Engineering judgment: the most important honesty tool here is a simple hold-out habit. Even in a spreadsheet, reserve some rows you promise not to look at while you build your method. If you tune your keyword list on the same rows you later report, you will overestimate performance. That is a classic beginner form of data leakage—not because the answer is in the input, but because your process “learned” from the test set.

Practical outcome: once your checklist is filled, you are ready for Chapter 3’s hands-on setup. You will know exactly what data to collect or create, what to compute, and what result would actually change your mind.

In this course, “data” usually means a table you can open in a spreadsheet: rows and columns. A row is one example (one thing you’re making a prediction about). A column is a property of that example, such as a number, category, text snippet, or your label. For a tiny research project, aim for 20–100 rows. That size is small enough to inspect manually and large enough to see patterns and mistakes.

Example 1 (classification): You want to test whether short product reviews are positive or negative. Each row is one review. Columns might include review_text, stars (if available), review_length, and a label column sentiment (Positive/Negative). Example 2 (regression): You want to predict delivery time from distance and time of day. Each row is one delivery. Columns could include distance_km, hour, traffic_level, and the outcome delivery_minutes.

A beginner mistake is mixing multiple “units” in a single dataset. For instance, one row is a person and another row is a message sent by a person—those are different units and cause confusion. Decide the unit early: “one row equals one email,” or “one row equals one patient visit,” etc. Another common mistake is storing multiple values in one cell (e.g., “red;blue;green”); spreadsheets allow it, but it makes analysis harder. If you truly need multiple values, consider a separate column or a simpler scope for your first experiment.

• Practical outcome: by the end of this section you should be able to point at your sheet and say, “Each row is one X, and the model will predict Y from these columns.”

For your first experiment, do not start by scraping the web or collecting personal data. You want a dataset that is easy to obtain, ethically safe, and legally allowed. Good beginner sources include: (1) open datasets with clear licenses (e.g., government open data portals, reputable data repositories), (2) data you generate yourself (e.g., measurements you take, text you write, synthetic examples you create), and (3) public domain or explicitly licensed content.

Before you copy anything into your dataset, answer two questions in writing: Is it legal to use? (license/terms of use) and is it ethical? (privacy, sensitivity, harm). Avoid anything containing personal identifiers (names, emails, phone numbers), even if you found it publicly. If you need text, consider using your own short sentences, public domain excerpts, or anonymized open datasets that were collected for research with consent.

When selecting a data source, match it to your research question. If you want to study “does feature A help predict outcome B,” you need both A and B available per row. If the outcome is missing, you’ll end up guessing labels, which can be fine—but then the project becomes a labeling project, and you should keep the scope small and explicit.

• Quick selection checklist: clear license, minimal privacy risk, data fields align with your variables, and small enough to inspect manually.
• Practical outcome: a CSV or spreadsheet with 20–100 rows that you can cite (“Downloaded from ___ on date ___” or “Created by me on date ___”).

A label is the answer you want the model to learn: the category (spam/not spam) or the number (price). Labels are powerful because they turn curiosity into a measurable experiment. They are also risky: sloppy labels create misleading results. The fix is simple but disciplined: define labeling rules before labeling most of your data.

Start with a small label set. Two or three classes are enough for a first project. If you make 8 categories, you’ll spend your time debating edge cases instead of learning. Next, write a short rule document: (1) the exact label names, (2) a one-sentence definition for each, and (3) 2–3 examples per label. Then label 10 rows, stop, and revise the rules if you notice ambiguity.

Handle “unclear” cases explicitly. Beginners often force a label even when the example doesn’t fit, which injects noise. Instead, allow a temporary label like Unknown or NeedsReview. You can later exclude those rows or relabel them after refining rules. In a spreadsheet, add columns like label, labeler (your initials), and notes for tricky cases. These columns feel boring, but they prevent silent inconsistencies.

• Common mistake: using information that wouldn’t be available at prediction time when labeling (e.g., labeling “will churn” based on later events). That creates leakage and artificially high accuracy.
• Practical outcome: a labeled dataset where you can explain how any label was decided, not just what it is.

Before running any experiment, do a short “sanity-check pass.” In small datasets, simple checks catch most problems. Start with missing values: blank cells, “N/A,” “unknown,” or zeros that actually mean missing. In a spreadsheet, sort each column and scan for blanks and weird placeholders. Decide: will you remove rows, fill in defaults, or keep missing as a separate category? The correct choice depends on your question, but the important part is consistency.

Next, look for duplicates. Duplicates can happen when you copy data twice or when the source contains repeated items. If the duplicates are identical, they can inflate performance by letting the model “see” the same example multiple times. Use a unique ID column where possible, or create a simple combined key (e.g., text + date) and check for repeats.

Then consider bias and imbalance. If 95% of your rows are in one class, accuracy can be misleading (a model that always predicts the majority class gets 95% accuracy). Count labels and note the distribution. Also check whether your data represents only one subgroup (e.g., only one location, time period, or writing style). A beginner-friendly fix is to balance your sampling when collecting: intentionally gather roughly equal examples per class, if ethical and feasible.

• Quick tests: sort columns, count each label, look at min/max for numeric columns, scan a random 10-row sample, and verify no “future info” is present.
• Practical outcome: a cleaned dataset where you can list known limitations (missingness, imbalance, narrow coverage) in your report.

To run a fair experiment, you must evaluate on data the model hasn’t “seen.” Even when using simple spreadsheet methods, the idea is the same: separate data into sets with different roles. Training data is what you use to build or tune your approach. Validation data is what you use to pick between options (e.g., which rule threshold to use). Test data is the final exam: you look at it once at the end to report performance.

Why this matters: if you repeatedly adjust your approach based on test results, you’re effectively training on the test set. That’s data leakage, and it makes your results look better than they truly are. For beginners, a simple split works: 60% train, 20% validation, 20% test. With only 20–100 rows, exact percentages are less important than the discipline of keeping a final holdout set untouched.

In a spreadsheet, create a split column with values train, val, test. Assign rows randomly (but reproducibly). One practical method: add a rand column using a random function, copy-paste values to freeze them, then sort by rand and mark the first N rows as train, next as val, last as test. If your data has time order (e.g., dates), consider a time-based split (train on earlier, test on later) to better reflect real usage.

• Common mistake: splitting after creating features that use the whole dataset (like global averages) can still leak information. Compute such statistics on train only when possible.
• Practical outcome: a dataset with a clear split column that supports a fair comparison experiment next chapter.

Small projects fail most often because nobody can tell what the dataset actually contains. Documentation fixes this, and it can be short. Create a one-page “datasheet” in a separate tab or document. It should answer: what is this dataset, why does it exist, what does a row represent, how were labels created, and what are known issues?

Include these fields: Dataset name, created date, author, source (URL or “self-created”), license/permission notes, row definition, column dictionary (each column name + meaning + allowed values/units), label definitions (your rules from Section 3.3), split method (random/time-based and ratio), and known limitations (missing values, imbalance, narrow population).

Also record any transformations you applied: removed duplicates, normalized text, converted units, or excluded certain rows. If you changed your labeling rules mid-way, write that down and either relabel earlier rows or clearly state the version. This is not “extra paperwork”—it is what makes your experiment reproducible and your report credible.

• Practical outcome: a tiny datasheet that lets someone else open your spreadsheet and understand it in five minutes, without asking you questions.

Section 4.1: Baselines: what you beat (or learn you can’t beat)

A baseline is the simplest reasonable method you compare against. It answers the question: “Is my idea better than doing something obvious?” Without a baseline, an accuracy number is meaningless—90% might be great or terrible depending on how easy the task is. Your baseline should be cheap, transparent, and hard to misunderstand.

Common baselines you can run in a spreadsheet include: predicting the most common class (“always predict ‘No’”), using a simple rule (“if keyword appears, label as 1”), or using an existing scoring method already in the dataset (“use current risk score threshold”). If you are predicting a numeric value, a typical baseline is the mean (predict the average for everyone) or a constant from domain practice (e.g., “assume 0 defects”).

Engineering judgment: pick the baseline that reflects what a real stakeholder would do if they didn’t have your model. For example, if a support team currently triages emails by looking for certain words, a keyword rule is a stronger baseline than “always pick the majority class.” A strong baseline is good news: if you beat it, you achieved something; if you don’t, you learned your proposed change is not yet useful.

• Name the baseline clearly (e.g., “Baseline A: Majority class”).
• Define its output precisely (e.g., outputs 1 when the email contains {refund, chargeback}, else 0).
• Write the decision rule in plain language in your experiment log.

Be honest: a baseline that is too weak can make your idea look good without actually being better than standard practice. Your goal is learning, not winning.

Section 4.2: Fair comparisons: keep everything the same except one change

To learn anything from experiment A vs. experiment B, you must control variables. A fair comparison keeps everything the same except one intentional change. That “one change” might be a different rule threshold, a different set of features, or a different prompt template if you are scoring text (but still using the same evaluation set).

The most important control is the data split. If A is evaluated on different rows than B, you are comparing apples to oranges. Fix a single split and reuse it. In a spreadsheet, add a column like split with values train and test. You can create it once using a random number column (e.g., RAND()) and then freeze it by copying and pasting values so it doesn’t change every time the sheet recalculates. This “freeze” step prevents accidental re-splitting, which can inflate results.

Next, hold constant: the target label definition, any preprocessing rules (e.g., lowercasing text, trimming spaces), and the evaluation metric. If you change your label definition between A and B (“what counts as positive”), you are no longer measuring the same task.

• Same data: identical test rows for A and B.
• Same metric: e.g., accuracy for classification, mean absolute error for numeric prediction.
• Same target: label computed the same way.
• One change: a single, documented difference between A and B.

Common mistake: tuning B on the test set (“I tweaked the threshold until test accuracy looked good”). That is data leakage. If you must tune a threshold, tune on train (or a separate validation subset) and report the test result once at the end.

Section 4.3: Simple prediction tasks: classification and scoring

Beginner experiments work best when the output is simple and checkable. Two spreadsheet-friendly task types are: (1) binary/multi-class classification (predict a label), and (2) scoring/ranking (predict a numeric score and apply a threshold or compare averages). Both can be evaluated with simple metrics.

Classification example: Predict whether a message is “urgent” (1) or “not urgent” (0). Baseline A could be “always 0” or “keyword rule.” Experiment B could be “keyword rule + sender is VIP.” Your evaluation is a confusion table: predicted vs. actual. From that, compute accuracy and error rate (1 − accuracy). If class imbalance is severe (e.g., only 5% urgent), accuracy can be misleading, so record the base rate (percentage of positives) in your results table.

Scoring example: Predict a satisfaction score from 1–5, or predict a risk score 0–100. Baseline A might predict the overall average score. Experiment B might predict a slightly different score based on a simple feature (e.g., add +1 if complaint contains “late”). Evaluate with average error: mean absolute error (MAE) is easy in a spreadsheet and interpretable (“on average, off by 0.8 points”).

Engineering judgment: choose a task where the label is stable and not subjective. If humans disagree often, your model cannot exceed that ceiling. If you must rely on human labels, document who labeled the data and whether you used a single rater or multiple raters. This context matters when interpreting “small” improvements.

Finally, define what counts as success before you run B. For example: “B is better if it reduces error by at least 2 percentage points” or “B is better if MAE decreases by at least 0.1.” Predefining this avoids accidentally celebrating noise.

Section 4.4: Using spreadsheets to compute outcomes step by step

Your spreadsheet should read like a pipeline: raw data → split → baseline prediction → variant prediction → evaluation. Make it easy to audit. Avoid “mystery formulas” spread across random cells; instead, use explicit columns with clear headers.

A practical layout (one row per example):

• id: unique identifier for the row.
• text / features: the inputs you use.
• label: the ground truth target.
• split: train/test (frozen values).
• pred_A: baseline output.
• pred_B: experiment output.
• correct_A, correct_B: 1/0 correctness flags.

For classification, a simple correctness formula is: =IF(pred_A=label,1,0). Accuracy on the test set becomes: average of correct_A where split="test". In Google Sheets or Excel, you can compute that with AVERAGEIF (or AVERAGEIFS). Error rate is =1-accuracy. For numeric prediction, compute abs_error_A as =ABS(pred_A-label) and take the average on the test set.

Save outputs in a clean results table on a separate sheet tab called Results. Each row in Results should be one experiment run (A and B evaluated on the same split). Include: dataset name/version, split method, metric values, and a short interpretation. This table becomes your source for charts later and prevents you from “recomputing” past results with accidentally changed formulas.

Common mistakes to avoid: (1) letting RAND() continuously reshuffle splits, (2) applying text cleanup to only one experiment’s features, (3) filtering the dataset differently for A and B, and (4) overwriting columns (always create new columns for new runs, or snapshot results).

Section 4.5: Experiment logs: date, data version, settings, notes

An experiment log is your lab notebook. It turns a one-off spreadsheet into research evidence. The log should let you answer: “What exactly did I do, on what data, with what settings, and what happened?” This is essential when you return a week later and cannot remember which threshold you used, or when you need to justify why the result is credible.

Create a sheet tab called Experiment Log and record one row per run. At minimum include:

• Date/time and your name/initials.
• Data version: file name, date pulled, any filtering (e.g., “removed duplicates,” “kept English only”).
• Task definition: target label rule (exact wording).
• Split method: random seed if applicable, percentage train/test, and how you froze it.
• Experiment A (baseline): rule/threshold/formula.
• Experiment B: the one change from A.
• Metrics: accuracy/error or MAE, computed on test only.
• Notes: surprises, suspected issues, and next action.

Engineering judgment: write notes as if you will hand this to a reviewer. If you made a subjective call (e.g., “dropped 12 rows that looked mislabeled”), record the reason. If you suspect leakage (e.g., a feature that directly encodes the label), note it and plan a rerun without that feature. The log is also where you state what you did not change, reinforcing fairness.

When you later write your report, the experiment log becomes your Methods section draft. It also protects you from accidental “research amnesia,” where you believe you improved something but cannot reconstruct how.

Section 4.6: Re-running the experiment: quick repeat checks

Good experiments are repeatable. Before you trust your conclusion, do quick repeat checks that catch spreadsheet errors and overly fragile gains. You are not doing advanced statistics here; you are doing basic sanity checks that prevent you from reporting a fluke.

First, re-run the evaluation calculation without changing the split: confirm the same metric values appear. If numbers change, something is unstable—often RAND() was not frozen, filters were left on, or formulas reference the wrong range. Second, spot-check a handful of rows: does pred_A match the baseline rule you wrote? Do a few test rows by hand to ensure the formula does what you think.

Third, do a small robustness check: run the same A vs. B on an alternate split (create a second split column, freeze it, and label it Split2). If B only wins on one split but loses on another, your improvement may be noise or dependent on specific examples. Record both outcomes in the Results table and log—this is honest research practice.

Finally, check for “silent unfairness.” Did B accidentally use a column that includes future information (e.g., resolution time) or a human decision made after the event (leakage)? Did you compute the baseline on the whole dataset (including test) instead of training only (e.g., using the global average rather than train average)? These mistakes can create impressive-looking gains that disappear when corrected.

Once these repeat checks pass, you have something worth reporting: a baseline, a variant, a fair A/B comparison on the same test set, and a documented trail of what you did. That is the core of beginner-friendly AI research.

Most beginner metrics for classification (yes/no labels) come from the same simple counting tool: the confusion matrix. It sounds fancy, but it’s just a 2×2 table that counts how many predictions fall into each bucket. Start with a binary question like “Is this review positive?” or “Will this email be spam?” You have the true label (what actually is) and the predicted label (what your system guessed).

The four buckets are:

• TP (True Positive): predicted positive, actually positive.
• FP (False Positive): predicted positive, actually negative. (A false alarm.)
• TN (True Negative): predicted negative, actually negative.
• FN (False Negative): predicted negative, actually positive. (A miss.)

In a spreadsheet, you can compute these counts with COUNTIFS. Example: if column A is true label ("Positive"/"Negative") and column B is prediction, then TP is =COUNTIFS(A:A,"Positive",B:B,"Positive"). Repeat for the other three combinations. Put the four counts in a small table with rows for “Actual” and columns for “Predicted.”

Why this matters: every “headline metric” (accuracy, precision, recall) is derived from these four numbers. If you don’t trust the counts, you can’t trust the metric. Also, the confusion matrix forces you to be specific about what kind of mistake is being made. In real applications, FP and FN often have very different costs (blocking a legitimate email versus letting spam through).

Practical check: before calculating anything else, scan a few rows manually. Do the labels look consistent? Are there unexpected values (blank, “N/A,” extra spaces) that would break your COUNTIFS and silently undercount? This is where many experiments fail: the metric is “wrong” because the categories don’t match perfectly.

Accuracy is the simplest metric: (TP + TN) / (TP + FP + TN + FN). Its partner is error rate: 1 - accuracy. Accuracy is attractive because it’s easy to understand, easy to compute, and works well when classes are balanced and errors have similar consequences.

Accuracy can mislead when one class is much more common than the other. If 95% of emails are “not spam,” a model that always predicts “not spam” gets 95% accuracy while being useless. This is why you often add precision and recall:

• Precision (how “clean” your positive predictions are): TP / (TP + FP). High precision means few false positives.
• Recall (how many real positives you caught): TP / (TP + FN). High recall means few false negatives.

Each can mislead, too. Precision can look great if your model rarely predicts “positive” at all; it only “speaks up” when it’s very sure, missing many true cases (low recall). Recall can look great if your model predicts “positive” too often; it catches most positives but creates many false alarms (low precision). The best metric depends on your question and the cost of mistakes.

Engineering judgment tip: write down which error is worse before you compute metrics. For example, in a medical screening toy dataset, false negatives can be more harmful than false positives, so you may prioritize recall. In a content moderation setting, false positives may unfairly silence benign content, so you may prioritize precision. This is also where you decide a classification threshold if your tool outputs probabilities: moving the threshold trades off FP and FN.

To summarize results clearly, create one spreadsheet table that includes TP, FP, TN, FN, accuracy, precision, and recall for each approach you tested (e.g., “Baseline rules” vs “Improved rules”). Then make one chart: often a simple bar chart with accuracy/precision/recall side-by-side is enough. The table gives exact numbers; the chart shows the trade-offs at a glance.

Beginners often run an experiment once, get a number (say 82% accuracy), and treat it as a property of the method. In reality, that number is a property of that particular split of data and that particular run. If your dataset is small, a few examples can swing the result dramatically. Even if you used a no-code tool, randomness can sneak in through how data was sampled, shuffled, or split.

A practical fix is to compute averages across multiple splits. In plain terms: run the same experiment several times, each time using a different random train/test split (or different held-out subset), and record the metric each time. Then compute:

• Mean (average): =AVERAGE(range) to summarize typical performance.
• Spread: use =STDEV.S(range) (standard deviation) or at least report min and max. This communicates uncertainty.

Even with only 5 runs, you learn a lot: if Model A averages 83% and Model B averages 82% but the results overlap widely (e.g., 79–86% for both), the difference is likely not stable. If Model A is consistently better across runs, your claim is stronger.

This is also how you check for overfitting in plain terms. Overfitting means doing well on the data you “learned from” but not on new data. In a spreadsheet workflow, you’ll often see it as: very high performance on the training set and noticeably lower performance on the test set, or wildly variable test results across different splits. Record both training and test metrics when possible; a big gap is a warning sign.

Practical outcome: add a second results table that lists each run as a row and metrics as columns, plus a final “Average ± spread” line. Then make one chart that shows variability—e.g., a simple line chart of accuracy by run for each approach, or a bar chart with error bars if your tool supports it. This keeps you honest: you’re reporting a range, not a single lucky number.

Data leakage is the most common reason beginner results look “too good to be true.” Leakage happens when information from the test set (or from the future) sneaks into training or into the features you used. The model appears brilliant, but only because it had access to clues it wouldn’t have in real use.

Common leakage patterns in beginner projects:

• Duplicate or near-duplicate rows across train and test (e.g., the same sentence appears twice). The model “memorizes” rather than generalizes.
• Using a feature that directly contains the label, even indirectly (e.g., a column like “approved_by_admin” when predicting “approved,” or a filename that encodes the class).
• Preprocessing using the full dataset before splitting (e.g., computing normalization constants, selecting top words, or removing “rare” categories using all rows). This lets test data influence training decisions.
• Time leakage: predicting an outcome using information not available at prediction time (e.g., using “final grade” to predict “pass/fail” early in a course).

How to spot leakage with simple checks: (1) If performance is extremely high on a hard problem, assume leakage until proven otherwise. (2) Compare training vs test: perfect (or near-perfect) test performance on a small dataset is a red flag. (3) Audit your columns: for each feature, ask “Would I know this at the moment I’m making the prediction?” (4) Check for duplicates: sort by an ID column, or use spreadsheet functions to detect repeated text; even a quick “Remove duplicates” on a copy can reveal surprises.

Prevention habits: split into train/test first, and keep the test set “read-only” for your decisions. Do not tune rules, thresholds, or feature choices by looking at test errors repeatedly. If you must iterate, create a third split (validation) for tuning, and reserve the test set for the final check.

Practical outcome: in your report’s methods section, include one sentence describing how you prevented leakage (e.g., “We split the dataset before any feature selection and verified no duplicate items appeared across splits.”). That single sentence makes your experiment more credible.

Metrics don’t just measure performance; they can hide who pays the price of mistakes. A system can look “good” on average while being consistently worse for a subgroup. You do not need advanced fairness mathematics to take a meaningful first step: break down errors by group when your dataset includes people and a relevant grouping variable (such as language variety, age bracket, region, or device type). If you don’t have such a column, your first fairness action may be to notice that you can’t evaluate fairness with your current data.

Start with the confusion matrix idea, but compute it per subgroup. For each group, calculate TP/FP/TN/FN and then accuracy, precision, and recall. In a spreadsheet, this can be done with COUNTIFS that also include the group column, or by using a pivot table.

• If one group has much lower recall, your system misses true cases more often for them (higher false negatives).
• If one group has much lower precision, your system falsely flags them more often (higher false positives).

Fairness is contextual: “harm” depends on the application. In an educational support tool, false negatives might deny help to students who need it; false positives might waste limited support resources or stigmatize students. In content filtering, false positives can silence certain dialects or communities disproportionately. Your job as a beginner researcher is not to solve fairness entirely, but to demonstrate awareness and basic measurement.

Practical reporting habit: add a small table beneath your main results with subgroup metrics (even if it’s only two groups and a note about small sample sizes). If the subgroup sizes are tiny, say so explicitly and treat the findings as tentative. This keeps your research honest and signals that you understand the limits of your data.

After you compute metrics, the final skill is interpretation: writing a short, honest explanation of what happened. This is where beginner reports often overreach. A metric is evidence about performance on your dataset under your experimental setup. It is not proof of general superiority, and it does not automatically translate to real-world impact.

Claims you can make (when supported):

• “On our held-out test set of N items, Approach A achieved higher average accuracy than Approach B across five random splits.”
• “Approach A improved recall but reduced precision, indicating more positives were detected at the cost of more false alarms.”
• “Results varied by split (± spread), so the improvement is modest and may not be stable on new data.”

Claims you generally can’t make from a small beginner experiment:

• “This method is best for everyone” (you didn’t test everyone).
• “The model understands X” (metrics measure outputs, not understanding).
• “It will work in production” (real data drifts; costs and constraints differ).

A practical writing template for your interpretation paragraph is: (1) restate the research question; (2) state the main metric result with numbers; (3) describe the trade-off (FP vs FN); (4) mention uncertainty/variability; (5) note key limitations (small data, possible bias, label noise); (6) propose the next experiment. Keep it short and factual.

Finally, make your results easy to audit. Include one table (with TP/FP/TN/FN and derived metrics) and one chart (showing the main comparison or the variability across runs). If a reader can reproduce your metric from the confusion matrix counts, your report becomes trustworthy. If the numbers can’t be traced, even a “good” accuracy is hard to believe.

Section 6.1: Report structure: title, question, method, results, discussion

A strong beginner report can fit on one page if you use a predictable structure. Predictability is a feature: it helps readers find what they need and helps you avoid “hand-wavy” claims. Use these headings in order: Title, Research Question, Method, Results, Discussion.

Title: be specific enough that a reader can guess the dataset type, comparison, and outcome. Example: “Comparing Two Prompt Styles for Classifying Customer Support Tickets in a Spreadsheet (Accuracy on 200 Labeled Rows).” Avoid vague titles like “AI Experiment.”

Research Question: write one sentence that is testable and includes your outcome metric. Example: “Does prompt template B improve accuracy over template A for labeling ticket categories, measured on a held-out test set?” If you cannot point to a metric and a dataset split, the question is not yet research-ready.

Method: summarize your dataset source, labels, tool used (e.g., spreadsheet formulas, no-code model, or an API), how you split data, and what you kept constant across conditions. This is where you show the comparison was fair.

Results: report numbers first (accuracy, error rate, averages) and then interpretation. Don’t hide the metric inside paragraphs. If you ran multiple runs or subsets, show them.

Discussion: answer the research question cautiously. Explain what you think happened and why, but separate evidence (“accuracy increased from 0.74 to 0.78”) from hypotheses (“template B may reduce ambiguity because…”). End the discussion with a short limitations + next steps paragraph (you will refine that in Section 6.4).

• Common mistake: writing “Conclusion” as marketing. A research discussion is not a victory lap; it is an honest map from data to claim.
• Practical outcome: a reader can skim headings and still understand your experiment end-to-end.

When you draft, write the research question and metric first. If the rest of the page doesn’t support that question, you will notice quickly and can fix the experiment design before you publish.

Section 6.2: Writing methods clearly: enough detail to reproduce

Methods are the difference between “I tried something” and “I ran an experiment.” Your goal is not to impress; your goal is to enable repetition. A good methods section lets another beginner replicate your steps in the same tools you used, ideally in under an hour.

Include five concrete elements. First: data description (what rows represent, how labels were created, and what columns were used). Second: data split (training/dev/test or just train/test), including counts. Third: procedure (step-by-step, including prompt templates or settings). Fourth: metric definition (exact formula and how you handled edge cases). Fifth: controls (what stayed constant across conditions).

• Example method detail (spreadsheet experiment): “We used 200 tickets labeled into 5 categories by one human labeler. We randomly split into 150 train (used only to design prompts) and 50 test (never viewed during prompt edits). We queried the model once per row using prompt A or prompt B. Temperature was fixed at 0.0. We computed accuracy as correct/total on the 50-row test set.”

Be explicit about anything that could change the outcome: random seeds (if any), model version/date, and whether you re-ran outputs. If you used manual cleaning, say what rules you applied. If you removed rows, say why and how many.

Common beginner mistakes to call out in methods: (1) tuning prompts on the test set (data leakage), (2) changing multiple variables at once (unfair comparison), and (3) unclear labeling (no one knows what “correct” means). If you made a mistake and fixed it, write the fixed procedure—then add a short note in limitations acknowledging the risk and how you mitigated it.

Engineering judgment matters here: aim for “reproducible enough,” not “academic perfection.” If your reader can recreate the dataset, rerun the same steps, and recompute the metric, you’ve succeeded.

Section 6.3: Presenting results: tables, charts, and plain-language captions

Results should be easy to verify and hard to misread. Most beginner reports fail because numbers are buried in narrative or because charts lack labels and context. Use one table for the primary metric, and one simple chart if it genuinely adds clarity.

Start with a results table that includes: condition name (e.g., Prompt A vs Prompt B), sample size (N), main metric (accuracy, average error), and possibly a secondary metric (e.g., “unknown” rate). Keep it small so it fits on the page. If your metric is accuracy, also show the numerator/denominator (e.g., 39/50) so readers can sanity-check the percentage.

• Good table row: “Prompt B | N=50 | Accuracy=0.78 (39/50) | ‘Other’ predicted=6%.”

Add a chart only when it answers a question your table cannot. Examples: a bar chart comparing accuracy across categories, or a small confusion matrix (predicted vs actual) if misclassifications matter. If you include a confusion matrix, keep it readable: 5–8 classes max, otherwise it becomes noise.

Every table or chart needs a plain-language caption that states what it shows, on what split, and what to notice. A strong caption looks like: “Table 1: Test-set accuracy (N=50) comparing two prompt templates. Prompt B improves accuracy by 4 points, with most gains in the ‘Billing’ category.” Avoid captions like “Results.”

Common mistakes: (1) reporting only relative improvement (“10% better”) without baseline, (2) mixing train and test metrics, (3) cherry-picking one run when results vary, and (4) making visual choices that exaggerate differences (truncated y-axis without stating it). Your job is to communicate faithfully, not to win an argument.

End the results section with one or two sentences that interpret the numbers cautiously. Save explanations and next steps for the discussion so results remain a clean record of what happened.

Section 6.4: Limitations and threats to validity (beginner-friendly)

Limitations are not an apology; they are part of honest reasoning. In AI experiments, a “threat to validity” is anything that could make your conclusion unreliable, non-generalizable, or unfair. Stating limitations helps readers interpret your result correctly and helps you plan a better follow-up experiment.

Use three buckets that are easy for beginners. Internal validity: did something other than your intended change cause the result? Example: you changed both the prompt and the label mapping at the same time, so you can’t attribute the improvement to the prompt alone. Another internal threat is data leakage (editing prompts after seeing test outcomes).

Measurement validity: is your metric capturing what you care about? Accuracy can hide problems when classes are imbalanced. If 80% of rows are “General,” a model can look “good” while failing minority classes. Note if you did not check per-class performance or if labels might be inconsistent.

External validity: will it work beyond this dataset? A dataset of 200 items from one week may not represent future weeks. If your data comes from a single source or a narrow topic, say so and avoid broad claims like “this prompt works for all support tickets.”

• Practical limitation template: “This study uses N=50 test items from one team, labeled by one person. Results may change with more diverse data or a second labeler. We reduced leakage by freezing the test set before prompt iteration, but minor unintentional tuning may remain.”

Then list next steps that are proportional: add more labeled rows, stratify the split, include a second annotator for a small subset, run a second random split, or test on a new time period. Overpromising is a common beginner mistake—avoid “future work will solve all bias.” Instead, write the smallest next experiment that would increase confidence.

Section 6.5: Ethics and privacy: what to include (and exclude)

Sharing your report should not expose people, sensitive data, or proprietary content. Ethical reporting is a core research skill, not a bureaucratic afterthought. Your rule of thumb: publish enough to reproduce the analysis, but not enough to identify individuals or leak confidential information.

In the report, include data provenance (where data came from, in general terms), consent/permission (did you have the right to use it?), and de-identification steps (what you removed or transformed). If you used public data, still state the source and any license constraints. If you created synthetic data, say how it was generated and whether it resembles real people.

• What to exclude: names, emails, phone numbers, addresses, unique identifiers, full free-text messages that could re-identify someone, and any internal company details not approved for release.
• What to include instead: aggregated statistics, short redacted examples, label definitions, and a small safe sample dataset if permitted (or a schema + generation script).

Also disclose model/tool usage in a way that respects policy: if you used an external AI tool, confirm whether data was sent off-device and whether that was allowed. If you cannot share the raw dataset, say so plainly and provide a “data statement” describing fields, counts, and collection window.

Common mistake: attaching the spreadsheet as an appendix “because it’s easier,” without checking for hidden columns, comments, or metadata. Before publishing, export a clean copy with only necessary fields and verify there is no personal information in any cell.

Ethics improves research quality: when you handle data carefully, you also reduce leakage, labeling shortcuts, and accidental memorization of sensitive text.

Section 6.6: Final checklist: reproducibility pack (data notes, log, report)

Your final deliverable is not just a one-page report. It is a small “reproducibility pack” that makes your work rerunnable and reviewable. This is how you publish or present with confidence: a clean report plus a clean appendix.

Include three artifacts. (1) The report: the one-page write-up with structure from Section 6.1. (2) Data notes: a short appendix describing dataset fields, label definitions, how many rows, and any filters/removals. If you cannot share data, include a schema and summary stats. (3) An experiment log: a simple chronological record of what you changed between conditions (prompt versions, parameters, date/time, and which split was used).

• Reproducibility checklist: model/tool name and version/date; exact prompt templates or rules; dataset split counts; metric formulas; links or references to data source; randomization method (if used); table of results with N; privacy check confirmation; known limitations; next experiment proposal.

For presentation, keep the main report clean and move details into the appendix. The appendix is where you can place: prompt text, label guidelines, screenshots of spreadsheet formulas, or a small sample of redacted rows. This separation helps you communicate clearly without overwhelming readers.

Final common mistakes to avoid: (1) publishing only screenshots (not reproducible), (2) changing the dataset after results are written, (3) forgetting to label which split results came from, and (4) drawing conclusions that the metric does not support. If you can hand your pack to someone else and they can reproduce the main table, you have completed your first real research report.