
evaluate explanatory media about AI using stated criteria for accuracy, clarity and honest imagery, and explain unsupervised learning well enough to judge whether an explainer gets it right.
Instruction
Before you design an explainer, you need to be able to judge one. A critique is not a list of likes and dislikes. It is an evaluation against criteria that anyone looking at the work can see, applied to evidence in the work itself. This lab builds the criteria you will use for the rest of the year.
A critique sequence. Work through four steps in order. Describe: what is literally there (text, images, colors, sequence) without judgment. Analyze: how the parts relate (what is largest, what the eye sees first, what each color stands for). Interpret: what the work says about AI and how it wants you to feel. Evaluate: how well it meets each criterion, with evidence. Skipping to evaluation is the most common mistake; it produces opinions without reasons.
Three criteria for AI explainers. Accuracy: are the technical claims true, and are uncertain claims marked as uncertain? Clarity: can the intended audience follow the sequence without outside help? Honest imagery: do the pictures suggest something the system does not do? Stock images of glowing blue brains, humanoid robots and floating code imply a thinking machine. When the subject is a narrow classifier, that imagery misleads, even if every sentence is correct. Researchers and designers have criticized these clichés for exactly this reason, and some have built libraries of alternative images.
Test an explainer’s accuracy: unsupervised learning. Many explainers blur two ideas you now need to keep apart. In supervised learning, every training example has a label. In unsupervised learning, there are no labels; the system looks for structure on its own. The most common form is clustering: grouping examples that are similar in their features. Imagine a set of patient records with two features, resting heart rate and hours of sleep, and no diagnoses attached. A clustering method might find three groups. It cannot tell you what the groups mean. A person has to look at them and ask whether they correspond to anything real, or to an artifact such as two clinics measuring heart rate differently. An explainer that says “the AI discovered three kinds of patients” has over-claimed; the honest version is “the method found three groups, which researchers then had to interpret”.
Rules for critique. Critique the work, not the maker. Begin every evaluative statement with the criterion and the evidence: “On accuracy, the second panel says the model understands X-rays; the model only outputs a probability.” Before you judge a choice, restate in one sentence what the maker was probably trying to do, so you are judging the work fairly. And give one specific suggestion for every weakness you name; a critique is supposed to make the next version better. You will use the same rules on your own drafts.
Why it matters. People make decisions about their health from media. An explainer that exaggerates what an AI system can do may lead a reader to trust it too much, or to dismiss useful tools. Critique is how media artists hold each other to the standard Nightingale set: show the evidence honestly.
📖 Read for this lesson
The reading opens over the lesson; close it to come back to exactly where you were. The checks below draw on it.
Formative check
Work through these before moving on. They are not graded — they tell you whether the standard below has landed yet.
An explainer says: ‘The AI studied 10,000 patient records and discovered three new diseases.’ The method was clustering with no labels. What is the most accurate criticism?
An explainer can contain only true sentences and still mislead through its images.
Describe
Tap to revealWhat is literally there, without judgment
Analyze
Tap to revealHow the parts relate: scale, order, color, emphasis
Interpret
Tap to revealWhat the work says and how it wants the viewer to feel
Evaluate
Tap to revealHow well it meets each criterion, with evidence
Evaluate one AI explainer from the set printed in this lesson’s reading on accuracy, clarity and honest imagery. For each criterion give a rating (weak, adequate, strong), one piece of evidence from the work and one specific suggestion.
How confident are you that you can evaluate an AI explainer against accuracy, clarity and honest-imagery criteria, and distinguish supervised from unsupervised learning?
Practice
Work these on paper or in your notebook, then open Check your answer. Aim for all of Fluency and Application; try at least one Challenge.
Printable version: this unit’s practice workbook (PDF)
Fluency
Build speed and accuracy with the core skill.
- Put the four critique steps in order and give a one-phrase description of each: Interpret, Describe, Evaluate, Analyze.
Check your answer
Answer: 1. Describe: what is literally there, without judgment. 2. Analyze: how the parts relate (scale, order, color, emphasis). 3. Interpret: what the work says and how it wants the viewer to feel. 4. Evaluate: how well it meets each criterion, with evidence.Skipping straight to evaluation is the most common mistake; it produces opinions without reasons. - Name the three criteria for AI explainers and the question each asks.
Check your answer
Answer: Accuracy: are the technical claims true, and are uncertain claims marked as uncertain? Clarity: can the intended audience follow the sequence without outside help? Honest imagery: do the pictures suggest something the system does not do?All three are needed; an explainer can pass accuracy and still fail honest imagery. - Supervised or unsupervised? (a) grouping patient records by resting heart rate and hours of sleep, with no diagnoses attached; (b) training on retinal photographs labeled ‘disease’ or ‘no disease’; (c) clustering.
Check your answer
Answer: (a) unsupervised; (b) supervised; (c) unsupervised.Supervised learning uses labeled examples; unsupervised learning has no labels and finds structure, most often by clustering.
Application
Use the skill in context. Show your reasoning.
- Rewrite this claim so it is accurate: ‘The AI discovered three kinds of patients.’ The method was clustering on unlabeled records.
Check your answer
Answer: ‘The method found three groups in the records, which researchers then had to interpret.’Clustering cannot tell you what groups mean. A person must check whether they correspond to anything real or to an artifact, such as two clinics measuring heart rate differently. - Turn this opinion into a critique statement that follows the lesson’s rules: ‘The poster is bad because it is creepy.’ The poster shows a glowing humanoid robot reading an X-ray; the product only outputs a probability that an X-ray shows a fracture.
Check your answer
Answer: Model: ‘On honest imagery, the main image shows a humanoid robot reading an X-ray, which implies a thinking machine; the product only outputs a probability. Suggestion: show an X-ray going in and a probability score coming out, with a doctor reviewing it.’A strong statement begins with the criterion, cites evidence in the work, critiques the work rather than the maker and gives one specific suggestion. - A clustering method finds two groups in heart-rate records. Later you learn that one clinic measured heart rate lying down and another sitting up. What might the groups really reflect, and what lesson does this hold for an explainer?
Check your answer
Answer: The groups may reflect the two measurement methods (an artifact), not two kinds of patients. An explainer should say the method found groups that people must interpret, not that it discovered real categories.This is the lesson’s example of an artifact: structure that comes from how data was collected. - Before judging a choice, the lesson asks you to restate what the maker was probably trying to do. Why does this rule make a critique fairer? Answer in 2 sentences.
Check your answer
Answer: Model: Restating the maker’s aim makes you judge the work against what it set out to do rather than against a work you would have made. It also keeps the critique about the work, not the maker, so the suggestion can actually improve the next version.A strong answer connects the rule to fairness and to making the next version better.
Review
Keep earlier skills sharp.
- Review 1.2: Explain why holding back a test set matters, using the word overfitting.
Check your answer
Answer: A rule tuned on the training cards will fit them well; only examples it has never seen show whether it learned the pattern. A rule that fits training data perfectly but fails on new data is overfitted.The test set must stay unseen until the rule is fixed. - Review 1.1: An explainer about a narrow classifier shows a glowing brain. Which of the three AI terms does that image suggest, and why is it misleading?
Check your answer
Answer: It suggests artificial general intelligence, a thinking machine. It misleads because the product is narrow AI doing one task.Images of glowing brains and humanoid robots suggest AGI when the product is a narrow classifier.
Two AI Explainers to Critique, a Claims Checklist and a Scoring Sheet
SLS original practice explainers and scoring sheet; claims checked against NIST AI 100-1 (Jan. 2023) and FDA De Novo summary DEN180001 (IDx-DR); retrieved Sept. 26, 2026 · SLS original, quoting public-domain U.S. government documents
Why this reading. The open question asks you to evaluate one AI explainer from the set printed in this reading. Two are printed here, described closely enough to critique with no image at all, with a claims checklist tied to primary sources and a scoring sheet. Both explainers are SLS originals, invented for practice. They are not real products, and every number in them is invented. Use the same four steps on any real explainer you find yourself.
Explainer A: a poster called “MediMind”
Layout. Portrait poster, 24 × 36 inches. Near-black navy background. In the center, filling about 60% of the height, an illustration of a human head in profile with a glowing blue brain made of circuit lines; light rays radiate from it. At the top in 96-point white type: MEDIMIND AI THINKS LIKE A DOCTOR. Under the brain, in 40-point cyan type: The AI studied 10,000 patient records and discovered three new diseases. At the bottom, in 14-point mid-gray type on the navy background, three lines:
- “MediMind understands your health better than you do.”
- “99% accurate.”
- “Method: clustering of resting heart rate and hours of sleep, no diagnoses supplied.”
A tiny 7-point footnote, the same gray, reads: “Results from one hospital.” Every heading uses the same cyan as the arrows that point from the records to the brain.
Explainer B: a five-panel handout called “How a Screening Program Learns”
Layout. One letter-size page, landscape. Five numbered panels left to right, joined by gray arrows, on white. Type is a single sans-serif, headings 18-point black, body 9-point dark gray. Each panel has a flat drawing of a retinal photograph (a circle with branching lines) and no other imagery. Color has one job: orange marks a labeled example, teal marks the program’s output. Panel text, complete:
- Examples. “Doctors look at 20,000 eye photographs and mark each one: disease or no disease.”
- Features. “The program measures thousands of small patterns in each photograph. Its convolutional layers extract hierarchical representations.”
- Training. “It changes its settings until its guesses match the doctors’ marks as often as possible.”
- Testing. “Then we show it 2,000 new photographs it has never seen and count how often it agrees with the doctors.”
- Limits. “It only looks for one disease. It may be less accurate for kinds of patients who were rare in the examples.”
In panel 4 the number “2,000” is printed in orange, though panel 1 uses orange for labeled examples.
The four steps, applied to Explainer A (a model answer)
Describe. Dark poster, glowing brain in the center, three lines of small gray type at the bottom. Analyze. The largest, brightest elements are the headline and the brain; the method line is the smallest and lowest in contrast, so the eye reaches it last, if at all. Cyan means both “heading” and “arrow,” so color carries no separate meaning. Interpret. The work says a mind, a doctor-like mind, lives inside the software, and it wants the viewer to feel awe and trust. Evaluate. Accuracy, weak: clustering has no labels and finds groups; people must decide whether a group is a disease, so “discovered three new diseases” over-claims, and “understands your health better than you do” describes no capability the method has. “99% accurate” is not defined (accurate at what, against what, on whose records?), and a clustering method has no correct answers to be accurate against. Clarity, adequate: the sequence is clear but the one line that explains what was done is unreadable at that size. Honest imagery, weak: the brain and the rays suggest thinking, which the caption of a narrow method contradicts. Suggestions: replace the brain with a scatter of dots in three groups; make the method line the second-largest text; rewrite the headline as “A method found three groups in 10,000 records. Researchers are checking what they mean.”
Your turn: Explainer B and a claims checklist
Score Explainer B on the three criteria below and write one piece of evidence and one suggestion for each. Some things to notice, without spoiling the whole answer: panel 2 uses two terms the audience may not know, orange is used for two meanings, and panel 5 includes the honest limits that Explainer A omits. Then check these claims against primary sources:
| Claim in an explainer | What the primary source says |
|---|---|
| “An AI system is any software that thinks.” | NIST’s AI Risk Management Framework defines an AI system as “an engineered or machine-based system that can, for a given set of objectives, generate outputs such as predictions, recommendations, or decisions influencing real or virtual environments.” It says nothing about thinking or understanding. (NIST AI 100-1, January 2023, public domain.) |
| “The AI screening tool detects eye disease.” | The FDA’s decision summary for IDx-DR (DEN180001) says it detects “more than mild diabetic retinopathy” in adults with diabetes not previously diagnosed with it, that it is used with the Topcon NW400 camera, and that it is “not intended to detect concomitant diseases.” The claim leaves out the specific disease, the population and the camera. |
| “The tool is 87% accurate.” | The same summary reports two different numbers, not one: observed sensitivity 87.4% (173 of the 198 patients who had more than mild disease were flagged) and observed specificity 89.5% (556 of the 621 who did not have it were cleared). One figure called “accuracy” hides which kind of mistake is being counted. |
| “The system explains why it decided.” | NIST separates transparency (“what happened”), explainability (“how” a decision was made) and interpretability (“why”). Say which one you are claiming. |
A worked test of a clustering claim (SLS original numbers)
Eight invented patient records, each with resting heart rate (beats per minute) and sleep (hours): A (58, 8.5), B (61, 8.0), C (60, 7.5), D (74, 6.5), E (76, 6.0), F (75, 7.0), G (95, 5.0), H (98, 4.5). A clustering method that puts records close together in one group finds three groups: {A, B, C}, {D, E, F}, {G, H}. It tells you the groups exist; it does not tell you what they are. Suppose records A–C all came from one clinic that measures heart rate after ten minutes of sitting and G–H from a clinic that measures on arrival. The three “kinds of patients” might then partly be two clinics’ habits. The honest sentence is “the method found three groups, which people then had to interpret.”
Scoring sheet
| Criterion | Weak | Adequate | Strong |
|---|---|---|---|
| Accuracy | One or more technical claims false, or claims a capability the method lacks | True claims, but uncertain or missing pieces are not marked (no limits, no source) | Every claim checkable; limits named; sources cited |
| Clarity | The intended audience could not follow the order or the key text | Order clear, but some text too small, too technical or with undefined terms | A member of the audience could retell it correctly after one look |
| Honest imagery | Pictures suggest abilities the system does not have | Neutral pictures that neither help nor mislead | Pictures show what the system actually does |
Write your critique in this form: On [criterion], [what the work does, with evidence]; that rates [weak / adequate / strong] because [reason]; one change would be [specific suggestion]. Critique the work, not the maker.
Sources: NIST AI Risk Management Framework 1.0 (NIST AI 100-1) (doi.org); FDA De Novo decision summary DEN180001 (IDx-DR) (accessdata.fda.gov)