Instruction

When you type a sentence into an image generator and a picture appears seconds later, it can feel like magic or like theft. It is closer to statistics: a model trained on a very large number of pictures. Knowing how it works lets you use it as an artist instead of being used by it.

Step 1: training data. A text-to-image model learns from pairs: an image and a caption describing it. Stable Diffusion, released in 2022, was trained on images drawn from LAION-5B, a public dataset of about 5.85 billion image-text pairs collected automatically from the web. Most of the artists, photographers and institutions whose pictures were collected were never asked. Keep that fact in mind; we return to it in Semester B.

Step 2: learning to remove noise. Most current image generators are diffusion models. During training, the computer takes a real image and adds random noise to it, a little at a time, until nothing is left but static. The model is trained to do the reverse: given a noisy image and its caption, predict the noise so it can be subtracted. Repeated across millions of images, the model learns statistical patterns of what things look like — how fur catches light, how a horizon divides a landscape.

Step 3: generating. To make a new picture, the model starts from pure random noise and removes predicted noise in a series of steps, steered by your prompt, which a separate text encoder has turned into numbers. Because the starting noise is random, the same prompt gives a different image each time. That is why generators are good at variations and poor at producing exactly what you pictured.

What it does and does not do. The model does not search a folder of stored pictures and paste pieces together. But “it doesn’t store images” is not the whole story: researchers have shown that diffusion models can sometimes reproduce near-copies of training images, especially ones that appeared many times in the data. And because a model learns what is common in its data, it repeats that data’s habits — who is shown doing which jobs, which places look “normal”, whose art styles are imitated.

Machines made art before this. Harold Cohen, a British-born painter who worked in California, spent from the early 1970s until his death in 2016 developing AARON, a program that made drawings by following rules Cohen wrote himself. AARON’s images came from an artist’s explicit instructions; a diffusion model’s come from patterns learned from other people’s images. That difference will matter when we discuss authorship.

An analogy in paint. Look closely at Georges Seurat’s A Sunday on La Grande Jatte — 1884 (1884-86, Art Institute of Chicago). Up close it is a field of separate dabs of color; step back and people, parasols and a river resolve. The analogy is loose — Seurat placed every dot deliberately, following a theory of color — but it captures the moment a diffusion model repeats over and over: structure emerging from scattered marks.

How we will use AI here. You will not need an account on any AI service, and nothing in this course asks you to sign up for anything. What you need is one machine-made image to answer with your hands, and there are three ways to get one. All three earn full credit, all three give you an image you can credit honestly by tool and date, and you will keep using the one you choose in Lessons 1.4 and 1.5.

Option A, the image set in this lesson. Three AI-generated works are reproduced inside free U.S. Copyright Office documents. Those documents are U.S. federal publications and so are in the public domain, and in each case the Office decided that the AI-generated material itself carries no copyright because no human authored it. Each work is described below in enough detail to plan a translation from the description alone, so this route works even with no internet and no printer.

• Théâtre D’opéra Spatial (Jason M. Allen, made with Midjourney, 2022). A tall, dark, heavily ornamented interior like an opera house or a palace hall, in golds, browns and deep oranges. At the center is an enormous bright circle, a round opening filled with pale haze, with a faint crowd and a landscape far beyond it. Three figures in long, heavy gowns stand on a wet-looking reflective floor with their backs to us, facing the circle; two more stand at the right-hand edge. Every architectural edge is crusted with ornament that dissolves into smears when you look closely. Reproduced in the Copyright Review Board’s decision letter of 5 September 2023 (copyright.gov, rulings-filings, review-board, Theatre-Dopera-Spatial.pdf).
• A Recent Entrance to Paradise (generated by Steven Thaler’s Creativity Machine). Railway tracks run straight away from the viewer and through a low stone arch smothered in leaves and purple blossom. A second, smaller arch shows through the first, and past that a green glow. The whole picture is mottled and grainy, as though built from tiny broken dabs, and the color runs from dark violet at the left to warm pink at the right. Reproduced in the Copyright Review Board’s decision letter of 14 February 2022 (copyright.gov, rulings-filings, review-board, a-recent-entrance-to-paradise.pdf).
• Zarya of the Dawn (Kris Kashtanova, images made with Midjourney, 2022). The comic’s panels are reproduced through the Copyright Office’s letter of 21 February 2023 (copyright.gov/docs/zarya-of-the-dawn.pdf). Two images repeat: a young woman with dark curly hair standing in a city park at golden hour, holding a small handwritten card, with a lit skyline behind her; and wide red-rock desert scenes under a pale sky, with a distant vehicle on a track. Choose one panel, not a whole page.

Option B, Wikimedia Commons. Commons hosts thousands of AI-generated images carrying its PD-algorithm tag, which states that the file is in the public domain because it is the work of a computer algorithm or artificial intelligence and does not contain enough human authorship to support a copyright claim. Browse Category:AI-generated images at commons.wikimedia.org and choose one with a strong main shape. Copy the file name, the tool and the date from the file page, and the prompt if the uploader recorded one.

Option C, make the image yourself. Run a generative sketch in the p5.js editor (editor.p5js.org), which needs no sign-in, and keep the frame you like. Be honest about what it is: this is rule-based generative art, the AARON tradition rather than the diffusion tradition, and saying so is part of your credit line. Record it as a p5.js sketch, your parameters and the date. The comparison between the two traditions is exactly what this lesson is about.

The machine image is where you start; your hands make the art.

Formative check

Work through these before moving on. They are not graded — they tell you, and your teacher, whether the standard below has landed yet.

function setup() {
  createCanvas(300, 300);
  pixelDensity(1);
  noStroke();
}
function draw() {
  background(255);
  fill(0);
  ellipse(150, 150, 160, 160);
  let amt = map(mouseX, 0, width, 0, 1, true);
  loadPixels();
  for (let i = 0; i < pixels.length; i += 4) {
    let v = pixels[i] * (1 - amt) + random(255) * amt;
    pixels[i] = v; pixels[i + 1] = v; pixels[i + 2] = v;
  }
  updatePixels();
}