The testing vocabulary, complete

The delivery feature’s tests are green, engineers’ shorthand for every test passing. Launch is a few weeks away.

A checklist has appeared in the team’s tracker: “pyramid green · smoke passed on staging · load tested at 3× peak · UAT signed off.” Right now, three of those four lines are unreadable.

This lesson fixes that. Like Module 10’s nine words, it is deliberately a vocabulary lesson: the ten testing terms that fill launch checklists and QA conversations, organized into three clusters.

The three clusters arrive in the order the feature meets them on its way to launch. First, the world the tests run in. Then the special patrols, four specialized kinds of test that sweep through that world. Last, the humans who sign off. By the end, you will be able to read that checklist line by line.

Cluster one: what the tests run against

The pyramid’s tests need a world to run in, and it cannot be the real one because a test that charges a real credit card or emails a real customer is an incident, not a rehearsal.

What is test data?

Test dataConcept · lights on your maptest dataMade-up records shaped like the real thing (fake customers, believable orders, realistic mess) for tests to chew on. One hard rule: no real people’s information, ever, because the worlds it lives in are less guarded than production. is the invented-but-realistic information the tests run against. Sample customers, plausible orders, records with production’s shape and messiness but none of its reality.

One rule is firm. Test data should never contain real customer PII. Test environments rarely have production’s protections, and PII is the category Module 7 taught you the law cares about, so a leaked test database must have nothing real to leak.

What are mocks and stubs?

Some dependencies are too slow, expensive, or dangerous to use for real even in a test world. The payment provider is one, the email service is another, and so, surprisingly, is the clock.

Mocks and stubsConcept · lights on your mapmockA stand-in fake for a real dependency during testing: a pretend Stripe that always approves, or always declines, since unhappy paths need rehearsing too. Stub is a near-synonym in daily use. are stand-in fakes for those dependencies:

• The checkout tests never talk to the real Stripe. They talk to a pretend one that approves instantly and for free, or one that always declines when Module 3’s unhappy paths need rehearsal.
• The fake email service sends nothing.
• The fake clock reports whatever time the test needs.

The tests gain speed, predictability, and failures on demand, with no accidental charges and no confusing emails to real people.

The two words have technical distinctions, but in daily conversation they are near-synonyms: fakes, for testing.

What is test coverage?

Once the tests exist, a natural question follows. How much of the product’s code do they actually reach?

Test coverageConcept · lights on your maptest coverageThe share of the codebase the tests touch, as a percentage (“ordering is at 92%”). Good for spotting the least-tested corner; bad as a goal, since running a line is not the same as verifying it, and teams chasing the number write tests that only run lines. measures exactly that, as a percentage:

Read a coverage report as a flashlight, never as a trophy.

“Refunds is at 41%” is useful. It points at the least-tested corner of the codebase, where the next bug is most likely hiding unwatched.

“We hit 100%!” proves much less than it sounds. A test can execute a line without meaningfully checking what it does. And when the number itself becomes the goal, teams start writing exactly those tests. High coverage built from weak checks is a well-lit room where nobody is actually looking.

Cluster two: the special patrols

With the world built, four specialized patrols sweep through it, each guarding against a different kind of failure.

What is regression testing?

In software, a regression is a new change quietly breaking something that used to work. It is the bug behind the sentence “it used to work.”

Regression testingConcept · lights on your mapregression testingReplaying every existing test after each change, so anything that once worked and quietly stopped gets caught immediately. A regression is that backslide; the testing is simply the whole suite, run again, forever. guards against exactly that backslide, and you have already seen it work once earlier in this module.

Last lesson’s boundary test promised that an order of exactly $25.00 delivers free. Suppose that months from now, an engineer reworking the pricing code accidentally turns that rule into “over $25.” The engineer is not thinking about delivery fees at all. But test_members_get_free_delivery is, and it fails on their PR within minutes, naming the broken promise.

That is regression testing. It is not a new kind of test to write but the practice of re-running the entire accumulated pyramid on every change, so that every old promise is re-verified forever.

What is smoke testing?

Smoke testingConcept · lights on your mapsmoke testingThe quick is-it-alive check run immediately after new code goes live: a few basic actions tried once, looking for anything obviously broken. Depth is not the point; speed of bad news is. The name is borrowed from electronics, where you power a repaired board on and see if it burns. is the fast sanity pass that runs right after a deploy. A deploy is new code going live, the act the next two lessons dissect.

It asks only the most basic questions:

• Does the site load?
• Can a user log in?
• Does one checkout succeed?

The point is not thoroughness; the deeper tests have already run. The point is to learn within minutes whether the newly deployed software is alive at all.

The name comes from hardware, where the first test of a repaired device is to plug it in and watch for smoke.

What are performance and load testing?

Performance testingConcept · lights on your mapperformance testingMeasuring response times and resource use under normal conditions, speed rather than correctness. Its louder cousin is load testing. and load testingConcept · lights on your mapload testingPointing fake traffic at the system, at peak volume and beyond, to find where it slows or falls over before real customers do. A question of capacity, not correctness; the way Black Friday gets rehearsed on a quiet weekday. ask a question no other test asks. Every other test asks “is the code correct?” These two ask “is it still fast, and still standing, with a crowd inside?”

The two names are close cousins. Performance testing measures how the system behaves under normal conditions, meaning how quickly it responds and how much memory and compute the work consumes.

Load testing turns up the pressure. Bots simulate the dinner rush, or ten times it, while the team watches Module 10’s numbers: latency, throughput, saturation.

The rehearsal found a real limit (checkout collapses at 3× peak) while nobody was in line.

You may also hear stress testing, a recognition term for now: deliberately pushing the simulated crowd past any realistic peak, just to find the point where the system breaks and to watch how it fails.

What is security testing?

Security testingConcept · lights on your mapsecurity testingAttacking your own product on purpose, from scanning tools through to paid experts trying to break in (penetration testing), so weaknesses are found by you rather than by strangers. What those weaknesses look like is Module 14’s subject. attacks your own system on purpose. Attackers will eventually probe the product whether or not anyone invites them, so the goal is for your own team to find the weaknesses first.

It comes in escalating forms:

• Automated scanners sweeping for known vulnerabilities
• Audits of Module 3’s dependencies for packages with published flaws
• At the serious end, hired professionals attempting real break-ins — a practice called penetration testing

Module 14 will show you what all of them are looking for.

Cluster three: who signs off

The machines have now verified everything they can. In the feature’s final week before launch, three gates remain, each one more human than the last.

What is quality assurance?

Quality assuranceConcept · lights on your mapquality assuranceQA: the discipline (and often the team) responsible for verifying software works before users meet it. Designing test plans, hunting edge cases, owning the “is this ready?” question. In automated-heavy teams, QA engineers increasingly build the test machinery itself. (QA) is the discipline (and at many companies, the actual team) that owns the question “is this ready?”

QA people:

• Design test plans — written lists of scenarios to walk through before launch, from ordinary checkouts to weird carts on tiny screens
• Hunt the edge cases the feature’s builders were too close to see
• On automation-heavy teams, increasingly build the testing machinery itself rather than clicking through the product by hand

What is acceptance testing?

Acceptance testingConcept · lights on your mapacceptance testingTaking the ticket’s checklist of what “done” means (Module 11) and checking off each item against the working feature. Turns done from something the builder claims into something observed. verifies the feature against its acceptance criteria, the checklist written on the ticket back in Module 11, now executed line by line. Time slots appear at checkout, the chosen slot saves with the order, a sold-out slot cannot be selected.

It turns done-ness from a claim into a verified fact.

What is UAT?

Last comes UATConcept · lights on your mapUATUser acceptance testing: the people who asked for the feature use it themselves, usually on staging, and put their name to it. Acceptance testing asks “does it match the ticket?”; UAT asks “is this what we meant?” Often the final sign-off before release., user acceptance testing, which hands the wheel to the people the feature was built for. The actual stakeholders or customers try it themselves and formally sign off.

The difference from acceptance testing is subtle but worth keeping. Acceptance testing checks the feature against the written criteria; UAT checks it against the stakeholder’s actual intent. It catches the case where the team built exactly what the ticket said, and the ticket was not quite what anyone meant.

In practice the vocabulary is loose. UAT is technically one form of acceptance testing, and some organizations use the terms interchangeably. The distinction here is the useful conversational default.

Here is the final week, with all three gates on it:

When your future self is asked to “do UAT on the delivery feature,” that is you in Dana’s seat, clicking through the feature on staging with a checklist in hand. Your signature means this is what we asked for.

The mental model to remember

What the tests run against: test data is the invented-but-realistic world; mocks and stubs are the stand-in fakes for slow or dangerous dependencies; coverage measures how much of the code the tests reach, a flashlight for finding dark corners, never a trophy.

The special patrols: regression testing re-verifies every old promise on every change; smoke testing asks “is it alive?” minutes after a deploy; performance and load testing rehearse the crowd before it arrives; security testing attacks your own system before strangers do.

Who signs off: QA owns “is this ready?”; acceptance testing verifies the ticket’s written criteria; UAT verifies the stakeholder’s actual intent, with a signature. Three gates, each more human than the last.

You should now be able to read the launch checklist from the top of this lesson, “pyramid green, smoke passed on staging, load tested at 3× peak, UAT signed off”, and know exactly what each line verified, who did it, and which distinct risk it retired.