Flaky tests don't get fixed with waits

A test that fails "sometimes" is telling you something true. There is a race somewhere — in the test, in the application, or in the data. Adding a wait doesn't resolve the race. It just tells the test to be quiet about it.

The awkward part is that both approaches produce the same visible outcome. The pipeline goes green. Everyone moves on. And "I added a wait" and "I found the collision" look nearly identical in a diff.

The routine I actually run

1. Run it alone, twenty times

Before anything else, find out whether the test is flaky on its own or only in company. Playwright makes this a one-liner:

npx playwright test login.spec.ts --repeat-each=20 --workers=1

Green twenty times solo but red in the pipeline? Then it probably isn't the test. It's a neighbour. That single distinction saves more time than anything else in this list, because it tells you whether to look inside the file or outside it.

2. Run the suite and watch who touches what

Most of the time — for me, anyway — it's shared state. Two tests quietly using the same test user, the same record, the same account, the same port. One of them mutates something the other is reading, and which one gets there first depends on machine load.

The usual suspects, roughly in order of how often I find them:

Shared test data. Two specs using the same login, or the same order ID, or the same seeded row.
Order dependency. The test only passes if another test ran first and left something behind. This one hides beautifully until someone enables sharding.
Leftover state. A previous run's records still sitting in the database, so a "create" assertion finds two results instead of one.
Real application races. The one worth celebrating, because you just found a genuine bug that users will hit and nobody else caught.

Run with --workers=1 and compare. If serial is stable and parallel is not, you're looking at contention, not timing.

3. Read the trace instead of guessing

Guessing is the expensive part of this work. Playwright's trace viewer removes most of the need for it:

// playwright.config.ts
export default defineConfig({
  retries: 1,
  use: { trace: 'retain-on-failure' },
});

Then open the trace from the failed run and step through it. You get the DOM at each action, the network activity, and the exact moment the locator resolved to something unexpected. Nine times out of ten the answer is visible rather than inferred.

4. Fix the cause, not the symptom

What that means in practice:

Isolate the data. Every spec creates what it needs and cleans up after itself. Unique emails, unique record IDs, generated at runtime rather than hard-coded.
Use web-first assertions. await expect(locator).toBeVisible() retries until the condition is true or the timeout expires. That's not the same thing as a sleep — it waits for a state, not a duration.
Make locators intent-based. getByRole and getByTestId survive a redesign. A brittle CSS chain that depends on layout will break on a Tuesday for no visible reason.
Don't slow the test to hide the problem. If a test only passes when it's slow, the race is still there and it will surface on a loaded CI machine.

5. Rerun fifty times before you believe it

npx playwright test login.spec.ts --repeat-each=50

This is the step people skip, and it's the one that separates a fix from a coincidence. A flaky test that fails one run in twenty will look fixed if you run it three times.

sleep(5000) also makes a test green. That is precisely what makes it dangerous.

Why any of this matters beyond the suite

A suite people don't trust is worse than no suite. Once a team learns that red sometimes means nothing, they stop reading the failures — and the one real regression arrives dressed identically to the noise. Every unexplained flake spends a little of that trust.

There's a career angle too, and it's the reason I keep writing about this. I interview SDET candidates, and "tell me about the last flaky test you fixed" is my favourite opening question, because you can't prepare for it. Either you have the story or you don't.

The weak version I hear often: it was failing sometimes, so I added a wait. The strong version sounds completely different — reproduced it locally, checked whether it failed alone or only in parallel, found two tests sharing a user, fixed the data isolation, then ran it fifty times to be sure.

Same tools on the resume. Very different engineers in the room.

If you're making the move from manual testing into automation, the 90-day roadmap covers where this fits — fixing one flaky test properly is a day 61–90 milestone, and it's the story worth having ready.