How-To

How to Test Two-Factor Authentication (2FA) and OTP Flows End-to-End Without Flaky Tests

A one-time passcode is built to defeat automation. Here is how we test 2FA and OTP flows end-to-end reliably. Stop reading the code off the screen and fetch it at its source.

Adithya Aggarwal

CTO & Co-founder at Pie

11 min read Posted Jul 23, 2026

A time-based one-time passcode changes every 30 seconds by design. It exists to defeat anything that is not a human holding a phone. Which is exactly why it is the single most reliable way to make an end-to-end suite flake.

The fix is not a smarter wait or a better selector but a change of source. Stop reading the code off a screen or waiting for a real text message, and go get the code where it is actually created. Generate the TOTP yourself, pull it from a programmable test inbox, or read it from a backend test hook. Here is what we learned wiring this up across dozens of login flows.

One Code That Breaks Your Whole Suite

The verification step is usually the flakiest line in an end-to-end suite. Every test that needs an authenticated user has to get past it, so one unreliable two-factor step multiplies across the entire suite. One racy code entry, replicated in front of two hundred tests, and your green build becomes a coin flip.

A time-based one-time password, defined in RFC 6238, is a code computed from a shared secret and the current time, and it rotates on a fixed interval the spec recommends setting to 30 seconds. In practice almost every implementation renders it as the 6 digits an authenticator app shows. An emailed or texted code depends on a delivery channel with a variable delay you do not control. Both properties are the point of the mechanism, and both are poison for a deterministic test. The step is not flaky because your framework is weak. A one-time passcode is engineered to be unpredictable, and a test needs the opposite.

Why OTP Is Built to Break Automation

An OTP is a deliberate barrier against automated access. Every property that makes it good security is precisely what breaks a test:

• Time-bound: A TOTP is valid for one 30-second window. Enter it a beat late, or run on a CI machine whose clock has drifted, and a code that was correct is now rejected. Clock skew between the test runner and the auth server is a real and intermittent cause.
• Out-of-band: Email and SMS codes travel through a channel your test does not own. Delivery can take one second or fifteen, so any fixed sleep is either too short (flake) or wastefully long (slow suite).
• Single-use: A retry that reuses a consumed code fails, so the naive “just retry the step” pattern that papers over other flakiness makes OTP worse, not better.

These are not edge cases you can harden your way out of at the UI layer. They are the design goals of the feature. That is why the durable fix does not touch the UI at all, and why teams keep chasing this in their flaky test backlog without ever closing it.

Why the Human Approach Keeps Flaking

The instinct is to make the test act more like a person, opening a real inbox, waiting for the text, reading the digits, typing them in. We tried the polite version of this and it fails in the ways you would predict:

1. A hardcoded wait races the clock: await sleep(5000) before reading the code passes on a fast day and fails on a slow one.
2. A shared mailbox crosses wires: Parallel test workers polling one personal inbox grab each other’s codes.
3. A real device chains CI to a carrier: Reading an SMS off a phone ties every run to carrier delivery and a physical handset.

The deeper mistake is running this gauntlet on every test. Prove the login UI works once, then give every other test an authenticated session without touching the verification screen. The frameworks already agree. Cypress recommends caching a programmatic login with cy.session(), and Playwright reuses a saved storageState so the browser starts signed in. One honest UI test for the full 2FA path, and 2FA deleted from the rest.

Fetch the Code at Its Source

Every reliable 2FA test follows the same rule. Stop observing the code and go produce or retrieve it at the source. A human reads the code off a phone because that is the only place they can get it. A test is not stuck with that, because the code is generated somewhere it can reach directly. Instead of racing the delivery channel, go around it.

There are three sources, ranked by how much of the real system they exercise, and each turns the OTP step into one that passes every run:

1. Generate the TOTP yourself from the shared secret. No channel, no wait, no drift. The test computes the exact code the authenticator would show.
2. Retrieve the code from a programmable test inbox that exposes an API, so the wait is on an event you own rather than a carrier you don’t.
3. Read the issued code from a backend test hook that skips the channel entirely. Nothing is delivered, so nothing can flake.

Three sources, four techniques, because retrieval covers both email and SMS. The next four sections are the core of this guide and wire up each technique in order, with working code and the trade-off you accept in return.

1. Generate the TOTP Code Yourself

For authenticator-app 2FA, you do not need the authenticator app. A TOTP code is a pure function of the shared secret and the current time, so if your test knows the secret, it can compute the exact 6-digit code the app would display. When the user enrolls in 2FA, your app generates a Base32 secret and shows it as a QR code. Save that secret for your test user in an environment variable, and any RFC 6238 library reproduces the code locally with zero delivery delay.

In Node, otplib does this in one call. The generated code is valid for the same 30-second window as the real authenticator, so compute it immediately before you fill the field:

import { authenticator } from 'otplib';

// The same Base32 secret your app stored when this test user enrolled.
// Keep it in CI secrets, never in the repo.
const code = authenticator.generate(process.env.TEST_TOTP_SECRET);
await page.getByLabel('Verification code').fill(code);
await page.getByRole('button', { name: 'Verify' }).click();

Python teams use pyotp the same way: pyotp.TOTP(os.environ["TEST_TOTP_SECRET"]).now(). One caveat matches the time-bound failure from earlier. Generate the code in the final fraction of a window and it can expire between generation and submission. Guard against it by checking authenticator.timeRemaining() and waiting for the next window when it is under a couple of seconds. This is the closest thing to a free lunch here, because it needs no external service and tests the real verification logic.

2. Read an Email Code From a Test Inbox

An emailed second factor changes the channel, not the rule. Read the code from a programmable inbox like MailSlurp or Mailosaur that exposes an API, never from a shared human mailbox that parallel workers fight over. Each test claims its own disposable address, waits on the message event rather than a fixed sleep, and pulls the six digits out of the body:

// One inbox per test. Wait on the message event, not a timer.
const email = await mailslurp.waitForLatestEmail(inbox.id, 30_000);
const code = email.body.match(/\b\d{6}\b/)[0];
await page.getByLabel('Email code').fill(code);

For a magic link instead of a code, extract the URL with a regex and navigate straight to it rather than clicking through a rendered email. The email-delivery architecture underneath this (isolated inboxes, the polling loop that kills the sleep, and how the hosted and self-hosted inbox tools compare) is the same one that drives password reset, so rather than repeat it here, the full treatment lives in the guide on testing email verification and password reset flows. For 2FA, an emailed code is simply one more source you can query, not a screen you have to race.

3. SMS OTP Without a Real Phone

SMS is the messiest channel, so lean on your provider’s test tooling instead of a physical device. If you send codes through Twilio, Twilio test credentials and magic phone numbers let you exercise the send path without touching a real carrier or getting charged. Authenticating with test credentials means Twilio never connects to real phone numbers, and magic numbers such as +15005550006 return predefined outcomes your assertions can rely on.

To test the receive side, provision one real Twilio number that your test suite owns, point the test user’s phone number at it, and have the test poll the Twilio Messages API for the newest inbound message rather than reading a screen. The test reads the provider’s API, not a device screen. That said, real SMS is slow and occasionally lossy even in staging, so reserve the live SMS test for a single smoke check and use TOTP or a backend hook for the bulk of coverage. This is a recurring theme in mobile testing automation, where the flakiest steps are almost always the ones that leave the app and depend on an outside system.

4. The Most Reliable Option: A Backend Test Hook

The fastest and least flaky approach removes the message channel from the test entirely. In your non-production environments, expose a small, guarded endpoint that either returns the last verification code issued to a given test user, or accepts one fixed code for accounts explicitly flagged as test users. The test asks the backend what code it just generated, submits it, and moves on. There is no message in transit and no clock window to race.

This is the same shape as the test-only control plane we described for subscription testing, a narrow contract that your test environment honors and production does not. A minimal version is a couple of routes:

// Enabled only when ALLOW_TEST_HOOKS is true, which is never in production.
if (process.env.ALLOW_TEST_HOOKS === 'true') {
  // Return the most recent code issued to a test user.
  app.get('/test/last-otp', (req, res) => res.json({ code: lastCodeFor(req.query.user) }));
}

One thing the hook will never do is test your real email or SMS delivery. Pair it with one real-channel test that catches a broken template or a misconfigured provider, and use the hook for the hundreds of tests that only need to get past the gate. The reliability difference is not subtle, and it is why this pattern anchors most serious CI/CD auth suites.

Which Approach to Use

Match the technique to the 2FA method under test and to how much of the real delivery path you need to cover. Use a backend hook or TOTP for the bulk of tests, plus one real-channel test to keep delivery honest.

Keeping the Bypass Out of Production

Every technique here is safe as long as the bypass cannot reach production. A test hook or a fixed test code is a security hole only when a real user can trigger it, so the entire question is containment.

Three controls cover it:

1. Gate the hook behind an environment flag such as ALLOW_TEST_HOOKS that is never true in a production config, and default it to off so a missing variable fails closed.
2. Scope every bypass to accounts explicitly marked as test users in the database, so even an enabled hook does nothing for a real account.
3. Fail the build if the flag reaches production: A startup assertion or CI check enforces the guardrail through the pipeline rather than through memory.

Done this way, production authentication is byte-for-byte unchanged, and the OWASP Authentication guidance on not weakening real auth paths still holds, because you never touched them.

How Pie Handles Verification Steps

The wiring above is the easy half. The hard half is everything around the OTP field. The verification screen moves, the input splits into six boxes, the resend button changes, and the selectors you wrote last sprint break this sprint. This churn is what rots a scripted suite, and it is the problem Pie was built for. Pie is an autonomous QA platform that drives your iOS, Android, and web app the way a person would, identifying the verification screen visually instead of by a brittle selector, so a redesign of the 2FA UI does not break the test.

For the code itself, Pie does not intercept a live SMS or an authenticator code. It works from a source you control:

• A static test code on the test account, entered like any other step.
• A 2FA-disabled test user for the flows that only need a session.
• A test inbox attached as a credential, so Pie reads the emailed code itself.
• A registered execution script that computes the TOTP or calls your backend hook.

Pie runs the verification step as part of the autonomous flow and enters the code, so you are not hand-maintaining that plumbing per test. Both of the expensive parts of 2FA testing, the unpredictable code and the shifting UI around it, come off your plate with no one on selector duty. That is the difference between a suite that survives an auth redesign and one that goes red the morning after it ships.

The One Rule

Two-factor tests flake because the code is built to be unpredictable, and humans cope by reading it off a screen. A test has better options. Generate the TOTP from the shared secret, pull the code from a programmable inbox, or read it from a guarded backend hook, and the flakiest line in your suite becomes deterministic.

Keep one real UI test for the full 2FA path, give every other test its session without touching verification, and never read a code off a screen you can go around. Pie handles the shifting UI around that code too, so verification keeps passing even after the auth team redesigns it.

Adithya Aggarwal

CTO & Co-founder at Pie

Eight years building search and delivery systems at Amazon. The kind of scale where flaky tests block billion-dollar releases. Now CTO at Pie, building AI agents that adapt when your UI changes. LinkedIn →