Playwright is free. Your test suite isn’t.
Ship without writing, reviewing or repairing another spec. Pie’s agents explore your web, iOS and Android apps and write the tests for you.
Pie is an agentic testing platform whose agents explore your web, iOS and Android apps and write the tests, leaving no spec to write and no locator to repair.
Playwright is an open-source framework for web testing and browser automation. Your team writes the tests in TypeScript, Python, .NET or Java and maintains them.
Pie vs Playwright at a glance
- What it is
- An agentic testing platform for web, iOS and Android apps.
- Who writes the tests
- Pie’s agents explore your app and generate them. Your team adds the cases that matter in plain English.
- What a test is
- Plain-English steps and checks. No scripts to write, no locators to break.
- Where it runs
- Web apps in a browser. iOS apps on the iOS Simulator, Android apps on the Android Emulator.
- AI help
- The whole product. Agents explore, write and run the tests, and group what they find into issues.
- What it is
- An open-source framework for web testing and browser automation, under the Apache 2.0 license.
- Who writes the tests
- Your team, in TypeScript, Python, .NET or Java.
- What a test is
- Code that finds elements with locators and checks them with assertions.
- Where it runs
- Chromium, Firefox and WebKit, on Windows, macOS and Linux, locally or in CI.
- AI help
- Planner, generator and healer agents that write and repair Playwright tests, plus an MCP server.
One refund check, written twice.
On the right is a real Pie test from a demo billing app we test with. It opens a partly refunded invoice and follows the refund to its credit memo. On the left is the same check as a Playwright spec, our translation, written with the user-facing locators Playwright’s docs recommend.
import { test, expect } from '@playwright/test';
test('partly refunded invoice links to its credit memo', async ({ page }) => {
await page.goto('/login');
await page.getByLabel('Email').fill(process.env.QA_EMAIL!);
await page.getByLabel('Password').fill(process.env.QA_PASSWORD!);
await page.getByRole('button', { name: 'Sign in' }).click();
await page.getByRole('link', { name: 'Configure' }).first().click();
await page.getByRole('button', { name: 'Payments' }).click();
await page.getByRole('link', { name: 'Invoices' }).click();
await expect(page.getByRole('heading', { name: 'Invoices' }))
.toBeVisible();
await page.getByRole('button', { name: 'Filters' }).click();
await page.getByLabel('Refunded').selectOption('Partly refunded');
await page.getByRole('button', { name: 'Apply filters' }).click();
await expect(page.getByTestId('invoice-status').first())
.toHaveText('Partly refunded');
await page.getByRole('link', { name: 'Open' }).first().click();
await expect(page.getByText('Refund amount')).toBeVisible();
const memos = page.getByRole('region', {
name: 'Credit memos against this invoice',
});
await memos.getByRole('link', { name: 'Open' }).first().click();
await expect(page.getByText('Payout method')).toBeVisible();
await expect(page.getByText('Refund issued')).toBeVisible();
await page.goBack();
}); 17 locators, and every one is code your team maintains.
Pie test- Enter the account email and password, then tap ‘Sign in’.
- Tap ‘Configure’ on any institute from the instance configuration list to enter its management view.
- Tap the ‘Payments’ dropdown in the sidebar and select ‘Invoices’.
- Tap the ‘Filters’ button, select ‘Partly refunded’ from the Refunded dropdown, and tap ‘Apply filters’.
- Tap ‘Open’ on the first invoice record in the filtered list.
- Scroll down to the ‘Credit memos against this invoice’ section and tap ‘Open’ on the associated credit memo.
- Tap the back button or navigation link to return to the customer’s payments overview.
- The Invoices view is displayed upon selection from the Payments menu.
- The invoice list is filtered to show only invoices with a ‘Partly refunded’ status after applying the filter.
- The detailed view of the selected invoice displays the refund amount and a list of associated credit memos.
- The credit memo details show the payout method and the date the refund was issued.
No locators. Seven steps anyone on your team could review.
The spec’s locator stops matching, the run goes red, and someone edits the file or asks the healer agent to. Pie re-reads the screen on each run, so a renamed label isn’t a locator edit for your team.
The Pie test is copied word for word from Pie, “tap” and all. The spec is ours, written to match it step for step.
Four reasons to stop writing specs.
- 01
Where the hours go
A Playwright suite is paid for in the hours your team spends writing specs and keeping locators current. In one study, flaky tests alone took at least 2.5% of a team’s productive time. Pie’s agents write and run your tests, so those hours go back into your product.
Source - 02
Who writes the tests
At Amazon, our CTO Adithya Aggarwal’s teams spent about six months on each test suite, then nobody maintained it and it got scrapped. Nobody likes writing tests, so Pie’s agents explore your app and generate them.
- 03
“But Playwright has agents now”
It does. They save typing, and your team still reviews, owns and maintains every line they write. Pie’s tests are plain-English steps, so you have no test code to maintain.
Source - 04
Past the browser
Playwright stops at the browser. A native app means a second framework and a second suite, the cost nobody prices at the start. Pie tests all three on one platform.
Source
Count the hours, not the license.
Playwright costs nothing to license and plenty to run.
| What you’re comparing | Pie | Playwright | Advantage |
|---|---|---|---|
| Cost to run it | No scripts to write and no locators to break, so your team spends those hours on your product. | Your team’s hours to write, review and repair specs, plus the CI to run them and a second framework for native apps. | Pie |
| Writing a new test | Agents generate tests by exploring your app. Anyone on your team can add one in plain English. | Someone on your team writes a spec in code, or reviews one the generator agent wrote. | Pie |
| When the UI changes | No locators to update. Pie re-reads the screen on every run. | A locator that stops matching fails the run until someone, or the healer agent, fixes it. | Pie |
| Native iOS and Android | Built in, on the iOS Simulator and the Android Emulator. | Not covered, apart from experimental Android support for Chrome and WebView. Pair it with Appium. | Pie |
| Fixing the bug | Pie can open a Fix PR with a proposed fix in a repo you connect. Merging it is always your team’s call. | The healer repairs tests. Fixing the bug in your app stays with your team. | Pie |
| Pull requests | Pie can test your pull requests on a preview URL or a build, and comments on the PR with a GO / NO-GO verdict. | Specs run inside your CI job on every commit. | Both |
| License cost | Custom pricing, tailored to your team’s requirements and testing volume. Contact Pie for a quote. | Free and open source. | Playwright |
Which fits your team, Pie or Playwright?
Pick Pie to
- Test more of your app without writing another spec.
- Cover web, iOS and Android on one platform.
- Give your team back the hours it spends repairing locators.
- Check your pull requests the way your users will use them.
- Test your UI, API and accessibility in one place.
Pick Playwright if
- Your engineers have the hours to write and maintain every spec.
Switch from Playwright one flow at a time
No big rewrite. Your specs keep running until Pie covers the same flow.
- 1
Send Pie the app
Give Pie a URL for your web app, or an iOS simulator build or an Android APK for mobile. If the app has a login, add a test login through a supported path.
- 2
Let Pie explore
Pie explores your app’s reachable flows and generates tests. You get results in hours, not weeks.
- 3
Add your edge cases
Type the cases only your team knows into Pie Chat in plain English. Pie turns them into tests and runs them with the ones it found.
- 4
Retire specs as Pie covers them
Compare Pie’s results with your Playwright runs, and delete each spec once Pie covers the flow. Connect GitHub or GitLab and Pie can test your pull requests too, given a preview URL or a build for each change.
FAQ
Did we miss a question?
Email us directly →
Yes. Pie replaces the specs your team writes and maintains with tests its agents generate by exploring your app.
Playwright’s license is free. Stay if your engineers have the hours to write and maintain every spec.
The license is. Playwright is open source under Apache 2.0. The running costs show up in engineer hours to write, review and repair specs, in the CI to run them, and in a second framework if you ship native apps.
In one study of about 30 developers over five years, flaky tests alone took “at least 2.5% of the productive developer time”, and half the organizations in the World Quality Report 2025-26 name maintenance burden and flaky scripts as a test automation challenge.
Pie’s agents write and run the tests, with no locators for your team to repair.
Yes. Playwright ships three test agents. A planner explores your app and writes a test plan, a generator turns the plan into Playwright tests, and a healer repairs failing tests.
They save typing, and your team still owns and maintains the code they write. Pie’s tests are plain-English steps that Pie runs against what’s on the screen.
Yes, and both are in beta.
Pie runs whole API flows call by call, and checks each screen its tests reach for accessibility problems.
Debugging and upkeep, more than the tool itself. G2 reviewers say working out why a test failed can take time, and that the code gets harder to maintain as a suite grows.
Either way a developer still writes and maintains every test. In Pie the agents do.
Whoever owns quality today, such as a QA lead or an engineer who owns releases. Pie does the exploring, writing and re-running, and the owner decides what matters and what counts as a bug.
Anyone on the team, including a product manager, can read a test or add one in plain English.
Pie’s pricing is custom, tailored to your team’s requirements and testing volume.
You buy one product: agents explore your web, iOS and Android apps, write and run the tests, group related findings into issues and can open a Fix PR.
Book a demo, tell us what you ship and how often you release, and we’ll send you a quote.
Your team has better things to fix.
Book a demo and bring the flow your suite skips. We’ll show you how Pie would cover it, and which specs you could retire first.
Sources
- Apache 2.0 license; “a framework for Web Testing and Automation”github.com/microsoft/playwright
- “Chromium, WebKit and Firefox on Windows, Linux and macOS, locally or in CI, headless or headed”playwright.dev/docs/intro
- TypeScript, Python, .NET and Java; one API for Chromium, Firefox and WebKitplaywright.dev
- Parallel by default; auto-waiting before actions; retrying assertions; Trace Viewerplaywright.dev/docs/test-parallel · /docs/actionability · /docs/trace-viewer
- Planner, generator and healer test agentsplaywright.dev/docs/test-agents
- “we recommend prioritizing user-facing attributes and explicit contracts”playwright.dev/docs/best-practices
- Codegen records your actions and generates testsplaywright.dev/docs/codegen-intro
- Device emulation; “experimental support for Android automation”playwright.dev/docs/emulation · /docs/api/class-android
- Cost of Flaky Tests in CI, ICST 2024 industry track. Five years of history, about 30 developers, one commercial project; “at least 2.5% of the productive developer time”. Not specific to Playwright.conf.researchr.org/details/icst-2024/icst-2024-industry/1
- World Quality Report 2025-26 (Capgemini, Sogeti, OpenText), Fig 15. 50% cite maintenance burden and flaky scripts as a test automation challenge.capgemini.com/insights/research-library/world-quality-report-2025-26
- G2, Playwright reviews. Reviewers cite time spent debugging failed tests and code that gets harder to maintain as a suite grows.g2.com/products/playwright/reviews