Puppeteer drives the browser. Pie tests the product.
Test your web app without building a framework first. Pie’s agents explore it, write the tests and report what they find.
Pie is an agentic testing platform with the tests, the runs and the reports included. Agents explore your web, iOS and Android apps, so there’s no framework to build first.
Puppeteer is a free, open-source JavaScript library that controls Chrome or Firefox. For testing, your team adds a test runner, assertions and the scripts.
Pie vs Puppeteer at a glance
- What it is
- An agentic testing platform for web, iOS and Android apps.
- Who writes the tests
- Pie’s agents explore your app and generate them. Your team adds the cases that matter in plain English.
- What a test is
- Plain-English steps and checks. No scripts to write, no locators to break.
- Where it runs
- Web apps in a browser. iOS apps on the iOS Simulator, Android apps on the Android Emulator.
- AI help
- The whole product. Agents explore, write and run the tests, and group what they find into issues.
- What it is
- A free, open-source JavaScript library that controls Chrome or Firefox.
- Who writes the tests
- Your team, in JavaScript or TypeScript, with a test runner and assertions it picks.
- What a test is
- A script of selectors and browser commands your team keeps in its repo.
- Where it runs
- In Chrome or Firefox, on your machine or your CI. Phones are emulated in the desktop browser.
- AI help
- Experimental WebMCP support. A separate project, Chrome DevTools MCP, lets coding agents drive Chrome.
One refund check, written twice.
On the right is a real Pie test from a demo billing app we test with. It opens a partly refunded invoice and follows the refund to its credit memo. On the left is the same check as a Puppeteer script, our translation, using Puppeteer’s locators and Node’s built-in test runner.
import { test } from 'node:test';
import assert from 'node:assert/strict';
import puppeteer from 'puppeteer';
test('partly refunded invoice links to its credit memo', async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto(process.env.BASE_URL + '/login');
await page.locator('#email').fill(process.env.QA_EMAIL);
await page.locator('#password').fill(process.env.QA_PASSWORD);
await page.locator('::-p-text(Sign in)').click();
await page.locator('::-p-text(Configure)').click();
await page.locator('::-p-text(Payments)').click();
await page.locator('::-p-text(Invoices)').click();
await page.waitForSelector('h1');
await page.locator('::-p-text(Filters)').click();
await page.select('#refunded', 'partly-refunded');
await page.locator('::-p-text(Apply filters)').click();
const status = await page.$eval(
'[data-testid="invoice-status"]', (el) => el.textContent);
assert.equal(status, 'Partly refunded');
await page.locator('::-p-text(Open)').click();
await page.waitForSelector('::-p-text(Refund amount)');
await page.locator(
'[aria-label="Credit memos against this invoice"] ::-p-text(Open)'
).click();
await page.waitForSelector('::-p-text(Payout method)');
await page.waitForSelector('::-p-text(Refund issued)');
await page.goBack();
await browser.close();
});
16 selectors, plus a test runner and assertions your team brings.
Pie test- Enter the account email and password, then tap ‘Sign in’.
- Tap ‘Configure’ on any institute from the instance configuration list to enter its management view.
- Tap the ‘Payments’ dropdown in the sidebar and select ‘Invoices’.
- Tap the ‘Filters’ button, select ‘Partly refunded’ from the Refunded dropdown, and tap ‘Apply filters’.
- Tap ‘Open’ on the first invoice record in the filtered list.
- Scroll down to the ‘Credit memos against this invoice’ section and tap ‘Open’ on the associated credit memo.
- Tap the back button or navigation link to return to the customer’s payments overview.
- The Invoices view is displayed upon selection from the Payments menu.
- The invoice list is filtered to show only invoices with a ‘Partly refunded’ status after applying the filter.
- The detailed view of the selected invoice displays the refund amount and a list of associated credit memos.
- The credit memo details show the payout method and the date the refund was issued.
No selectors. Seven steps anyone on your team could review.
The selector stops matching, the run goes red, and someone edits the script. Pie re-reads the screen on each run, so a renamed label isn’t a selector edit for your team.
The Pie test is copied word for word from Pie, “tap” and all. The Puppeteer script is ours, written to match it step for step.
Four reasons to stop testing with Puppeteer scripts.
- 01
A library is the first piece of a test suite
Puppeteer’s docs call it a library that gives you an API to control Chrome or Firefox. A test suite also needs a runner, assertions, reports and CI, and your team assembles those. Pie runs, reports and checks pull requests out of the box.
Source - 02
Every script is yours to repair
With Puppeteer, your team writes every script and repairs it when a selector stops matching. In one study, flaky tests alone took at least 2.5% of a team’s productive time. Pie’s tests have no selectors to repair.
Source - 03
Unscripted flows go untested
A flow nobody scripted is a flow nobody tests. Pie’s agents explore your running app and write tests for the flows they find, and your team adds the cases only it knows.
- 04
A phone-sized browser isn’t a phone app
Puppeteer’s phone support is emulation. It sets a device’s screen size and user agent in a desktop browser. Pie tests native iOS and Android apps as well as your web app, with API and accessibility testing in beta.
Source
A browser library next to a testing platform.
Puppeteer gives you an API to drive Chrome. Pie gives you the tests, the runs and the reports.
| What you’re comparing | Pie | Puppeteer | Advantage |
|---|---|---|---|
| Cost to run it | No scripts to write and no locators to break, so your team spends those hours on your product. | Your team’s hours to build the test setup, write each script and repair it, plus your own CI. | Pie |
| Writing a new test | Agents generate tests by exploring your app. Anyone on your team can add one in plain English. | Your team writes it in JavaScript or TypeScript, with a runner and assertions it chose. | Pie |
| When the UI changes | No locators to update. Pie re-reads the screen on every run. | Selectors live in your scripts. When one stops matching, someone edits the file. | Pie |
| Native iOS and Android | Built in, on the iOS Simulator and the Android Emulator. | Phone screens are emulated in a desktop browser, by screen size and user agent. | Pie |
| Reading a failure | Pie groups related findings into issues, and can open a Fix PR in a repo you connect. Merging it is always your team’s call. | Debug with a visible browser, slow motion and console logs. Reports come from the runner you add. | Pie |
| Pull requests | Pie can test your pull requests on a preview URL or a build, and comments on the PR with a GO / NO-GO verdict. | Scripts run in your CI job, through the runner you pick. | Both |
| License cost | Custom pricing, tailored to your team’s requirements and testing volume. Contact Pie for a quote. | Free and open source, under the Apache 2.0 license. | Puppeteer |
| Browser automation beyond testing | Pie is a testing product. It tests your app the way a user meets it. | Screenshots, PDFs, crawling, request interception, performance traces and Chrome extension testing. | Puppeteer |
When the job isn’t testing
The job is automation, more than testing
Puppeteer generates screenshots and PDFs, crawls single-page apps and automates forms. Pie is a testing product, so those jobs stay with Puppeteer.
SourceYou need Chrome’s internals
Puppeteer talks to Chrome over the DevTools Protocol. It can intercept requests, record a performance trace and load a Chrome extension to test it. Pie tests what your users see and do.
SourceWhich fits your team, Pie or Puppeteer?
Pick Pie to
- Test your app without building a framework around a browser library.
- Test your native iOS and Android apps, not a phone-sized browser.
- Give your team back the hours it spends on selectors and flaky runs.
- Check your pull requests the way your users will use them.
- Get UI, API and accessibility testing without assembling a stack.
Pick Puppeteer if
- The job is PDFs, screenshots, crawling or other browser automation.
- You need Chrome’s internals, such as request interception or extension testing.
Switch from Puppeteer one flow at a time
No big rewrite. Your test scripts keep running until Pie covers the same flow.
- 1
Point Pie at your app
Give Pie your web app’s URL. If the app has a login, add a test login through a supported path.
- 2
Let Pie explore
Pie explores your app’s reachable flows and generates tests. You get results in hours, not weeks.
- 3
Add your edge cases
Type the cases only your team knows into Pie Chat in plain English. Pie explores each flow, drafts the test and runs it once you confirm.
- 4
Retire test scripts as Pie covers them
Compare Pie’s results with your Puppeteer runs, and delete each test script once Pie covers the flow. The scripts that make PDFs or crawl pages stay where they are.
FAQ
Did we miss a question?
Email us directly →
For testing, yes. Pie replaces the test scripts your team writes and maintains with tests its agents generate by exploring your app.
Stay on Puppeteer if the job is browser automation, such as PDFs, crawling or Chrome internals, more than testing.
Puppeteer’s docs describe it as “a JavaScript library which provides a high-level API to control Chrome or Firefox”. UI testing is one of its uses.
A test suite on Puppeteer also needs a test runner, assertions and reporting, which your team picks and wires together.
Yes. Puppeteer is open source under the Apache 2.0 license. The cost is your team’s time to write the scripts and keep them passing, plus the CI they run on.
In one study of about 30 developers over five years, flaky tests alone took “at least 2.5% of the productive developer time”.
Yes, and both are in beta. Pie runs whole API flows call by call, and checks each screen its tests reach for accessibility problems.
Your UI, API and accessibility testing sit in one place.
Getting Chrome to run where the tests run. Puppeteer’s own troubleshooting guide says, “Getting headless Chrome up and running in Docker can be tricky.”
Puppeteer is a browser automation library, so the tests, the runner and the reports are yours to build. In Pie they come with the product, and the agents write the tests.
Whoever owns quality today, such as a QA lead or an engineer who owns releases. Pie does the exploring, writing and re-running, and the owner decides what matters and what counts as a bug.
Anyone on the team, including a product manager, can read a test or add one in plain English.
Pie’s pricing is custom, tailored to your team’s requirements and testing volume.
You buy one product: agents explore your web, iOS and Android apps, write and run the tests, group related findings into issues and can open a Fix PR.
Book a demo, tell us what you ship and how often you release, and we’ll send you a quote.
Retire the test scripts. Keep shipping.
Book a demo and bring your web app’s URL. We’ll show you the tests Pie writes for it, with no runner to set up.
Sources
- Puppeteer docs. “a JavaScript library which provides a high-level API to control Chrome or Firefox”; listed uses include form automation, UI testing, screenshots and PDFs, and crawling single-page apps.pptr.dev/guides/what-is-puppeteer
- Puppeteer on GitHub. Apache 2.0 license; “JavaScript API for Chrome and Firefox”.github.com/puppeteer/puppeteer
- Puppeteer API, page.emulate. Emulates a device’s screen metrics and user agent; a shortcut for setting the user agent and the viewport.pptr.dev/api/puppeteer.page.emulate
- Puppeteer docs, WebDriver BiDi. The default protocol for Firefox; features not yet supported over BiDi include accessibility, coverage and tracing.pptr.dev/webdriver-bidi
- Puppeteer guides. Request interception can abort, continue or answer each request; Chrome extensions can be loaded and tested.pptr.dev/guides/network-interception · pptr.dev/guides/chrome-extensions
- Puppeteer debugging guide. Headful mode, slow motion, DevTools, forwarded console events and protocol logs.pptr.dev/guides/debugging
- Puppeteer WebMCP guide, an experimental API that is “subject to change”. Chrome DevTools MCP is a separate project that “Uses puppeteer to automate actions in Chrome”.pptr.dev/guides/webmcp · github.com/ChromeDevTools/chrome-devtools-mcp
- Cost of Flaky Tests in CI, ICST 2024 industry track. Five years of history, about 30 developers, one commercial project; “at least 2.5% of the productive developer time”. Not specific to Puppeteer.conf.researchr.org/details/icst-2024/icst-2024-industry/1
- Puppeteer docs, Troubleshooting. “Getting headless Chrome up and running in Docker can be tricky.”pptr.dev/troubleshooting