Blog / Mobile App Testing: The Complete 2026 Guide and Checklist
Guide

Mobile App Testing: The Complete 2026 Guide and Checklist

Web-testing habits quietly break on mobile. A practitioner's guide to testing across the device chaos and shipping without drowning in test maintenance.

Your test suite passes on Monday. By Friday, 40% of your automated tests are failing. Nobody changed the test code, and the app works fine when you tap through it by hand. Welcome to mobile testing.

Google publishes the bill for what slips through. Once 1.09% of your daily users hit a crash, or 8% on any single phone model, Play may cut your app’s visibility and warn shoppers on your store listing. Teams rarely get there by shipping careless code. They get there by testing mobile like web on a smaller screen, a habit that gets more expensive as AI assistants take on more of each commit.

What you’ll learn

  • How native, hybrid, and web apps change what you test
  • Ten mobile testing types and when each one earns its place
  • A five-step QA process with severity triage and the metrics that matter
  • E2E frameworks compared: Appium, Espresso, XCUITest, Detox, Maestro
  • How to test push notifications, deep links, payments, biometrics, and upgrades
  • Which mobile testing tool fits your stack, and what testing actually costs

What Is Mobile App Testing?

Mobile app testing is the process of validating that your application works correctly across devices, operating systems, and real-world conditions before users find the bugs for you. Unlike web testing, it has to account for hardware diversity, OS version sprawl, manufacturer customizations, and networks that change mid-session. Instead of one platform, you’re testing thousands of platform combinations at once. What you test also depends on what kind of app you shipped, and that choice changes everything downstream.

Native, Hybrid, and Web Apps Test Differently

  • Native apps. Written once per platform, in Swift or Objective-C on iOS and Kotlin or Java on Android. Full device-API access puts the whole hardware surface in scope, from camera and GPS to biometrics, background processing, and push. Two codebases mean two test suites, or one cross-platform suite that abstracts both. The iOS vs Android testing differences that force separate code are a short list.
  • Hybrid apps. A web view wrapped in a native shell (Ionic, Cordova, and similar). You inherit two failure surfaces plus the bridge between them, and the classic hybrid bug lives at that bridge, where a JavaScript call silently fails against a native API that changed.
  • Mobile web apps. Run in the browser, so the burden shifts to responsive layouts and browser engines instead of device APIs. Lighter to test, but you give up the hardware features that usually justify building an app at all.

React Native and Flutter apps sit between the categories. They compile and ship through the stores like native apps, but bring their own rendering and bridging quirks that need framework-aware tooling. We’ll come back to that in the E2E section.

Why Mobile Breaks Web Testing Assumptions

When you test a web app, you control the environment. The browser behaves predictably, the network is usually stable, and the screen size falls within a known range. Mobile breaks all three.

  • Device fragmentation. Thousands of Android device models are in active use across dozens of manufacturers, each combining its own screen size, pixel density, memory profile, and skin. Samsung’s One UI kills background processes more aggressively than stock Android. Xiaomi’s MIUI, now HyperOS, adds its own permission prompts. A Pixel 8 sails through a flow that crashes a Galaxy A12 at the same step. The full breakdown of why this wrecks test strategies is in our guide to mobile app testing challenges.
  • OS version sprawl. Apple’s numbers show why iOS teams sleep better. 86% of iPhones introduced in the last four years run iOS 26, as measured June 7, 2026. Android users scatter across five-plus major versions, each with different permission models, background limits, and API behaviors. Your app needs to work on all of them simultaneously.
  • Network variability. Users switch from 5G to 3G to WiFi to airplane mode mid-session. Payment flows fail halfway. Data syncs conflict. Offline mode never works the way you tested it, because you tested it on office WiFi.

10 Types of Mobile Testing (And When Each Matters)

Each testing type catches a different failure mode. Knowing when to reach for each is what separates real coverage from wasted effort.

1. Unit Testing

Unit tests verify individual functions and components in isolation. They run fast, catch logic errors early, and form the foundation of the pyramid. For mobile, they cover business logic, data transformations, and state management. They don’t touch UI or device behavior, and that’s by design. If you’re starting from zero, our unit testing guide covers the foundations.

2. Integration Testing

Integration tests check that components work together once they’re wired up. API calls return the data you expect, the local database persists what it should, and services talk to each other without errors. On mobile, integration testing earns its keep at the network boundary. What happens when the API times out on a train? When the response is malformed? When the user loses connectivity mid-request? Each answer is an integration test.

3. End-to-End (E2E) Testing

E2E tests simulate a complete user journey, from log-in through browsing and add-to-cart to checkout. They’re the most expensive tests you’ll run and the only ones that catch what users actually hit. A unit test won’t find the checkout button that disappears on a Galaxy S21 in dark mode. An E2E test will. E2E is also where flakiness, framework choice, and CI integration get hard, which is why it gets its own section below.

4. Visual Testing

Visual testing catches the UI regressions functional tests sail right past, like the button that moved three pixels, the font that rendered wrong, or the layout that breaks on notched screens. Pixel-by-pixel screenshot comparison drowns you in false positives from antialiasing and animation timing. Modern approaches use AI to flag meaningful visual changes while ignoring noise, which turns visual testing from a maintenance burden into a usable signal.

5. Performance Testing

Performance testing puts hard numbers on startup time, memory usage, battery drain, and behavior under load. Mobile makes this harder than web. You’re profiling on a constrained device that throttles itself when it heats up. An app that feels instant on your flagship can take several seconds to cold-start on the mid-range phone most of your users own.

6. Security Testing

Security testing hunts the vulnerabilities attackers look for, from insecure local storage and missing certificate pinning to hardcoded secrets and weak session handling. The practitioner standard is OWASP’s Mobile Application Security project, which organizes controls across eight categories from storage and cryptography to platform interaction and reverse-engineering resilience. Run its MASVS checklist against your app in CI instead of inventing your own definition of “secure.”

7. Usability Testing

Usability testing answers the one question automation can’t reach, whether real people can actually complete your flows. There is nothing to automate here. You watch users struggle with the onboarding screen you thought was obvious, and five sessions before a major release surface friction no automated suite can detect, because the suite already knows where the button is.

8. Compatibility Testing

Compatibility testing verifies your app behaves consistently across device models, OS versions, and screen configurations. In practice that means running your critical flows against a device matrix built from your analytics instead of assumptions. The question of which devices and how many is covered in the checklist below.

Where you run those flows matters as much as which devices you pick. Emulators and iOS simulators are fast and cheap, so they belong in the development loop and early UI checks, but they fake the hardware. They cannot reproduce real sensors, thermal throttling, or manufacturer skins like One UI.

Release validation belongs on physical hardware, your own or a rented device cloud, because that is the only place the hardware behavior your users hit shows up. The trade-offs between the three environments, and what each one catches, are laid out in real device vs emulator vs simulator.

9. Accessibility Testing

Accessibility testing puts your app in front of TalkBack on Android and VoiceOver on iOS, checks that it respects Dynamic Type, and holds touch targets and contrast to WCAG guidance. The payoff goes well past legal exposure. Accessibility bugs cluster in the same custom components that break for everyone else under edge conditions. Run your top three flows with a screen reader on before every major release.

10. Interruption Testing

Interruption testing checks how your app survives the chaos of living on a phone. Three kinds matter. The device itself interrupts when the battery dies or storage runs low. The network interrupts when the connection drops mid-checkout. And the OS interrupts when a call lands during payment or a permission dialog fires mid-gesture. A clean test lab simulates none of it, so script one interruption of each kind into every release pass.

How to Build a Mobile App QA Process

A list of testing types isn’t a strategy. Knowing that interruption testing exists doesn’t tell you who runs it, when, or what happens when it fails. The teams that ship reliably have answered five questions in order.

  1. Scope by risk, not by feature list. Map the flows where a bug costs real money or trust. Payments, authentication, data sync, and anything regulated get the deepest coverage. The settings screen’s dark-mode toggle does not. Write the risk map down, because it becomes the argument for every coverage decision that follows.

  2. Write a test plan that names the matrix, then turn it into real test cases. Which devices (from your analytics), which OS versions, which environments (staging, production-like sandbox), which test data. A plan that doesn’t name real device models is a wish. This is also where you decide the manual/automated split per flow, and where each flow becomes a concrete test case with clear preconditions, steps, and an expected result. Automation codifies whatever the case says, so a vague test case becomes a vague test that passes while the app is broken.

    A mobile case needs preconditions a web case never does, because the same steps can pass on one device and fail on another. Write each case with these fields:

    • Preconditions. Device model, OS version and manufacturer skin, app version, network state, account state, and permission state.
    • Steps. One action per step.
    • Expected result. Something you can see or measure, like the home screen within a stated time. A vague outcome like works correctly is not a result.
    • Example. On a mid-range Android phone with one item in the cart and the network on 4G, switch to airplane mode during checkout. Expected: the cart is intact and the app shows a retry prompt instead of a blank screen.
  3. Execute in layers. Unit and integration tests run on every commit. E2E regression runs on every release candidate, at minimum. If coding agents open pull requests against your app, testing AI-generated mobile apps means adding a layer before merge: a build of each pull request that changes a screen, tested on a simulator. Exploratory and usability sessions run on a cadence, not a trigger. The layering matters because feedback speed determines whether engineers act on failures or mute them.

  4. Triage defects by severity, not by who shouted. A working severity scale is boring and four levels deep. Blocker stops the release. High gets fixed before release without stopping it. Medium waits for the next cycle. Low goes to the backlog. Every bug gets a severity, an owner, and a decision at triage. Bugs without owners are how “known issues” become App Store reviews.

  5. Review with metrics that measure the process, not just the app. Four numbers tell you whether QA is working:

    • Defect escape rate measures bugs found in production versus bugs found before release. Rising escape rate means coverage is pointed at the wrong flows.
    • Mean time to detect (MTTD) measures how long a bug lives before you know about it. The gap between “introduced Monday” and “found Friday” is your real exposure window.
    • Mean time to resolve (MTTR) measures detection to deployed fix. Slow MTTR usually indicts your release process more than your engineers.
    • Flake rate measures the share of test failures that aren’t real bugs. Once flakes become routine, engineers stop trusting red builds, and then every other metric goes blind.

A note on team structure. Plenty of product companies run without a dedicated mobile QA department, and that’s fine. What matters is that ownership is explicit. Someone owns the risk map, someone owns the release decision, and engineers own the quality of what they ship. Adding headcount without explicit ownership just adds people to the confusion.

See the Process Run on Your App

Drop your staging build. We'll cover your real flows, no scripts to maintain.

Mobile E2E Testing: Frameworks, Flake, and CI

Everything in your QA process eventually funnels into the E2E layer, because end-to-end flows are where users live and where mobile testing gets hard. It deserves more than the paragraph most guides give it.

Black-Box vs. Gray-Box: The Distinction That Predicts Your Flake Rate

E2E frameworks split into two architectures, and the split predicts how stable your suite will be.

  • Black-box frameworks drive the app from outside, the way a user would, with no knowledge of its internal state. Flexible and often cross-platform, but blind. The framework can’t see whether a network call or background job is still running, so tests lean on timeouts and hope.
  • Gray-box frameworks hook into the app’s internals and synchronize with them. Espresso, Android’s native framework, waits before every action until the message queue is clear, no AsyncTask is running, and your idling resources report idle, which retires most of the sleep-and-pray pattern. Detox bills itself as a gray box end-to-end framework, built with React Native in mind, for the same reason. Synchronizing with the app’s internals is the design goal.

Five Frameworks Worth Comparing

The five mobile testing frameworks worth a serious look split along two axes, platform reach and architecture. Appium remains the cross-platform default, with one black-box API, every major language binding, and both web and native support. XCUITest is Apple’s native answer, fast and built into Xcode but limited to Apple platforms. It drives the app from a separate process through the accessibility layer, the black-box approach. Maestro is the newest of the set, an open-source declarative framework where flows are YAML files, built for fast smoke suites.

The choice inside each platform is narrower than it looks. On iOS you end up running XCUITest either way, directly or wrapped by a driver, which is the whole of XCUITest vs Appium. On Android, Appium can drive Espresso underneath rather than replace it, which is why Espresso vs Appium is less of an either-or than it sounds.

FrameworkPlatformsArchitectureBest Fit
AppiumiOS, Android, mobile webBlack-boxCross-platform suites, existing Selenium skills
EspressoAndroid onlyGray-boxAndroid-native teams who want built-in sync
XCUITestApple platforms onlyBlack-box, out of processiOS-native teams inside Xcode
DetoxReact NativeGray-boxRN apps that need sync-aware testing
MaestroiOS, Android, RN, FlutterBlack-box, declarativeFast-start smoke flows in YAML

Notice what every row has in common. A human writes the scripts, and a human fixes them when the UI changes. Keep that in mind for the next section.

Why Tests Pass Locally and Fail in CI

The most demoralizing pattern in mobile E2E is the one that refuses to reproduce. Green on your machine, red in the pipeline. Four causes account for most of it:

  • Cold emulators. A fresh CI emulator boots slower and stutters through animations, so timing-based waits expire.
  • Shared state. Suites running in parallel against one backend leak data between tests.
  • Timing drift. Animations and system dialogs land at different moments on different hardware.
  • Fixed sleeps. Time-based waits (sleep(3000) and friends) encode an assumption about speed that CI hardware refuses to honor.

The mitigations are unglamorous and they work:

  • Disable animations. Turn them off on test devices and emulators before anything else. Google’s Espresso setup guide recommends switching off the window, transition, and animator duration scales specifically to avoid flakiness.
  • Make test data hermetic. Every test creates what it needs and cleans up. Tests that share accounts fail in whatever order CI happens to run them.
  • Replace timers with synchronization. Espresso’s idling resources and Detox’s sync model exist precisely so tests wait for state instead of seconds.
  • Quarantine flaky tests. Move them into a separate, non-blocking lane and track them with a flake budget. Blind retries hide real bugs; quarantine makes the problem visible while protecting the release pipeline. We’ve written up the full pattern in our guide to flaky tests in CI/CD.

Do all of it well and a ceiling remains. Each redesign sends engineers back anyway, to fix tests that “broke” without a single bug behind them. That ceiling is why more teams have started looking past the scripted model altogether.

Manual, Automated, or Autonomous: Choosing Your Mix

Every framework above still assumes a human writes the test and a human repairs it. Whether that assumption should hold is now a choice, and the manual vs automated testing trade-off is worth settling deliberately instead of by default.

  • Manual testing wins where judgment matters, like exploratory sessions, usability checks, and “does this feel right” passes on new features. It stops scaling the moment you need the same few dozen flows verified on every release candidate. Repetitive verification is exactly what humans do worst on Friday afternoon.
  • Scripted automation wins on stable, high-value regression flows. Automate first where three things intersect. The flow rarely changes, a failure is expensive, and you re-test it constantly. Login, checkout, onboarding. The trap is the maintenance curve. Each script binds to a UI that keeps moving, and a suite that started as a shortcut slowly becomes the thing you spend the week maintaining. The suite becomes the product.
  • Autonomous testing changes what a test is. Instead of scripts bound to selectors, the platform explores your app the way a user would, generates tests from real flows, and adapts them when the UI changes. No locators to repair, no scripts to rewrite after a redesign. Fi, which makes smart GPS collars for dogs and cats, moved its mobile release testing onto our autonomous testing platform without reworking how its engineers ship, and the Fi case study has the before and after.

”The way that Pie set up has allowed Fi to work alongside our development process. We didn’t have to change how we did things.”

— Philip Hubert, Director of Mobile Engineering, Fi

Autonomous platforms are the wrong choice for teams that want line-by-line control of test code, and they don’t replace API-layer contract testing. The tradeoffs line up across four dimensions.

DimensionManualScripted AutomationPie
Test creationNone (human executes)Engineers write scriptsPlatform generates from real flows
Maintenance burdenNone, but execution never shrinksGrows with every UI changeSelf-healing, with a review after big redesigns
Best atExploratory, usability, judgmentStable revenue-critical regressionBroad regression at release speed
Breaks down whenFlows multiply past headcountUI changes outpace script repairYou need hand-written script control

The mix that holds up keeps manual judgment where it’s irreplaceable and a thin scripted layer where you want hand control, then lets autonomous coverage carry the regression load. Hand off first whichever layer eats the most repair time.

Mobile App Testing Tools and Real-Device Clouds

Frameworks decide how tests run. They still need hardware to run on and something to manage them. Mobile app testing tools split into four jobs, and most stacks buy the first three while underbuying the fourth. That imbalance is where the maintenance hours come from.

  1. Real-device clouds rent you the hardware. Buying and racking real phones does not scale. You rent instead.
    • BrowserStack advertises 30,000-plus real iOS and Android devices, enough to validate a mid-range Galaxy and a four-year-old iPhone you never bought.
    • Firebase Test Lab runs instrumented tests across Google’s device farm, but Google has deprecated it and will shut it down on September 30, 2027, pointing teams to the Developer Device Platform on Google Cloud.
    • AWS Device Farm gives the same real-hardware access from inside the AWS console.
    • When the pricing or device mix does not fit, we compared the field in our guide to BrowserStack alternatives.
  2. E2E frameworks drive the app. Appium, Espresso, XCUITest, Detox, and Maestro (compared above). All of them run hand-written tests that are yours to repair when the UI moves.
  3. Test-management platforms track the work. TestRail and Xray hold the cases, runs, and coverage. They do not execute anything.
  4. Autonomous platforms write and repair the tests. This is the job that decides whether the calendar fills with maintenance.

A low-code branch (mabl and similar) lowers the scripting cost instead of the hardware cost, and its auto-healing patches locators inside the selector model, but someone on your team authors every flow. The device clouds have moved as well.

BrowserStack, Sauce Labs, and TestMu AI now each sell AI test authoring on top of the hardware. Treat it as a purchase separate from the devices, and judge it on who fixes the test when the screen changes. Rent tens of thousands of devices for a brittle suite and you have only multiplied the surface that can flake.

CategoryWhat it doesExamplesWhat it doesn’t solve
Real-device cloudRuns tests on real hardware you don’t ownBrowserStack, Firebase Test Lab (shutting down 2027), AWS Device Farm, TestMu AI (formerly LambdaTest)Writing and repairing the tests, unless you buy their AI authoring separately
E2E frameworkDrives the app like a userAppium, Espresso, XCUITest, MaestroReal-device access, test maintenance
Test managementTracks cases, runs, and coverageTestRail, XrayActually executing the tests
Autonomous platformGenerates and self-heals tests from real flowsPieLine-by-line control of hand-written scripts

That last column is the one we built Pie to close. Its agents generate the regression suite from your real flows, and because it drives the app by what’s on screen instead of by locators, a moved button leaves no locator to patch. It runs native apps on iOS simulators and Android emulators, with real devices available on demand. What comes off your team’s plate is writing and repairing the suite.

What Is the Best Mobile App Testing Tool?

There is no single best mobile app testing tool. The right tool depends on where your cost lands. Teams buried in device coverage need a real-device cloud. Teams buried in test upkeep need a platform that writes and repairs its own tests. Pick for your bottleneck, not the longest feature list.

Match the tool to the pain that is slowing the team down:

  1. Device coverage is the pain. You ship fine, and then bugs surface on models nobody touched. Rent hardware. A new framework won’t fix this, and a real-device cloud will.
  2. Setup and upkeep is the pain. Appium hands you everything and makes you wire all of it. If that tax is what hurts, weigh the Appium alternatives built to remove the setup rather than relocate it.
  3. Regression breaks on every device refresh. Selector-bound suites shatter when the UI or the OS shifts underneath them. The durable fix is mobile regression testing that identifies elements by sight.
  4. Writing and maintaining tests is the pain. Coverage that survives UI churn comes from mobile app automated testing on a platform that generates the suite and heals it, so a redesign stops meaning a week of repair.

Look, the best tool is the one that removes your biggest recurring cost. For most product teams shipping weekly, that cost is maintenance, and renting more devices does nothing to shrink it.

What Mobile App Testing Costs

Mobile app testing costs land in three buckets, and teams usually budget for the first two while the third quietly eats the calendar.

  • Devices. Real-device clouds bill per parallel session or per device minute. AWS Device Farm prices an unmetered device slot at $250 a month, and slots cap how many tests run at once, so ten in parallel is $2,500 a month before anyone writes a test.
  • People. Testers, in-house or through a crowdtesting marketplace, bill by the hour, the test cycle, or the bug.
  • Maintenance. Engineering time spent repairing tests that fail while the app itself works fine. No quote mentions it because no vendor invoices it, and a script tied to a selector becomes a line item every time the UI moves, which on mobile means across every device and OS version at once.

You can model that third bill against your own suite with our test maintenance cost calculator before the next redesign sends it to you.

Stop Paying for Broken Tests

Bring your flakiest flow. Watch Pie test it without a script.

How to Test the Hard Mobile Features

The process and frameworks above organize the work. What follows is the part most guides skip entirely. These are the features practitioners keep searching for help on, because they’re miserable to test.

How to Test Push Notifications

  • Cover all three app states. Foreground, background, and killed. The notification path differs in each.
  • Verify the payload and the tap. It renders correctly, routes to the right screen with the right state, and nothing crashes when it arrives mid-flow.
  • Use the test channels. iOS delivers to development builds through the APNs sandbox; Android’s FCM test messages target specific registration tokens.
  • Catch the classic bug. A tap that opens the app cold and drops the user on the home screen instead of the promised content.
  • Test every entry state. Cold start (app not running), warm start (backgrounded), app not installed (does the store fallback work?), and logged-out (does the link survive the login wall and still land correctly after authentication?).
  • Mind the platform quirks. Universal Links on iOS and App Links on Android verify differently, and both silently fall back to the browser when domain verification breaks.
  • Suspect verification first. A deep link that works in your dev build and dies in production is almost always a verification or routing-table problem.

How to Test In-App Purchases

  • Never use production billing. Apple’s sandbox environment runs on Sandbox Apple Accounts, and by default a one-month subscription renews every five minutes, so a year of renewals fits in an hour. Google’s Play Billing testing guide gives license testers test payment methods, including one that simulates a declined card.
  • Cover the ugly paths. Interrupted purchase mid-transaction, restore purchases on a new device, subscription expiry and renewal, and refund handling.
  • Expect scrutiny here. Apple’s App Review Guideline 3.1.1 tells you to ship a restore mechanism for any restorable in-app purchase, so a broken Restore button can cost you a review cycle on top of the bug.

How to Test Permissions and System Interruptions

  • Test all three permission states. Granted, denied, and revoked-later-in-Settings. The third is where apps crash, because nobody re-checks a permission granted at onboarding.
  • Layer in interruptions. Fire an incoming call during checkout, trigger a system dialog mid-gesture, and fill device storage to see what your cache writes do.
  • Target aggressive Android skins. Test Samsung and Xiaomi battery optimizers specifically; they kill background work that stock Android allows.

How to Test Biometric Authentication

  • Use the simulators. Xcode simulates matching and non-matching Face ID; the Android emulator simulates fingerprint touches.
  • Test the full fallback chain. Success, failure, lockout after repeated failures, fallback to passcode, and the no-biometrics-enrolled state.
  • Don’t forget the opt-out. A user disables Face ID in Settings, and your next launch must degrade gracefully instead of looping on a permission it will never get.

How to Test App Upgrades and Data Migration

Fresh installs lie to you. Real users upgrade over existing data, and that path is where databases corrupt and preferences vanish. The test is simple and rarely run.

  1. Install the previous production version.
  2. Generate real state (accounts, cached data, in-progress flows).
  3. Upgrade to the release candidate over the top.
  4. Verify everything survived.

Automate the app upgrade test per release candidate, from the previous release and the oldest version with a meaningful share of installs, and you’ll catch migration bugs that would otherwise become your worst App Store review week.

Mobile App Testing Checklist for 2026

Run this list against every release candidate. It compresses everything above into the order most teams should execute. Items 18 to 21 are store and SDK rules with hard dates, so check them before the build goes to review.

  1. Build your device matrix from your own analytics, weighted by session volume and crash rate.
  2. Cover at least one device per major manufacturer you serve, including one mid-range Android model.
  3. Run unit tests on business logic with every commit.
  4. Verify API contracts, timeouts, and malformed-response handling with integration tests.
  5. Automate E2E regression for revenue-critical flows first, starting with login, onboarding, and checkout.
  6. Disable animations on all test devices and emulators.
  7. Make test data hermetic so parallel runs never share state.
  8. Run visual checks across screen sizes, notches, and dark mode.
  9. Test under degraded networks, including throttled bandwidth, packet loss, and airplane-mode transitions.
  10. Verify offline behavior and sync-conflict resolution after reconnect.
  11. Test push notifications in foreground, background, and killed states.
  12. Validate deep links from cold start, logged-out state, and app-not-installed fallback.
  13. Run purchase flows in Apple’s sandbox and with Google Play license testers.
  14. Check your top three flows with TalkBack and VoiceOver enabled.
  15. Audit against OWASP MASVS categories and track flake rate, quarantining unstable tests.
  16. Confirm privacy labels and permission-purpose strings match what the app actually collects and does.
  17. Configure a staged rollout so a regression caught in production can be halted before it reaches everyone.
  18. Confirm Android builds target Android 16 (API level 36) or higher, which Google Play has required for new apps and app updates since August 31, 2026.
  19. Test on a 16 KB page-size emulator image. From February 1, 2027, Google Play blocks updates for apps targeting Android 15 or higher that don’t support 16 KB page sizes on 64-bit devices.
  20. Build iOS releases with Xcode 26 or later, which App Store Connect has required for uploads since April 28, 2026.
  21. Declare approved reasons for required-reason APIs and include privacy manifests for third-party SDKs, both required for App Store Connect uploads since May 1, 2024.

Twenty-one items looks heavy. The first time through, it is. The teams that run it every release are the ones whose crash dashboards stay boring.

Mobile Testing, Easy as Pie

Go back to the opening. The app worked all week while that suite went from green to red. Every script was welded to a UI that moved, and a person had to notice, diagnose, and repair each one by hand.

Pie automates the part of that loop that burns hours and needs no judgment, and leaves the rest to your team.

  • Autonomous test generation. Point Pie at your app and its agents explore it on their own, mapping the flows and generating a working suite before anyone writes a test.
  • Plain-English test authoring. Describe a flow the way you’d say it out loud, like log in, add an item, and check out, and Pie turns it into a test it runs and maintains.
  • Self-healing tests. When a button moves or its label changes, Pie can find it again from its context and position on the next run. That cuts the red builds that routine UI changes cause, though a big redesign still needs a review.
  • Vision-based execution. Pie drives the app by sight, tapping what it sees the way a person does, so there are no selectors to break the moment the layout moves.
  • Fixes, not just findings. Pie Loop can test your team’s pull requests once it’s set up for a repo, and when it catches a regression it can draft the fix as a pull request for an engineer to review and merge.

None of this replaces your testers. It buys back the hours they used to lose to scripting and upkeep and puts that time where it belongs, on the exploratory passes, usability calls, and release decisions that only a human can make.

Ship Mobile Apps Without the Testing Nightmare

Calm release weeks don’t come from hiring more testers or writing more scripts. Every script you add is tied to a UI that changes weekly, and you pay for that coupling in maintenance that never made the budget.

What works is the groundwork this guide lays out. Scope by risk, build the device matrix from your own analytics, synchronize on app state, and stop paying engineers to repair tests that fail only because the screen moved.

We built Pie to take that last job off your team. Android will keep fragmenting and your designers will keep tearing up checkout, and self-healing tests can absorb routine churn before it reaches your engineers. Whether the next redesign costs you a lost sprint or a quiet afternoon of review is your call now, and Pie is how you aim for the quiet one.

See Pie on Your App

Drop your staging build. See results in hours, not weeks.

Frequently Asked Questions

Test in layers against a risk map. Scope the flows where a bug costs money or trust, build the device matrix from your analytics, run unit tests on every commit and E2E regression on every release candidate, and validate on real hardware, since emulators miss manufacturer skins, sensors, and throttling.
Start from your own analytics, not industry averages. Rank devices by session volume and crash rate, then test the top of that list on real hardware. A short, well-chosen list usually covers the bulk of your sessions, so add models only when your analytics show a gap.
Automate the stable, revenue-critical regression flows first, starting with login, onboarding, checkout or subscription, and your core feature loop. These flows change rarely, break expensively, and get re-tested before every release. Keep exploratory and usability testing manual.
Four causes cover most of it. Cold CI emulators boot slower than your warm local one, animations and dialogs land at different moments, parallel runs leak shared state, and fixed sleeps assume a speed CI won't match. Disable animations, isolate test data, and wait on app state instead of seconds.
Yes. Autonomous testing platforms explore the app the way a user would, generate tests from the flows they find, and adapt them as the UI changes. There are no selectors to write. You still choose which flows matter and review results after a big redesign, but the scripting goes away.
You need someone who owns the risk map and the release decision, not a large team writing and maintaining scripts. Plenty of product companies run without a dedicated mobile QA department. That works when ownership is explicit and autonomous testing carries the repetitive regression work, leaving people the judgment calls.

Device clouds bill per parallel session or device minute, and testers bill by the hour, cycle, or bug, so a small team spends a few hundred to several thousand dollars a month.

The bigger cost is engineering time repairing tests that broke without a bug, and no invoice shows it.

Yes. A GitHub Action can upload each new build for Pie to test, Pie Loop can test pull requests on GitHub or GitLab once it's set up for the repo, and scheduled runs cover the rest. What changes is everything after the trigger, since Pie generates and maintains the tests instead of your engineers.
Dhaval Shreyas
Dhaval Shreyas
CEO & Co-founder at Pie

13 years building mobile infrastructure at Square, Facebook, and Instacart. Now building the QA platform he wished existed the whole time. LinkedIn →