The graveyard of QA is full of abandoned automation suites. Every one of them was built with good intentions. Most of them reached an inflection point — typically around 6–18 months in — where the maintenance cost exceeded the value, and the decision was quietly made to stop running them. Sometimes a defect they should have caught made it to production shortly thereafter. Usually the suite was blamed. Rarely was the suite the actual problem.
The failure modes are well understood. The solutions are not complicated. The reason the same mistakes recur is that automation suites are often built by engineers who are excellent developers but haven't thought carefully about what makes tests maintainable over time. This article is about what to do differently.
Why automation suites die
Three causes account for the overwhelming majority of abandoned suites:
Fragile selectors. Tests written against CSS classes, XPaths that traverse 14 DOM levels, or visual positioning ("the third button in the second row") break every time a front-end developer breathes near the markup. When 40% of your suite fails after a routine UI refactor that introduced zero functional changes, the suite becomes noise. Teams start ignoring failures. Ignoring failures is worse than having no tests — it creates false confidence.
No ownership. Someone built the suite. That someone then changed teams, left the company, or got pulled onto other work. Nobody else understands the test data setup, the environment assumptions, or the test case reasoning. Tests start failing for unclear reasons. Nobody has time to investigate. The suite slowly dies of neglect.
Wrong layer coverage. Teams frequently over-invest in end-to-end UI tests and under-invest in API and unit-level tests. E2E tests are slow, brittle, and expensive to maintain. The testing pyramid suggests — and ISTQB's Technical Test Analyst guidance agrees ISTQB ATTA — that most test automation should sit at the unit and integration layers, with E2E covering only the critical journeys that must work end-to-end.
Start with the right layer
Before writing a single test, answer this question: what is the cheapest level at which this behaviour can be verified? If a calculation is wrong, a unit test will find it faster, more reliably, and more cheaply than an E2E test that navigates through five screens to reach the calculation output. If an API contract is broken, an API test will find it before a UI test can even render.
The ISTQB testing pyramid isn't just a diagram — it's a cost model. Unit tests are cheap to write, fast to run, and stable. Integration tests are more expensive but still fast and stable when written correctly. E2E tests are expensive, slow, and brittle by nature. Your automation effort should be inversely proportional to cost: many unit tests, fewer integration tests, a handful of E2E tests covering the scenarios that genuinely need end-to-end validation.
If you have 500 end-to-end tests and 50 unit tests, your pyramid is upside down and your suite will collapse under its own weight.
Selector strategy: the foundation of stability
This is the single highest-leverage decision in UI automation. Every test in your E2E suite is anchored to some element in the DOM. The resilience of that anchor determines the suite's longevity.
The selector hierarchy, from most resilient to least:
data-testidattributes — purpose-built for testing, survive visual redesigns, survive class renaming, survive DOM restructuring- ARIA roles and labels (
getByRole('button', {name: 'Submit'})) — semantically meaningful, tied to accessibility attributes that don't change capriciously - Text content — fragile to copy changes, but acceptable for content that genuinely defines an element's identity
- CSS classes — dangerous unless you use BEM-style naming conventions that treat class names as stable contracts
- XPath — avoid unless absolutely unavoidable; XPath that traverses multiple DOM levels is a maintenance liability
The investment required to add data-testid attributes to your application is trivial. The stability improvement is substantial. If you're building an automation suite and your application doesn't have test IDs, negotiate for a sprint of instrumentation work before writing tests. It will save multiples of that time in future maintenance.
Playwright's built-in locator strategies prioritise accessibility attributes and test IDs, making it naturally aligned with resilient selector strategy. Its auto-waiting mechanism also eliminates a category of flakiness caused by timing issues in less sophisticated frameworks.
Structure: Page Object Model and beyond
The Page Object Model (POM) is a design pattern that encapsulates page-specific logic — selectors, interactions, and assertions — in a dedicated object, keeping it out of test cases. ISTQB ATTA It's the most widely used automation architecture pattern for good reason: when a page changes, you update the page object, and all tests using that page object automatically benefit.
POM is the right starting point. It has limitations at scale. When applications grow complex, page objects can become bloated and tests become dependent on a sprawling inheritance hierarchy. More modern approaches — component objects, the Screenplay pattern, or simple functional helpers — can offer better modularity.
The principle underneath all of them is the same: separate what you're testing from how you're interacting with the UI. Test cases should read like user behaviour ("user logs in, navigates to checkout, applies promo code, completes purchase"). Page objects or helpers handle the "how to click the login button" mechanics. Keep these concerns separate and your tests will survive redesigns.
Test data: the silent killer
Many suites that appear to fail due to flakiness are actually failing due to test data problems. Test A creates a user. Test B expects that user not to exist. They run in parallel. Something breaks. Nobody can reproduce it locally. It's labelled "flaky" and ignored.
A robust test data strategy requires three things:
- Test isolation: each test should set up its own preconditions and clean up after itself. Tests that rely on state left by previous tests are a maintenance disaster.
- Seeded or generated data: use factory patterns or database seeding to create test data programmatically rather than relying on manually created accounts or records that may be modified or deleted.
- Environment separation: your test suite should be able to run against any environment without manual configuration. Connection strings, usernames, and URLs should be environment variables, not hard-coded.
Ownership and maintenance
The operational discipline around automation is as important as the technical quality. Every suite needs a named owner — not a team ("the QA team owns it"), a specific person who is accountable for its health. When tests fail, someone investigates within 24 hours. When new features ship, someone updates the relevant tests. When the suite's failure rate exceeds a threshold, someone escalates.
Schedule maintenance cycles. Automation suites require regular refactoring — removing obsolete tests, updating selectors, consolidating duplicated logic. If maintenance never happens, entropy accumulates until the suite is more hindrance than help.
CI/CD integration done right
The whole point of automation is that it runs continuously. A suite that requires manual triggering provides a fraction of the value of one that runs on every commit. The integration requirements are usually straightforward: a Docker container or test environment, the relevant credentials as CI secrets, a test run command, and a result publishing step.
Critical decisions: what runs on every PR (fast tests: unit, integration, smoke E2E) versus what runs on schedule or at release (full E2E regression). A full E2E suite that takes 45 minutes is fine as a nightly run. It's not acceptable as a PR gate. Design your suite with execution time in mind — parallel execution and layered triggering keep the CI feedback loop fast.
The testing pyramid is still right
Fifteen years after Mike Cohn described it, the testing pyramid remains the most useful mental model for automation strategy. ISTQB FL Fast, cheap, stable tests at the bottom. Slow, expensive, brittle tests at the top. Invest proportionally.
The suites that survive are the ones where this discipline was applied from the start — not retrofitted after the suite had already accumulated 400 E2E tests that take two hours to run and fail 15% of the time for environmental reasons. Build it right once. Maintain it consistently. The value compounds over time.
References: ISTQB Foundation Level Syllabus v4.0; ISTQB Advanced Technical Test Analyst Syllabus; Playwright documentation; "Succeeding with Agile" — Mike Cohn.