Why the Two-Decade-Old Testing Pyramid Still Works

Automated testing (AT) isn’t the knight in shining armor that protects project quality. It’s an approach to it that does. A well-structured automation strategy secures cost-effectiveness, unloads teams, and improves testing accuracy. And the Testing Pyramid might be a good way to kickstart your AT.

The Testing Pyramid is a framework that guides the distribution of various test types across development. You can use this model to organize or polish your test automation.

Overview of the Testing Pyramid

Before exploring the practical applications of the pyramid, let’s recap the logic behind its structure.

The Original Testing Pyramid: Unit – Integration – UI

The Testing Pyramid offers an organized testing hierarchy. It reflects the placement and number of tests at each level. It’s divided into three layers:

1. Unit tests.
2. Integration tests.
3. User interface (UI) tests.

These represent distinct testing objectives and focus areas. And the pyramid’s shape emphasizes the “volume” of tests needed at each layer:

1. Unit tests come first as they’re easy and quick to execute. You can dedicate many resources to them and find smaller issues that may be harder to fix later.
2. Integration tests are more complex and relatively slower. You ought to have fewer of them but target relevant aspects.
3. User interface tests are the most intricate and difficult to run. You should have only a few of those and prioritize user-critical elements.

The Base: Unit Tests

Unit tests are the foundational layer of the Testing Pyramid, checking isolated software components. Their purpose is to verify proper code performance. The point of such a large chunk dedicated to unit tests isn’t to reach 100% coverage. It’s to put more effort here to not struggle later.

Think about it. When you find lots of tiny errors early, bigger issues will be less likely to appear. The pyramid’s diminishing form precisely demonstrates this approach.

Yet, it doesn’t mean to go all in on unit tests. To be productive, they don’t really need to check every element. They just need to execute a lot of code. In other words, unit tests are to:

• Target a single code unit.
• Execute quickly.
• Present consistent results.
• Have predictable outcomes.

Here’s what you can do to ensure the above:

1. Use descriptive test names.
2. Test one thing at a time.
3. Write small, focused tests.
4. Check edge cases and boundaries.
5. Use the Arrange-Act-Assert* pattern.
6. Avoid if-else and switch-case statements.
7. Use assertions that target a test’s purpose.
8. Test positive and negative scenarios.
9. Avoid duplicating test code.
10. Keep the tests updated.

*Arrange-Act-Assert pattern:

• Arrange – set up preconditions and inputs.
• Act – perform the action.
• Assert – verify the outcomes.

The Middle Layer: Integration Tests

Integration tests assess the interactions and data exchange between internal or external systems. They verify that various elements can communicate without issues.

The pyramid places these in the middle, smaller-scale, layer. Since integrations are more intricate than unit tests, you shouldn’t have too many. Instead, increase their productivity by:

• Determining what to test and why.
• Preparing a testing plan with test data, acceptance criteria, and testing methods.
• Testing regularly.
• Monitoring the results to prevent mistakes.

You could also focus on high-priority points for integration scenarios. For example:

• A user adds new products to their cart. Here, you prioritize confirming that the app correctly reflects updates as it communicates with e-commerce databases.
• A customer tries to log in with incorrect credentials. In this case, you evaluate error handling to prevent usability or security issues.
• A consumer tries to pay for their purchase. Accordingly, you assess how multiple services interact, centering on payment, inventory, and shipping.

The Apex: UI Tests

At the top of the pyramid are UI tests. They concentrate on the users’ point of view, securing consumer value. These tests simulate user interactions to verify correct UI behavior.

They occupy the top layer (the smallest) due to the difficulties in their execution:

• They tackle complex web components and scenarios.
• An app’s interface changes often, meaning frequent UI tests’ rework.
• UI tests are the hardest to maintain, and more.

So, by having fewer UI tests that address key user flows, you can produce a quality app and save time and effort. Here’re some pointers on how to maximize them:

1. Create clear and descriptive test names.
2. Use page object model (POM) to denote UI interactions and elements.
3. Rely on data factories or generation methods to minimize test data dependencies.
4. Implement test automation frameworks to simplify scripting.
5. Parameterize UI tests to cover scenarios with varying inputs.
6. Implement data cleanup procedures.
7. Review and refactor tests to keep them up-to-date.
8. Monitor test results to identify potential app problems.
9. Use version control for test code for better collaboration.
10. Implement peer reviews to improve test quality.

Iterations of the Testing Pyramid

The Testing Pyramid didn’t stay the same for long. Its iterations appeared to accommodate the developing testing landscape. Each version adapts the original form, upgrading aspects deemed unrealistic.

Let’s discuss the four primary alterations, explaining the changes.

Unit – Integration – E2E

Why is the apex now for end-to-end (E2E) tests? Because the UI changes all the time. Frequent alterations lead to more testing, increasing the resources spent. With E2E tests at the top, QA engineers can check user-oriented aspects that aren’t impacted by visual shifts. So, you get an in-depth app analysis and prevent extra efforts.

E2E testing examines complete workflows, from start (e.g., user registration) to finish (e.g., receiving a purchase). It replicates authentic user interactions, ensuring a successful consumer journey. So, unlike UI, E2E tests offer a holistic perspective of an app in real-world scenarios.

Unit – Integration – UI & API – Manual & Exploratory

This model adds APIs to the middle, as UI tests aren’t always needed. For instance, your app may not have a web UI or be too intricate. Given that UI tests are brittle, you could check APIs instead. They offer decent coverage without investing or compromising much.

Exploratory and manual software testing take the top layer. Why? Because they prioritize user-centricity and can uncover missed issues. For instance, automated testing won’t do much for usability or accessibility. But manual and exploratory tests bring you a step closer to excellence.

Unit – Integration – Contract (API) – UI – E2E – Acceptance

In software development, delegating tasks to different teams is common. So, you need to ensure that separately created modules have no contradictions. Contract tests focus precisely on that. This add-on layer secures consistency between autonomously developed parts.

UI and E2E tests are split due to their scopes. In a way, this division lets you pick. Remember how the two previous models added APIs and E2E (basically allowing to skip UI)? With this iteration, you can tailor the testing structure to your project needs.

Acceptance tests verify that the system corresponds to both business and user expectations. They offer a balanced view of software functionality and value. So now they occupy the apex, serving as a link between higher- and lower-level tests (E2E and unit).

Unit – Acceptance – Integration – Monitoring & Alerts – Smoke – Manual

In this iteration, acceptance tests move down the pyramid. They verify that you’re building the right product. And the sooner you ensure it, the better. In effect, early acceptance testing acts as the gatekeeper before engaging integrations. If you’ve been thorough with unit checks, acceptance testing will validate it, and you can confidently move on.

Monitoring and alerts motivate a proactive approach. With these, you track issues and resolve them promptly before conducting more complex evaluations.

Smoke tests quickly validate system functionality, emphasizing key user paths. They’re non-intrusive and limited in number. So, implementing them early in the development benefits the testing phase and the end product.

As automation works with expected outcomes, manual tests here prioritize exploration to get the most value out of them. Going off script lets you fully investigate the system. This extra layer isn’t critical. But it can show you how users may interact with an app “in the wild”.

The Testing Pyramid in Practice

Referring to the previous section, you might ask, “why did the pyramid change at all?” The answer to this question lies in the unrealistic view of the testing process. For instance, its original form does not account for the unique needs of different projects.

We’ll start with the model’s advantages and go over its limitations next.

Pros

• The pyramid’s hierarchy helps teams organize their testing efforts.
• It encourages a balanced distribution of tests, leveling-out test coverage.
• Unit tests’ fast execution lets developers receive feedback on their code quickly.
• Running unit and integration tests frequently helps catch issues early, minimizing error hoarding.
• The resilience and simplicity of unit tests mean reduced maintenance and targeted efforts.

Cons

• The Testing Pyramid stretches the value of unit testing. While unit tests are essential, overstressing them can lead to gaps in test coverage.
• The model also raises UI tests above all else. But they’re more fragile and often break under change.
• The focus on technical aspects doesn’t capture the consumer perspective. Real-world scenarios and user behaviors call for more intricate approaches.
• The pyramid omits the worth of manual testing for usability.
• It oversimplifies the testing process and might not be applicable to some projects.

Striking the Right Balance

You may not find a practical application of the pyramid as is. But its principles can be your guide. So, it’s more about balancing the tests and approaches to satisfy your testing needs.

• Determine the areas of your app that need extensive testing and focus your efforts on them.
• Prioritize tests with increased value for defect detection. For example, avoid complicated tests if they don’t contribute much to the overall quality.
• Identify high-risk aspects, for example, vital functions or security, and concentrate your testing there.
• Maintain a short feedback loop to prevent technical debt.
• Support automated testing with user-centric approaches, for example, usability and beta testing. This way, you’ll secure positive user experience.

Most importantly, consider the unique characteristics of your project. Tailor your testing strategy to align with them. For instance, your team may be more proficient in particular approaches. In that case, let them do what they know best to maximize testing value.

Testing Pyramid’s Evolutions

So, if you can bend and stretch the Testing Pyramid, why is it still here? Simple. It still works. You can build up your testing based on its original shape and tap into the models that emerged from it.

Diamond

In the Test Diamond, the testing volume shifts. It allocates most effort to the integration layer, trying to balance out the effort and functionality.

This model prioritizes integration tests as they secure business-critical features. Unit and E2E tests take roughly the same amount of effort – enough to cover the essentials (and a bit more), not too much to get lost in them.

The Diamond is best suited to projects where system integrations matter most, for example, data transformation, APIs, and databases.

Ice Cream Cone

The Testing Ice Cream Cone is an inverted pyramid with a scoop on top. It’s the only model that spotlights manual tests. The cone admits the importance of the end-user experience, promoting user-centric testing.

Since the model encourages adopting a user perspective, it considers unit tests the least useful. It motivates you to test only the essentials and give it your all when it comes to UX.

The Ice Cream Cone suits projects with user satisfaction and usability as priorities. It’s also a good fit for legacy systems, prototypes, MVPs, and small projects where you need to rely on manual QA.

Trophy

The Testing Trophy is a well-rounded approach to testing. It strives to optimize the resources put into tests and their value for the product.

At the bottom are static tests that catch typos in the code. They are cheap enough to run in real time and can eliminate basic errors. With them, you have a stable base for the application.

Like the Diamond, the Trophy gives greater significance to integrations. You don’t need many tests to find real issues, and they’re fast enough to cover bigger chunks of code.

The Trophy offers a good balance between cost and benefits. It’s flexible, fitting various projects, such as those with Agile/DevOps environments, complex business logic, and evolving requirements.

Crab

The Test Crab model introduces a lateral approach to testing. It prefers parallel testing levels to hierarchy. The crab has component, API, and unit tests as equals. They prop up the principal piece – functional and visual E2E testing.

The model promotes rational division of testing efforts to detect issues faster. It also motivates developers and QA engineers to work together, securing better quality. The crab is the only one to recognize the importance of “good looks” for users, hence visual E2E tests.

Projects with high agility and extensive parallel testing could benefit from the Test Crab. It’s also excellent for products where user experience comes first.

Best Practices for Using the Testing Pyramid

You didn’t come all this way to end up with a bunch of alternatives to the Testing Pyramid, did you? Don’t worry. As we noted before, the original model has plenty of uses, in Agile, too (which we’ll discuss later). So now, let’s figure out how you can make the pyramid work for you.

Embrace a Risk-Based Approach

Apply thorough unit and integration tests to high-risk areas. For example, you’d want to secure critical functionality or frequently changing code. And elements like static content or peripheral features will be just fine with fewer or manual tests.

Place your automation efforts where they matter most.

Avoid Test Duplication

Tests at different levels shouldn’t check the same components. Consider an E2E test executing functionality already covered by unit or integration tests. You get time and resources spent on the same thing twice.

Structure your tests and test data to prevent duplication.

Write Clean Test Code

Keep your test code clean and easy to understand. It’ll be easier to identify and fix issues. You’ll also see fewer confusions with detailed test names and descriptions.

Follow coding conventions and avoid unnecessary complexity in your test code.

Set Up the Deployment Pipeline for Fast Feedback

Set up the deployment process to run tests automatically when code changes. With quick testing iterations, you can catch and resolve errors faster.

Apply continuous integration/delivery practices to streamline testing and deployment.

Maintain a Consistent Test Environment

Create a test environment that closely mirrors production. So, keep an eye on configurations and software dependencies. Using identical data will also help you get consistent results.

Automate the setup and teardown to minimize variability.

Refactor and Optimize Tests

Prioritize regular refactoring and optimization for your test code. You might need to adjust the tests as your app evolves. Regular reviews and updates will keep them relevant and accurate.

Routinely revise and remove redundant or obsolete tests.

Testing Pyramid in Agile

You might think that the pyramid’s lack of flexibility is the polar opposite of Agile. But Mike Cohn created the model specifically to support it. So, the Testing Pyramid’s concept, in fact, empowers agile methodologies. Here’s how.

Better Time Management – Increased Team Productivity

Agile teams thrive on efficient time management. And the pyramid provides a clear framework for distributing testing efforts. This optimization saves time, leading to higher team productivity.

Test-Driven Development (TDD) – Better Code Quality

TDD is a natural companion to Agile development. By writing tests before code, you can improve code quality and avoid regressions. The pyramid’s focus on unit tests echoes this approach.

Clear Structure – Easier Prioritization

The Testing Pyramid offers a risk-based approach to testing. It lets teams prioritize tests based on the risks associated with different test types. High-risk areas get more attention, and supplementary components receive fewer checks.

Automation Roadmap – Reduction of QA Expenses

Agile’s iterative nature means frequent testing. It’s great for continuous improvement, but not so much for the budget. The pyramid implies automation for cost-effectiveness. The model clearly outlines what you need to automate to not strain the capital or the team – extensive, time-consuming tests.

Early Defect Detection – Shorter Time-to-Market

The Testing Pyramid’s structure helps with early defect discovery. It also encourages keeping your software stable during development. With quick issue patching and system consistency, you’re more likely to deliver a product faster. You can also adapt to shifting user expectations swiftly.

And the best thing about using the Testing Pyramid in Agile is that you can build it up. By adding new layers and restructuring the rigid form, you can tailor it to your needs and maximize its integral values.

To Sum Up

The Testing Pyramid is an exceptional concept. It’s not perfect, sure. But the amount of experts that got behind its idea and developed it to better serve QA engineers is inspiring. You can use the original pyramid to better organize your testing. And you can modify it to fit your project needs.

So, in a way, the Testing Pyramid is your clay. Shape your ideal testing strategy with it.