Software is becoming more complex and multi-layered. It seems that there is always a way to improve a product by adding more features, building new subsystems, and integrating with existing third-party apps and systems. While creating a more advanced software product, we turn it to a more challenging to test and support.
A minor malfunction can cause a chain reaction breaking the entire flow that engages several layers of an application. Thus, it becomes critical to provide testing coverage at and through all the levels, checking how a flow that passes through several functionalities works. In simple words, QA engineers need to test features from start to finish. That’s what end-to-end testing allows us to do.
End-to-End Testing: Definition and Particularities
End-to-end testing (or E2E testing) is a methodology used to test software functionality and performance under circumstances that replicate live settings and data. The goal of E2E testing is to check real user scenarios from start to finish.
Since end-to-end testing implies vast coverage of business-critical features, using test automation is a good idea. Covering the frequently repeated workflows will leave the team more time for exploratory testing.
Benefits of End-to-End Testing:
- End-to-end testing provides wide coverage. By setting to check a single workflow (for example, purchasing an item), we test several levels of an application and interactions between them (in particular, UI, functionality, API, and payment gateway).
- E2E testing allows us to see how different subsystems work in tandem. Thus, a development team gets a good understanding of how a product will behave in the live setting, where users don’t evaluate UI, compatibility, and other QA aspects separately.
- It is suitable for testing applications with complex architecture effectively. By checking a particular scenario from start to finish, we can engage numerous subsystems into action and understand at what layer a problem appears.
Challenges of End-to-End Testing:
- Test design and maintenance may turn into a laborious process since end-to-end testing implies simulating complete user journeys. Close study of the requirements and being specific about mechanics is a must.
- E2E testing can be resource-consuming for the abovementioned reason. Thus, a QA engineer should prioritize the functionality to cover with E2E testing smartly. It can be complicated in case communication isn’t set up well or there’s a lack of transparency in the team.
- As a user journey entails several functionalities, it might be difficult to find the defect root cause when the test fails. But that’s more of an issue for developers who are to fix the problem.
E2E Testing Methods
There are two E2E testing methods — vertical and horizontal testing. Both are used to check a complete flow using different approaches.
In horizontal testing, QA engineers check a particular user flow from start to finish, employing and investigating different layers of the system. For example, when a software tester simulates an item purchase from entering a website to completing a payment, they check a little bit of functionality and performance, interact with UI and API, etc.
In vertical testing, QA engineers check a particular level of application architecture testing each subsystem fully but independently from other subsystems. For example, a software tester starts working with API tests and later on moves to UI tests.
Both methods are targeted at the same goal and work best in combination since each has its prerequisites and provides specific benefits based on that.
Stages of E2E Testing
- Studying specifications.
- Test environment setup.
- Requirements review.
- Describing the test scope.
- Defining test methods and tech stack.
- Writing test cases.
- Running tests and analyzing the results.
Planning End-to-End Testing: Things to Keep in Mind
In a perfect world, every product would undergo meticulous E2E testing. But as you already know, it is a resource-consuming process. Therefore, it makes sense to cover business-critical features with E2E tests to comply with the deadlines and budget. But is there a foolproof strategy for choosing the best cases for E2E testing that works for every product and situation?
The Testing Pyramid
Relying on end-to-end tests completely isn’t the best solution. However, they simulate real user scenarios, making them valuable for both testers and decision-makers.
Some suggest using a Test Pyramid — a practice similar to the one used for writing a test automation strategy. According to Google Testing Blog, unit tests should lie at the base of the pyramid making the greatest share — around 70%. The remaining scope should be split between integration tests (20%) and end-to-end tests (10%).
Unit tests, run by developers, test a piece of product in isolation. This particularity enables them to create a perfect feedback loop — fast, reliable, with bugs isolated. Integration testing allows checking those isolated units in small groups, verifying that they work well together. Finally, with end-to-end testing finalizing the list of activities, you get to check complete user flows when the smaller pieces that compose those flows have been already verified.
If you go to online forums or comments below blog posts on E2E testing, you will notice heated discussion regarding how the pyramid works in practice. Some find it difficult to measure the actual percentage of tests and maintain the pyramid shape. Given the total number of tests in a large project, E2E can still require too many resources. The others claim there is a way to make a feedback loop in end-to-end testing fast, reliable, and with bugs isolated, and it makes sense to turn to E2E tests more frequently.
Therefore, it makes sense to start planning drawing on the Testing Pyramid but build on the product particularities, team size and personnel, and previous experience.
Best Practices for End-to-End Testing
- Always create test cases with an end-user perspective in mind.
- Use E2E tests to cover, first of all, the core business functions of a product.
- Use end-to-end testing to cover common user scenarios but not exceptional ones.
- Run end-to-end tests after unit and integration testing are completed.
- Consider automation for ongoing projects with stable functionalities.
To Sum Up
End-to-end testing is a method that allows us to see how a system reacts to real user scenarios. However, it works only in combination with other methodologies. It is essential to execute unit and integration tests before running E2E testing. Otherwise, finding the root causes of bugs can be complicated and time-consuming. So remember to keep a balance between end-to-end and other test cases, and you’ll get an opportunity to experience all the benefits mentioned above.