Automated QA Process and Ongoing Support for the Legal Software Provider

The client*, a company trusted by Fortune 50 and top law firms, provides a suite of purpose-built AI solutions to help legal professionals practice at the highest level. Their software generates responsive pleadings, discovery requests, and other legal documents tailored to each case while remaining consistent with jurisdictional rules and an attorney’s style. The platform builds on an organization’s existing work and provides accurate analytics with valuable insights.

* We recognize the importance of protecting our clients’ privacy and follow the policies to maintain their confidentiality and security. That is why the company name will not be disclosed.

The ultimate goal was to increase the overall quality of the platform from the earliest stages possible. Supporting each update with a corresponding autotest would act as a preventive measure for costly fixes and delayed releases. Basically, we are speaking about the shift from quality control to quality engineering.

The client’s team aimed to implement and scale automation with two main goals in mind:

• To reduce the time required for manual testing and, thus, accelerate release cycles.
• To improve the test accuracy while expanding the coverage given the product growth.

The client didn’t plan to fully replace manual testing, which has its role and place in the sprint. They were looking for a way to optimize the team composition to drive maximum efficiency from the testing effort and QA engineers’ skills. It would prevent technical debt accumulation and inefficient resource usage as the development scales and the product grows more complex.

To prioritize test automation that would strengthen the overall QA process, we suggested hiring a General QA Engineer—a person who specializes in both manual and automated testing. Unlike the traditional approach, where a separate person is in charge of each part, general testing allows for more flexible and dynamic testing.

Before the start of test automation, the QA Engineer:

1. Studied the product. The client’s team shared access to all relevant documents covering the product details. The Project Manager hosted a demo to explain the main aspects of the product.
2. Set up the framework. The task included setting up the framework from scratch, i.e., selecting corresponding technologies and basic tools, as well as creating the basic components, such as page objects and login tests.
3. Analyzed the test coverage. Proper estimation was necessary to define the test scope and coverage for the important parts of the product’s functionality.

The framework was set up in six steps:

1. Deciding on the tech stack. TypeScript was selected as the programming language since it’s the one used for the product. Hence, developers can also use the test base with ease. Cypress was selected for test automation since it integrates well with TypeScript.
2. Defining the documentation. The QA engineer studied the product requirements in close detail to understand what is expected to be automated and where automation is feasible.
3. Planning for manual tests. Since there was no manual test documentation, the QA engineer prepared the strategy for manual QA and corresponding test cases, which are necessary to prepare the basis for auto scripts and prioritize them.
4. Writing automated scripts. Manual test cases were turned into automated scripts based on their importance in terms of the software’s functionality.
5. Establishing the automation procedures. The test writing style was standardized, and mandatory code review for all test cases, along with the specific procedure, was approved.
6. Setting the automation algorithm. The QA engineer defined the mechanisms for tracking test results. The tests were integrated into the CI/CD for regular test launch.

Gradually, two more General QA Engineers joined the project to handle the increasing scope of tasks. Our QA experts are working in close collaboration with the client’s in-house team, including Software Engineers, Manual QA Engineer, DevOps Engineer, and Project Manager.

Automation started with covering the selected, high-priority functionality. Later, QA Engineers moved on to managing the coverage with more independence. Their main goal is to maximize test coverage, but the requests within releases and smaller objectives can vary from sprint to sprint, which is natural.

For the most part, manual testing features regression and exploratory testing. A great share of functional, UI, smoke, and regression tests are executed automatically. This includes both positive and negative black box testing scenarios. Each test run is followed by a detailed bug report.

The QA engineers helped set up the test management with X-Ray to keep the test process more organized, trackable, and understandable for everyone involved with the product.

QA engineers initiated more active knowledge sharing to keep up with the platform’s details as it scales. Instead of focusing solely on one’s isolated part of work in charge of which they normally are. This approach helps learn the product in the smallest nuances, generate more accurate insights, and diversify the risks.