Client Overview

The client is a financial services group headquartered in South Africa and the largest insurance company in Africa. It is listed on the Johannesburg Stock Exchange, the Namibian Stock Exchange and the A2X.

Problem statement

The client had a greenfield application that was behind on its development schedule. This put strain on testing, and issues that could have been picked up during development were only later found on the front-end.

Because new features would reach UI functional testing before any problems were discovered, development times increased dramatically, which in turn increased costs and delayed project timelines.

Developers were also under great pressure and had little to no time to create unit tests because business needs were tied to the development of new features. Older code, which required refactoring or removal, was buried under new development work, and potential issues were exposed by a lack of automated regression testing.

Solution

Inspired Testing began test automation in March 2020, allocating a four-strong team of onsite automation testers. Almost immediately, South Africa was thrown into lockdown due to the Coronavirus pandemic, forcing all teams to work remotely. Fortunately, the transition from onsite to remote work was seamless, and the project was not delayed in any way.

The team spent the first month investigating and becoming familiar with the codebase of the company’s backend server. Time was also spent with the development team to establish testing priorities for the different modules in the project. Based on its findings, the team created unit tests to achieve the highest possible code coverage.

After reaching approximately 60 % line coverage, the automation team assigned one tester exclusively to API testing, creating a suite of automated tests to cover backend server controllers. The team also implemented UI testing, though this was kept to a minimum while the client rewrote the front end. Once the development team completed work on the front-end (using Angular JS) the automation team created Selenium sanity tests.

Inspired Testing worked closely with developers to make their code more automation friendly. The team was granted access to the front-end source code and is in the process of adding automation properties to the elements to create very stable Selenium scripts as a result. Inspired Testing also provided an Angular unit testing service, with the team using the built-in testability functions of Angular to create component-level tests, which run much faster than process-driven tests.

Angular unit tests combined with Selenium will create a suite with optimal test coverage, speed, and stability.

After improvement

Between April and June 2020, Inspired Testing created 8001 unit tests for the backend server. These tests were merged with the development staging branch, and subsequently rolled out to development branches, allowing developers to execute regression tests while building the project, and ensuring there are no errors before committing new work to the staging environment.

These tests (all 8001) run within 10 seconds, at approximately one test per millisecond. This allows developers to get fast results, and make changes accordingly. Inspired Testing integrated the unit tests into the company’s development pipeline using the Atlassian suite (Bamboo). This allowed the team to execute the unit test suite at compilation time and create a quality gate for deployments.

The API tests were also actively used to ensure that all endpoints returned the correct response code and data payload, and that tests execute in an acceptable time frame. A report was generated from each execution, complete with the relevant information for each request.

All test results – unit, API, and UI – were represented in a CI-friendly report, allowing the client’s application pipeline to integrate seamlessly with the greater context of other applications within the environment. This made it possible to fit into the strategic CI pipeline being used to deploy applications within the group, and not just the project.

The testing team was able to identify potential bugs and redundant code that was then scheduled for refactoring or removal. This was the first time the old code was interrogated to ensure it was providing value. The team also provided developers with a motivating factor for why they should be allowed to refactor old code, presenting Inspired Testing’s findings as risks to the project.

Inspired Testing’s unit tests and experience in testing and coding allowed the team to share and implement processes such as proper branch approvals within the client’s source control management strategy. Prior to this, there was no official process involved in committing work to the staging environment. Developers are now expected to submit unit test results for their commit to show that there are no regression errors.