Test Managers in Agile

21 Nov 2022

Recently, a large logistics firm removed testers from their Agile teams, instead relying on business analysts to do software testing. Soon, their development efforts were solely focused on fixing their mistakes instead of new development. This might be an extreme example that cost this company dearly, but we all know multiple examples of companies getting quality in Agile wrong. The disappearance of Test Managers and the absorption of software testers into Agile teams run the risk of sacrificing product quality in favour of development at speed. Are we still aiming to ‘do it right the first time?’

The advent of Agile methodologies completely changed the software landscape, including that of software testing. The attraction of product development that’s faster, more reliable, and more satisfying to the end user is irresistible. Unsurprisingly, the majority of teams report using Agile or Agile-like models as well as DevOps development and testing models:

Agile and waterfall practices differ in terms of the way they measure success, manage teams, organise, and communicate. Traditional waterfall models have test teams separate from the development team and overseen by a test manager. Typically, the test manager would assign tasks, keep on top of progress through meetings, review and approve estimates, and provide technical guidance.

Agile teams integrate developers, testers, and business analysts in smaller, nimbler teams led by development leads. The responsibility of organisational product quality shifts away from the testing team and lies with the entire Agile development team. In Agile, the focus is less on quality management than it is in waterfall.

In my current role, I am responsible for assessing large development teams and advising them on how to improve their development and testing processes to produce better quality products successfully and efficiently. Two-thirds of the organisations I have worked with have removed the Quality Manager and included his responsibilities to the Head of development to save costs and reduce overheads. Some have gone so far as removing the tester entirely and giving the test responsibilities to the business analysts and developers

This raises the following concerns:

We are so focused on “faster to market” with Agile, did our product quality also improve or are we just fixing the production defects faster?
Are we still true to our slogans that quality comes first in our organisations, or did we just move to the faster we go the faster we can fix our mistakes?
What happened to “getting it right the first time”?

The approach to quality differs between a developer (and therefore development manager) and a software tester. A developer’s focus will be on the quality of the code. Does it meet coding standards in terms of maintainability, scalability, readability, and transparency while reducing the risk of bugs? Developers don’t have a ‘product quality mindset’, and rightly so. While quality might be important to them, their reason for being doesn’t revolve around quality as is the case with software testers.

In comparison, a software tester’s approach to quality is different. Quality is assessed more from the end user’s perspective. In addition to the quality of the code, the software tester will also ensure that the software does what it’s supposed to do functionally and behave as expected. A tester advocates for quality within the Agile team, allowing developers to broaden their view and develop with overall software quality in mind. Testers are usability-driven, not solution-driven and thus focus on the quality of the product to the end user than simply providing a solution.

The redistribution of testers into Agile teams and the diminished role of test managers have implications for the quality of the end product. While production defects might be decreasing, does that automatically imply that the overall quality of the product is improving?

Organisations are reaping the benefits of Agile, with two-thirds reporting improvements in productivity, predictability, and cost of quality. Interestingly, only 56% of surveyed organisations reported an improvement in quality of 20% or more, compared to other areas such as on-time delivery (64%), predictability (63%) and cost of quality (62%). To put it differently, 44% of surveyed organisations did not achieve at least a 20% improvement in quality with Agile. Did these organisations just see a smaller percentage improvement, or perhaps even experience a decrease in quality? One would expect that more organisations achieve better quality since testing takes place much earlier in the development cycle and is now seen as the responsibility of the entire Agile team.

Perhaps the answer lies in the makeup of the various Agile teams. Teams with 6-15 testers have decreased from 27% in 2020 to 23% in 2022, while a larger number of teams only have 2-5 testers in 2022 (39% compared to 34% in 2020).

There is also an increase in the number of ‘other team members’ that take part in the formal testing process. These include developers, product owners, support, and end users.

In summary: Agile teams include decreasing numbers of testers, test efforts are more distributed among non-dedicated testers, and the quality effort is overseen by the Head of Development, whose focus will be on the speed of delivery, maintainability of the code, cost-saving, and generalised uptime of services rather than the quality and usability of the software product. It is little wonder that more organisations don’t rank ‘Better quality’ as a benefit of their Agile efforts.

With the loss of the role of Test Managers in Agile, the following areas show a lack of capabilities:

To measure the quality of the products
To develop strategic processes for verification and validation of requirements
To introduce monitoring procedures
To understand the various metrics used to ensure customer satisfaction and the quality of products delivered to customers.

Are we fooling ourselves by thinking that there is no longer a place for the Test Manager in Agile? Perhaps we should rethink the role and still include Test Management in our strategies if we are serious about quality.

Test management in Agile

With the responsibility for quality now resting with the entire Agile team, a strategic Test Manager will ensure that software quality is achieved rather than just overseeing a test team.

What is the expectation of an Agile test manager?

Focus on organisational test strategy and test process improvement
This focus will ensure sufficient and efficient quality in the development life cycle.
Empower rather than manage and control testers
Through guidance and process development that improves testing within the development environment and teams, test managers can empower testers rather than manage and control them. Agile test managers can give guidance on testing more efficiently in the modern world, ensuring test alignment between manual and automation, and adding value to the organisation with smarter and evolved testing.
Create more efficient teams
The Agile test manager will have insight into business needs and should increase test efficiency by matching the right skill sets to meet those needs.
Help resolve roadblocks for team members
Test managers bring a wealth of experience which should be used to resolve any particular obstacles test team members may encounter.
Grow capabilities and skills of staff
With their insight into business requirements, test managers should develop the capabilities of the testers to improve organisational skills and competencies.
Take responsibility for the collaboration of all releases over multiple teams
The test manager should collaborate all test efforts across multiple teams to ensure the quality of the product rather than the quality of the release or deployment within a single team.

Companies promise quality products but focus more on productivity and cost savings. We should bring back the strategic test manager that is focused on improving our testing capabilities and processes, thereby improving the customer experience and quality of products.

As the adoption of Agile continues to grow, the responsibility for quality increasingly moves away from quality specialists towards all members of the team. There is a definite place for the strategic test manager that’s focused on improving testing capabilities and processes to ensure that customer experience and the quality of products keep improving. Companies promise quality products yet focus on productivity and cost savings. Utilising the role of the strategic manager will ensure that all three goals can be achieved without sacrificing one or the other.