Test Management in an Agile World
We're always happy when someone gets excited about test automation. Sometimes, however, people ask for test automation without really knowing what they want to test.
Some common misconceptions about test automation:
- It's a fix-all solution. Just because "test automation" is used together with “continuous integration”, “fast feedback” and “faster releases”, doesn't mean that deploying it will cause all the other pieces of the jigsaw to fall into place.
- Running Unit Tests is enough. Running Unit Tests is fine, but unless you understand what the Unit Tests are doing and how they fit into the wider picture, running thousands of tests asserting that 1=1 won't add up to much at the end of the day. What really matters is using these Unit Tests to provide real value by supplementing and building out other types of automated tests. Think Integration Tests, UI Tests, API Tests, or other tests you run regularly for feedback.
- Test failures can be ignored. Running tests won't add much value unless someone obsesses over the results. Test failures provide valuable insights into underlying issues that must be addressed. If an automation is running sporadically or turning up hundreds of failures, ignoring the gaps will simply cause people to lose confidence in the value of automation.
How to get people to care about testing:
- Get people involved - We’ve found that getting people involved in creating the tests, setting up the pipeline, and understanding why and what the pipeline is being used for is key to making people care. #SharingIsCaring.
- Get senior management buy-in - Taking senior management through the benefits of the automation, what it’s doing, and how it’s doing it, can go a long way to getting their buy-in. Once you have their buy-in, that will inevitably cascade down to the rest of the team.
- Visualise your automation tests - Visualising automation tests helps people understand the results of the automation, and that helps them care. Running a test locally on someone’s machine, where nobody ever sees the results, can be abstract. But providing a test dashboard that shows when tests are running, when there are testing failures, and when tests have been muted or ignored helps makes testing tangible and keeps it top-of-mind.
The other CI we need to talk about
We’ve spoken a bit about one type of CI (Continuous Integration), but there’s another CI that’s just as important: Continuous Improvement.
Setting up automation tests, then leaving them to run in the background doesn't mean much. Your pipeline and automation must be continuously monitored. Be on the lookout for improvement areas, and improve them.
Four CI principles to remember:
- Work together with developers. If developers are working on something with associated tests, encourage them to review the tests with you. Together, you will leave the tests in a better state than you found them.
- Keep an eye on your pipeline. How long does it take to give you feedback? How often does it fail unnecessarily? Monitor its performance regularly and, if it's underperforming, investigate.
- Data is king. Set up metrics around your pipeline, just like you would for production software. Use these metrics to drive changes and improvements across the piece.
- Implement a prioritised backlog on improvements - and dedicate time to it. Your pipeline and the associated automation are critical for deploying software with confidence. You can have the best software in the world, but if you can’t deploy it safely and reliably, it loses much of its value.
So, what does this all have to do with Test Management?
Someone has to run with all this and own it. Someone needs to manage which tests are happening, where they are happening, and why. Someone needs to keep the importance of testing top-of-mind and involve other stakeholders where necessary.
Whether that’s a Test Lead, a Test Manager, or something else, the title doesn’t matter. It is a form of Test Management, in the traditional sense or otherwise. As the Senior Tester, you know what good testing looks like, you know what's important to the customer, what's important to the teams and, most importantly, how to get all the necessary stakeholders on board to get to where you need to be.
Remember: your test automation is only ever as good as the tests that underpin it. Make people understand this, and half the battle will be won.