UI Testing - Understanding the Automation Testing Pyramid (Part II)
In my previous article I discussed ‘What is UI Testing’ and ‘Why do UI Testing’?
If you’ve already tested something at the Unit level, and again at the API level, testing it again at the UI level is just a redundancy that’s going to waste your time, so you need to identify:
- What are manual testers testing?
- Has this already been tested at a lower level of automation?
- If so, should it be tested again at the UI level, or just validated?
Automation at the UI testing level is going to reduce the impact on your manual testers, which means they’ll have a smaller subset of tests they need to do, and will therefore shorten regression cycles and reduce your turnaround time.
Most companies only think about testing (as a whole) from a UI level, because they only see it from a manual tester’s perspective. Many companies have very limited experience with Test Automation, so they think if they automate Manual Testing via the UI, it will automatically be faster.
But, if you’ve followed our article series on the Automation Pyramid, you’ll know this lacks the understanding of Pyramid and benefits thereof, and only results in very long estimates with lots of scripts that all require complex and time-consuming maintenance.
Automation is not something that you can finish in one go, it is a commitment that runs as long as development. All it takes is for a developer to change something small and seemingly insignificant, like an attribute on an element to cause an automation to failure. There is an effort that needs to go into Test Automation – you have to keep maintaining your scripts, you can’t just finish scripting and think you can move your testers to another project.
When you talk about UI, you have to check if the process flow has changed, are there timeout rules, how is it supposed to work, did a locator change - there are so many things that can go awry at a UI level that it takes significant work to maintain. A focus only on UI can get you into a difficult spot, because the more tests you create, the more you have to maintain, and it becomes a balancing act.
So what should you be focusing on when it comes to UI Testing?
- Focus on the highest risk. Create a priority list and focus on the high priority scripts. If you just look at regression pass rate, for example, your testers may focus on low priority tests to keep them passing.
- Don’t put all your eggs in one basket. Don’t only use UI tests. They are valid and valuable, but you should push your tests down to the lower levels of the Automation Pyramid wherever possible. If you keep that in mind you will have a much stronger and more stable automation project.
- It’s not going to be quick. UI Testing is not a quick solution that can be done by one person in a month. You may not even have it done in a year. It’s related to your testing environment and culture.
- Develop a strong testing culture. Following on from point 3, it’s important that you develop a strong testing culture because automation is not a silver bullet. It’s not going to suddenly reduce the impact on your manual testers. You need the proper frameworks to be in place, you need proper test management and to have your tests documented. Your testers should be working in a way that’s auditable and automatable.
- Automation is not a replacement for good testing standards. Those good testing standards will help you have a good automation project. When something goes wrong with an automation project it’s often because there are no standard in place beforehand to guide and shape the direction and scope of Test Automation.