Part I: The Automation Pyramid

Test Automation is a relatively new discipline, only really becoming part of the Software Development Life Cycle (SDLC) in the last decade. As a result, it’s easy for companies that are new to automation to be swayed starry-eyed by the promise of flashy UI-driven Test Automation solutions like Selenium.

I believe there’s a smarter way to automate your software testing, and it’s called the Automation Pyramid. As the name suggests, the Automation Pyramid is made up of tiers, with a wider, flatter tier at the bottom, and two narrower tiers up to the pinnacle at the top.

Each of the three tiers is home to a different type of automated test, as follows: 

• Unit Testing at the bottom
  • Unit tests are fast, easy to execute, highly stable tests that take place early in the SDLC.
  • As soon as developers check in their code – or even before they check it in – they can run unit tests on the code and get instant feedback. If the unit tests don’t pass, they know there’s an error, which can then be fixed before it goes up the development chain.
  • If you develop a stable set of unit tests at the base of your Automation Pyramid, it creates a solid Test Automation foundation for the entire development process.
     
• API Testing in the middle
  • One layer up from unit tests are slightly more complex non-UI tests called API tests.
  • Like unit tests they don’t rely on a graphical interface to run, so there’s no wait time for screen loads and other UI-related tasks, but they’re not quite as quick as unit tests to run.
     
• UI Testing at the top
  • Right at the tip of the pyramid is where your automated UI tests (like Selenium) should go.
  • These are known as end-to-end tests, because they test that the front end and the back end are working together from the user’s perspective.
  • It also means that when any graphic UI element changes, the tests need to be adapted to fit the changes, so these are high-maintenance tests that are also typically the costliest and most time consuming.

The Automation Pyramid

Benefits of using the Automation Pyramid

I’ll use the example of building a simple web-based calculator application to illustrate the benefits of using the Automation Pyramid.

• Bottom-up testing. The actual calculations are the underlying logic of the application and can quickly and easily be tested at a unit test layer. You do not require the UI to be available to validate the logic of your calculations, so you could have a strong base by running your logic tests at the unit layer.
• Top-down testing. If a company were to use a UI-driven Selenium tests, they’d have to wait for all the code to be written, compiled and deployed before executing the test, and should anything go awry, go back into the code to troubleshoot the problem. If you have confirmed your logic is correct at the unit layer, you do not need to focus on that with Selenium type tests, but rather just have a handful to ensure your application is behaving as expected. Business / calculation logic should be tested at the unit layer as far as possible, not at the UI layer.
• Doing it right. By building fast, efficient unit tests for the main function of the calculator, the company can get the app to market faster, and use API or UI tests to only test the latter, user-facing functions of the software, thereby saving time, and limiting the spend on expensive tests for where they’re most applicable.

In the above example, it’s clearly more beneficial to run unit tests on the core function of the software, and limit slower, more expensive API or UI Testing for more complex features and UI elements up the chain.

Doing so also helps avoid duplicate tests, because once you’re testing at the UI layer, your core logic is already a known quantity. There’s nothing wrong with using UI tests to validate the results of your unit or API tests, but only a small number of functions will typically need to be validated on any given project.

In Part II of this article, I’ll discuss how to use the Automation Pyramid correctly, some pitfalls to avoid, and finding a partner to help you through the process.

About Leon

Leon Lodewyks joined Inspired Testing in 2019 as Head of International Delivery – having spent 15 successful years in the testing and software development industry in the UK. He worked for numerous high-profile companies in the UK as a test manager, where he developed a keen understanding of the importance of testing as a business strategy, particularly in the context of agile development frameworks. Leon now fulfils the role of Managing Executive of the Coastal region of South Africa + Global Delivery.