Here’s a scary statistic for you: test automation has a failure rate of 65 percent (or more, depending on your sources). That doesn’t come as a surprise when you consider companies spend up to 40 percent of their project budgets on quality and testing, and yet one of the first things they cut when things turn south is testing.

This vicious cycle of inflated budgets and failed projects was made clear to me the other week at a client meeting when, at one point, the CFO questioned the value the company was getting from testing while, at the same time, the chief tester was motivating for more budget.

While disconnects between business and IT are common in every industry, I couldn’t help but think this sad scenario could so easily have been avoided if both parties had agreed on a test automation strategy before they kicked off the project.

It may seem obvious, but in my experience setting a test automation strategy is exceptionally rare. Instead, project sponsors have it in mind that, if any part of the testing is going to be automated, they may as well automate everything. The logic goes something like this: “If we can automate, let’s automate, and the savings will follow.”

Instead, the opposite is usually true. Testing – and test automation in particular – is like any other IT project. It will absorb all the money you can throw at it, but the more money you throw at it, the more complex it becomes, and the more complex it becomes, the more time it takes resulting in a higher risk of failure.

I make no bones about it: building test automation scripts is not cheap, and, at least initially, can be very time-consuming. Time is not a test automation scripter’s friend; the longer the delay, the further along the project has moved by the time the initial batch of scripts is ready. By then, many of the scripts already written are either redundant or need to be rewritten to account for changes in the software - which costs more money, and time.

Test automation requires a lot of analysis; are we scripting the right thing? Are we focusing on the right functionality? Or are we scripting for the sake of it? Testing never ends, so it’s important to identify problem areas and balance risk and budget accordingly.

Incidentally setting and following a test automation strategy is fairly simple. If you were paying attention earlier, the three factors that determine the success or failure of any IT project are time, money and risk. So think of a test automation strategy like a box made up of time, money and risk as its three axes.

The more you fill up the box with automation scripts, the larger it grows (more time, more cost, more risk). So all you have to do is keep your script writing in check, which keeps the box small enough to manage.

I know this goes against the very essence of “throwing the kitchen sink” at your new shiny project, trying to automate anything that moves and hoping for the best. However, it’s a safe bet that the same gung-ho approach at the beginning will do nothing but hasten your project’s path to the scrapheap.

Instead, why not think of the small, nimble, Agile-motivated methods you’re probably already using to develop your new application (or website, or software) and apply it to testing?

Start with small, effective automation scripts for the basic building blocks of the software – and build on the successes of those scripts to fund new scripts incrementally as the project progresses. In that way, your box always stays small enough to manage, you don’t blow your budget, changes are quick to implement, and risk remains consistently low.

Because you’re automating effectively, the business can immediately see the value of your work, and by rights should continue to fund your efforts. More value, more budget.

But what, if like so many projects, your app takes a different direction, or your software splinters into multiple versions?

Easy – build a new box for each of them, and keep those boxes small enough to manage, just like you did the first time. After all, it’s much easier to manage a group of small boxes than it is a large, bloated, unwieldy box growing out of control.

Even if you’re not into boxes and building blocks, setting a test automation strategy is really about setting realistic expectations at the start of the project so you can maximise the chances of getting the desired return on your investment at the end.

Let’s face it, you’re not going to spend the budget if all you’re likely to get is a 20 or 30 percent success rate after a year or two, but that’s all you’re going to get if you don’t have a strategy in place that gets you more.