A core problem for many businesses today is that their critical systems were developed more than two or three decades ago using technologies that were developed long before their modern counterparts. These businesses, here and now, are in the situation where they are required to either modify or replace their software but it’s written in a language that’s 30 or 40 years old.

For us the challenge is how can we assist these businesses in testing systems that were developed before most modern automation tools and before most of our testers were born?

The first step we take around this is to split the types of projects, since we have a different approach for different scenarios. I’ll break down the two primary approaches below on how we look at testing in these scenarios.

1. Testing a replacement of a legacy system

Replacing a legacy system is normally easier for a large business than making extensive changes to one. This way the testing and automation is also simplified, because when new systems are built, they are built in current industry standards. That allows the automation of these systems to be done with modern tools.

• Where, What, Why? Knowing the original purpose is critical in testing a viable replacement of a system as it allows us to interpret what the correct requirements are to test against manually or through automation for a new system. Through documentation, usage, user interviews and analysis we can determine what to test for and how to assist the users during acceptance testing.
  
  There are a variety of problems that can still occur, such as a lack of understanding of the software, missing documentation or difficult integrations. But all have solutions and can be catered for as such.

2. Testing a modified legacy system

Changing a legacy system is really where the difficulty lies for both the developers and the testers. There are a variety of issues that are commonly encountered by developers, testers and analysts. Below I encapsulate a few of the categories and how to approach them.

• “Written in the Language of the Ancients” is a joke used by some for applications written in development languages like C, C++ and Cobol. They were the most relevant or purpose fit development languages of their era, but are nearly completely unknown today as most of the people using them are either retired or in positions of power.
  
  So how do we test in something written like this? The solution is normally to look up. The people in charge of our testers were once testers and developers, and grew up in the business world with these languages, so the understanding of these languages remains with them.
   
• The sacred texts. This is what we call the documentation for these legacy systems. In many cases the specifications, original requirements and designs are lost over the decades. This creates a unique issue where users have a perception of what the system does but do not have an true understanding of how it does so or why.
  
  When we encounter this issue, our approach is to build a proof of concept where we attempt to replicate the expected data flow of the system. If this can be done, we can test the alterations on the system and verify against our own POC that the process flow is working as expected.
   
• Unique network interfaces is a problem that regularly comes around from older systems that were developed before modern APIs. If a system is built with a network interface or a terminal service, these systems have their own network interface, data structures and interaction patterns built specifically to fill their needs.
  
  If we encounter this, we normally mark it as the most difficult issue to solve if no documentation is available. If we have the documentation, we can simply build a modern interface for an old network interface.
  
  Without documentation we either have to reverse engineer a solution from the original source code, or intercept the network traffic. This approach is highly time consuming as some systems have very complicated communication structures.
   
• If it’s not on the internet, does it exist? An interesting problem faced by many testers and developers is that the older commercial software has nearly no footprint on the internet. The community of modern software development share and work together on finding solutions for problems, and as legacy software predates the modern era, the solutions for issues with the software are rarely online.
  
  Our solution to this problem is using specialists, technical people specialising in solving the most complex problems a business can face, archiving solutions and documentation as they find them, and gathering information for existing archives.
   
• Look at the pretty picture. Image-based automation is a time consuming method of test automation that allows testers to automate systems that cannot be interfaced with directly. Many legacy systems do not contain anchors or interfaces available in modern languages that are used to automate a user interface.
  
  Currently there are about a dozen major open source image-based libraries available on the market, and half a dozen major commercial tools available as well. This gives us a very confusing yet broad choice when automating applications.
  
  The approach to take when dealing with legacy systems is to prepare a platter of choices, so we can choose the best combination of tools and services to meet the requirements of the system. It is important for any company never to sign in a high cost single solution, as no tool or library can do everything.
   
• “It works on my machine” is a line that many project managers and testers have heard from developers and sometimes users. This applies to legacy systems on legacy operating systems, where the company has opted to stay with legacy operating systems to support old software to stay in business.
  
  Application automation is not always possible on many of the older operating systems. There are ways around it, difficult but sometimes viable, such as creating a virtual machine of the legacy OS with the software loaded that can be automated remotely by a visual automation tool. This allows a modern tool to automate a legacy system, albeit very slowly.
   
• Friends with benefits. Knowing when a person or group does not have the ability to actually perform development or testing on legacy systems is very important. Be willing to reach out to known experts, specialists and associates that can assist in the process.
  
  What we do here is twofold, first keep a contact list of people and their skills that can be used in the future. This proves to be a highly valuable resource as nobody can know everything.
  
  Second, as a company we are part of a group of companies that can work together, share resources and knowledge and leverage on each other’s skills. We also keep detailed lists of resources and skilled people who are no longer in our group and who we could call upon if the need arises.
   
• “Kill it with fire”. Also known as plan B, this is sometimes the solution we would advise when dealing with certain legacy systems. Many developers will directly want to go to plan B when they are approached with the request to modify a legacy system, but our approach is slightly different.
  
  When it comes to testing these systems, if we have done several proofs of concept without success and there is truly no cost-efficient way to automate or test a system, we will advise the client to replace the system instead of modify it. This is an extreme scenario, but has merit since a newer system is easier to build, modify and test, and will have modern support for features and functionalities that legacy systems do not.