Client Overview

CMS works in more than 40 countries with more than 1,200 partners and more than 5,000 lawyers. It prides itself on being a future-facing firm delivering legal and tax advisory services.

Problem statement

CMS Legal Services EEIG had to migrate from Thomson Reuters Enterprise Business Management platform to Thomson Reuters Elite 3E. CMS discovered errors and differences in calculations after migrating data into Elite 3E. Comprehensive testing was needed as the smallest error had the potential to snowball to catastrophic proportions. Inspired Testing provided an automated testing service of CMS' data migration efforts which allowed them to identify migration errors at speed. This helped to alleviate the risk of moving from Enterprise to the new Elite 3E platform.

CMS needed to migrate from legacy Enterprise to Elite 3E. Errors in the migration process and differing configurations between the two systems lead to data errors. These data errors carry the risk of financial losses and damage to their reputation as it may lead to clients being billed incorrectly. The sheer volume of data that needed to be imported was tremendous. Simple calculations of Matter numbers and Timekeepers within the system totalled 1 085 840 unique combinations, all of which carried risk. CMS needed a reliable method of testing all these combinations regularly and at speed.

Solution

CMS' key requirement of Inspired Testing was that all unique combinations of data must be tested. It was left to Inspired Testing's discretion as to how they would approach this challenge.

Based on conversations with CMS, Enterprise was selected as test oracle to which calculations done by Elite 3E should be compared.

Front-end or UI testing would take 24.7 months to complete a full cycle. Based on the best-practice Software Testing Pyramid, UI testing was removed and the focus was placed on a lower level of testing.

A test was designed that would simultaneously execute the same API calls against Enterprise (SOAP requests) and Elite 3E (REST requests). The test would then extract the created entities from each application's respective database and perform a one-on-one comparison of the data.

Inspired Testing also created a mapping library for the database tables which was mismatched between the two systems.

The process of combining API and database tests could be completed within 4 seconds, including executing requests against both systems, database queries, and validation.

The sheer scale of testing meant that the stress on the system was tremendous. The executor slowed to an unacceptable 4-30 seconds per test. A new system was designed which allowed us to continually kick off batches of 1000 tests without slowing down the executor.

First, we queried the Enterprise database for all the Unique combinations of Matter numbers and Timekeepers. This data set was then stored in a testing database table with an integer to represent its place in the queue. Each time a test would kick off it would take a subsection of the table and increment its queue position so that it would be moved to the back of the queue after each execution. Results were then stored in a database table with a timestamp, allowing us to create an Excel / Power BI report which could pull data from the database for a given period.

This allowed the visualisation of the progress of the execution, how many unique combinations were still outstanding, and the results of the execution. Errors were investigated and repeated errors were given identifiers, which allowed unique defects/failures per run. This in turn allowed the CMS team to focus on the distinct errors rather than having to go through EVERY error in the dataset.

Initial runs outputted a success rate of approximately 60%, which suggested there was a significant risk to going live with the migration. Over time CMS was able to reduce the rate of failure and demonstrate a 99.98% success rate of the migration.

After improvement

CMS has successfully migrated Elite 3E into production and was able to do so with confidence as a result of the exhaustive level of testing provided by the Inspired Testing team.

Benefit to the client after implementation

All 1,085,840 unique combinations of Matter and Timekeeper were tested every week

• Testing which was not previously thought possible was achieved
• An unthinkable number of hours were saved:
  • A full regression run took an entire week of execution
  • Tests ran 24/7
  • Each test took 4 seconds with a parallelism level of 15 tests running at any given time