Client Overview

Cloudsmiths is a well-known player in the implementation and upgrades of Salesforce instances within South Africa. Its client, one of the largest major banking institutions in South Africa, was undertaking the largest Salesforce upgrades worldwide. The bank provides a range of retail, business, wealth management, corporate, and investment solutions to its clients. Salesforce is a Customer Relationship Management (CRM) tool that provides the bank with enhanced customer relationship management, streamlined sales and marketing processes, and efficient data management and analytics.

Problem statement

Cloudsmiths looked for a reliable Quality Assurance (QA) and testing service provider well equipped to assist in its foray into automating the end-to-end test processes for the new builds of Salesforce.

The various divisions within their client’s large organisation required unique instances of Salesforce as they had different requirements for the system. Due to the high frequency of builds released, Cloudsmiths required a quick mechanism to verify that all regression functionality still worked as designed with each new release through regression testing. There could be multiple builds released in a single day.

Inspired Testing was required to provide a recommendation on the tools to be used. The tools being considered by the client were Selenium – open source used in conjunction with Inspired Testing’s U-TAH framework and evaluating a commercial tool called Copado which is a centralized tool that caters for building, deploying and testing of Salesforce solutions.

Solution

Inspired Testing was given the freedom to recommend and implement a solution that would best fit the requirements of both Cloudsmiths and its client. Inspired Testing used a tried and tested way of working to plan, design, and implement the solution that would meet the abovementioned requirements. The Proof of Concept (PoC) was delivered using an Agile methodology, with the Inspired Testing team based on-site at the large financial organisation’s offices to promote collaboration between the team and the client’s Product Owner. The Inspired Testing team partook in the SCRUM ceremonies held daily at the organisation’s offices and produced a report indicating the status for the day, any potential impediments, and planned work for the next day. Inspired Testing used Selenium due to the tool being open source and having the ability to perform all functions that licensed tools could do from a test automation perspective.

The open-source U-TAH solution:

• Inspired Testings’ U-TAH Framework written in Java provides support for Selenium Web Driver, Mobile, and API testing. The requirement, in this case, was purely Web automation.
• Hybrid Methodology. Inspired Testing used the hybrid methodology when creating the automated test scripts. Inspired Testings’ vast experience in test automation implementation meant that we could extract the best benefits from stacks to allow for Operating Systems and different browser combinations to be tested.
• Advanced scripting techniques to cater for Object Identification issues as well as Iframes containers
• Integration into TFS for continuous integration
• Integration into Bitbucket for source and version control for the automation scripts
• Excel spreadsheets for the test data
• Sikuli for image-based object recognition
• JUnit to execute tests from TFS
• ExtentReports to report on the automated test results
• Dedicated TFS Agent Pool for test execution

The tests were executed on Google Chrome, but the scripts are cross-browser compatible and can be executed on multiple browsers concurrently, even from a single instance of Selenium running on one agent’s PC.

The Copado tool allowed testers to log into various Salesforce instances through a unique plug-and-play cloud-based testing solution. Copado dramatically simplifies Salesforce deployments by orchestrating and amplifying the DevOps process

Before improvement

The client faced the daunting task of the largest implementation of Salesforce to date, complicated by the need for multiple instances and the high frequency of daily builds. This posed a tremendous quality assurance challenge to achieve a unified SQA solution across the Salesforce implementation.

After improvement

Inspired Testing’s software testing experts suggested a combination of tools, including Selenium and its own in-house automation framework, to provide an automated regression testing capability to the client, which made regression testing possible in this complex environment.

Benefit to the client after implementation

• Reusability of the automated scripts. Demonstrating that these tests can be reused to test multiple data permutations and combinations and minimise the amount of maintenance required.
• Externalised input data. Test data is read from an Excel Spreadsheet. This demonstrated that, with only basic Excel knowledge and no exposure to the test automation tool, scripts and code, a non-technical resource can create and execute multiple automated test scenarios.
• The ability to execute automated tests 24/7 on a schedule or through build triggers without any human intervention required.
• The repeatability of automated tests. The tests can be executed on a schedule or ad-hoc repeatedly and still execute successfully.
• Robustness of Inspired Testing‘s automated scripts. The scripts execute regardless of whether the UI layout changes in terms of position.
• Integration between Selenium and TFS. Inspired Testing demonstrated that the U-TAH framework and latest automated scripts would be deployed automatically to the predefined TFS agents, and results written back into TFS.
• Classic UI and Lightning UI are automatable. The client wanted to see that both UI interfaces were compatible with Selenium and can be automated. Three processes were automated on the Classic UI and one on the Lightning UI.
• A reduction in QA costs over the long term. Test automation provides a vast reduction of costs over the long term. Scripts are created once and executed over multiple cycles thus making the effort once off with minor maintenance required should functionality change on the previous scripted process.
• Objective evidence for all automated testing. Inspired Testings’ automated test reporting provided screenshots for every step during the test process including all input data used. This can be used to accurately reproduce any defects found as well as provide proof of testing during the auditing processes.
• An evaluation of the commercial tool called Copado which was eventually procured and utilised to deliver on this engagement due to the centralisation of all build, deploy, and test functionality into a single tool.

Inspired Testing’s extensive experience in outsourced engagements and providing independent testing services (manual, automation and performance) combined with our agile way of work helped the client to focus purely on adding new features to their product with Inspired Testing providing the assurance that the product still meets the high-quality standards.

The daily reporting provided the client with a day-to-day view of the current project status as well as any impediments.