About Client
They are world’s leading live entertainment company and they work with artists to bring their talent to life on stages around the world. Their partner company is a global leader in ticket management for sports and entertainment.
The ‘Ask’
- They have an in-house development team but do not have an in-house QA team. They approached Musikaar with the need for both Manual & Automation QA. After discussions about their requirements and expectations, Musikaar was engaged to ensure thorough test coverage of their Mobile and Web Application.
- Since the client has a major consumer base in USA, the primary goal was to ensure that the test coverage is performed seamlessly for the US market. The client’s need was to further ensure that the mobile application works impeccably on iOS and Android devices; and the web application works smoothly on all popular US browsers and devices.
- They have frequent releases and announcements of new shows/promotions hence it was not possible to do full manual regression every time and required automation tied up with their build pipelines to prevent any critical/major failures to slip through to production.
Our Plan
- Our core team built a team of Engineers with relevant skills and experience. Building the team with right people helped with quickly identify real world scenarios and troubleshooting issues faster.
- After initial kick-off meetings, the team set up the environment and then started coming up with tests for each component gradually to cover all features as part of regression testing.
- The team used to have weekly sync up with the client’s technical team. Once the team got all test cases ready and reviewed, we worked on increasing productivity without affecting the quality.
- Based on the available data analytics, the devices and browsers that were primarily being used in USA (Android or iOS) were finalized. We got a mix of latest and widely used real devices to perform device testing.
- The Manual team also marked tests which should be automated and were important to be in the build pipeline.
- Selecting the correct tool is the most important step in automation testing. Keeping the requirements and nature of the application and expectations in mind, the automation team started with R&D on the automation tool to use.
- After looking at various tools, Appium was selected as the most viable open-source automation tool for mobile automation testing. Appium seamlessly integrates with CI and CD tools like Jenkins, Git, etc. and supports both real devices as well as simulators. For web app automation, Selenium was the winner.
- First, we started with real devices ,but to cover end to end testing on different devices, screen sizes and OS versions it was not cost effective to keep getting new devices in the current rapidly growing device market. Hence, after some R&D, talking with different providers, we found our solution in Browser Stack.
- Being in a different time-zone also has a strategic advantage. Onsite developers would finish their work and generate a nightly build which was tested overnight (i.e. our day) and a detailed report was sent before they would come to work the next day. This enabled us to make efficient use of full 24 hours, reduce turnaround time, get faster feedback, and increase overall productivity.
- We had a weekly sync up with client tech team.
- We sent daily status reports for Manual and Automation testing tasks.
- We communicated with Developers over Slack as needed.
Value Addition
- The team constantly tweaked the plan to run the optimal number of tests to ensure right coverage and meet the deadlines.
- Daily status reports helped them track progress and issues in the builds.
- With the help of Automation, manual testing efforts were reduced, thus saving time.
- Due to good coverage in automation, the manual team could focus more on finding bugs in new features early in the development cycle.
- We achieved broad coverage on various devices and multiple browsers.
