Just start typing...
Technologies

Open Source approach helps optimize costs by 20 times

Published November 19, 2020

In 2020 the increasing demand for online services resulted in the explosive growth of the expenses for software maintenance. After monthly cloud consumption in myQuiz, our own .NET application, jumped from $4,400 to $11,000, we faced the challenge of speeding up the delivery of new features, and optimizing the related expenses. In this article, we share how Open Source and Microsoft technologies allowed us to do this.

myQuiz is a multicomponent .NET application created to host online quizzes for 10 to 100,000 players at once. From April, 2019, the platform has grown almost 20 times (increasing from 45,000 to 1 million visitors). Inevitably, monthly cloud consumption in myQuiz also jumped from $4,400 to $11,000, so we faced the challenge of optimizing the expenses of the application - and speeding up the delivery of new releases to deliver only positive experience to incoming users.

Users play directly on their mobile devices and compete against each other. They answer quiz questions invented by a host, or purchased in the platform’s marketplace. While, for players it is a fun game based on the speed of their answers, for businesses it is a gamification platform. Brands use the platform to mix fun and promotional content in branded gameplay. Thousands of players can compete in the same quiz held at a certain date and time. As an example, Vodafone used myQuiz to engage 13, 000 fans at the Gulf Cup.

High load capacity and ability to handle a large number of requests per second is a vital component of myQuiz. It must be accessible, quick, and scalable to handle every kind of event: from a small game of a dozen home players to an online party for over 5K users. Being a Microsoft Azure-based web service, the platform can handle these challenges. However, this scalability was not enough for our goals, and we decided to further optimize both the expenses for the application, as well as some steps of our development process.

Step 1. Switched to a parallel component delivery using TFS + Ainsible as an Open Source tool

As we said above, the platform is based on Microsoft Azure and .NET technologies. We use TFS with VisualStudio IDE, and PowerShell for the administrative tasks. The delivery process looked like this:

1 Scheme

The parallel delivery of multiple application components could reduce the delivery time and minimize errors in the solution so we started our research of the best tool for the job. The built-in ability of TFS (so-called ‘parallel job’) allows for the defining of multiple agent phases in a single release and the assigning of tasks for each phase. This approach is a convenient way of deploying multicomponent applications.

So this could, in turn, be our ‘after’ delivery scheme:2 Scheme

When running more than one agent phase at a time, you need more parallel ‘jobs’, and normally, would need to pay for every additional job. To avoid additional expenditures, we chose an Open Source solution Ansible - a configuration management tool which can be used for many IT needs. Why Ansible: 

  • A simple declarative tool (there are ready scripts, which allow for minimizing of developers’ efforts) with detailed documentation
  • It is able to run parallel tasks on multiple hosts out-of-the-box, with no licensing
  • Scalable and secure (uses SSH or WinRM)

Pairing Ansible with windows services wasn’t an easy task, because Ansible was built to work with Linux. It took WaveAccess some time to research and try different approaches, until we ended up with the following combination:

  • Ansible for application deployment (we wrote an Ansible Playbook - a set of scripts to show Ansible how a system should be configured)
  • PowerShell as a configuration platform. Being a declarative platform, PowerShell offers a wide range of scripts to perform configuration tasks, setup services, etc, which saves us hours of work
  • Microsoft DSC module to enable windows systems configuration. DSC became a ‘glue’ to stick Ansible and PowerShell together

From now on, our deployment scheme is:

3 Scheme 09

Step 2. Use the TFS + Ainsible scheme to automatically renew SSL certificates

The strategy that we created in Step1 was then extended to automate another time-consuming task. Some SSL certificates expire every month, this previously had necessitated our developers manually renewing them for each component of myQuiz. Now we have successfully automated it, allowing developers to focus on delivering new features instead of engaging in busy work.

Step 3. Ainsible + Gatling for load testing

To test the application's performance, the myQuiz team uses Gatling Open Source Load testing: there are more than 20+ virtual machines, with 50,000 ‘users’ attacking the app simultaneously. Instead of manual checkout of every machine at once, we automated this task as well.

Despite being an Open-Source in its core, Gatling appears to be excessively costly when it comes to the use of parallel configured simulations: in the case of myQuiz, $19,00 per hour. In addition, it is found on Amazon Web Services hosting, which would require additional payments for the AWS cloud services:

Task4 (1)

Again, we used Ansible to parallel these ‘attacks’. To achieve this, REST API was used. Every virtual machine in our scale set has its own IP address. They can be reached via Azure REST API. Basically, Gatling containers start with a slight delay of 1 to 3 minutes one after another. However, because of the need to test the true app's load resistance we needed to start them all at once.

The result

Demand for myQuiz is increasing day after day and the question of quality has always been pivotal for us, but now is brought into sharp focus.  After myQuiz consumption increased by 20 times, we had to optimize the overall costs for the application, deliver new features faster and perform even more extensive load testing.

Using PowerShell and Ansible OSS technologies we have developed an approach that can be used for any Azure based service, especially, in the case of its rapid growth and expansion. What started as a small project, ended up with multiple appliances:

  • To cut costs for parallel delivery of the application to virtual machines in Azure cloud (cost savings is now $40.00 per each packet)
  • Cut costs for performance testing: to parallel Gatling simulations (cost savings are now $5,000 to 4,000 per month)
  • Eliminate routine: to renew SSL certificates automatically every month, allowing the team to focus on more important tasks

Open Source approach transforms the way the application is delivered and tested. It becomes easier to overcome today’s challenges along with significantly reducing costs, and  still enjoy the freedom of Microsoft Azure cloud.

We used the following Open Source technologies:

Ansible - to parallel delivery of artifacts (SSH certificates, Gatling containers, a ZIP archive with an application version), the core of this OSS project
Redis - a fast key value storage
Gatling - to perform load resistance tests
Selenoid - a cross browser Selenium solution for Docker to perform UI tests
Grafana, Prometheus - to monitor performance tests and to gather metrics every 5 seconds 

Thank you for reading! If you enjoyed this article, then probably you'll be happy to learn how to work with Open Source projects in Microsoft Azure! 

How to work with Open Source projects in Microsoft Azure. Part 3

In this article, we will shed light on the not so obvious ways to reduce costs in a well-established infrastructure, and the proper management of business process optimization, as well as share the reasons why you actually need Azure Kubernetes Service and multi-cloud.
October 6, 2020

How to work with Open Source projects in Microsoft Azure. Part 2

In this blog post, you will find a roadmap for an efficient and cost-effective IT landscape upgrade with Microsoft Azure, along with a coherent journey from MVP to commercial operation with Kubernetes.
September 24, 2020

How to work with Open Source projects in Microsoft Azure. Part 1

How to optimize the cost of software development and delivery, where to start the transition to Open Source technologies and how to work with Open Source projects in the Microsoft Azure cloud? Find detailed instructions in our new blog posts series.
September 14, 2020

Related Services

DevOps Services
Application Development
We use cookies to make your experience of our website better and assist in our marketing efforts. By continuing to browse or by clicking “Accept” you agree to our use of cookies as described in our Privacy Policy.

How we process your personal data

When you submit the completed form, your personal data will be processed by WaveAccess Services. Due to our international presence, your data may be transferred and processed outside the country where you reside or are located. You have the right to withdraw your consent at any time.
Please read our Privacy Policy for more information.