Building Scalable Applications (Part 2): Design for the cloud

+ No comment yet

This is part 2 of our series of blog posts targeting building scalable applications on the cloud. In this post, Rahul (rahul dot amodkar at indexnine dot com) looks at 5 key design principles to follow while building a cloud-based application.

In the previous post, we covered the benefits of moving to the cloud for scale. In this post, we will look at some best practices to build applications for the cloud.

1. Use messaging to decouple components

One of the important principles of scalable application design is to implement asynchronous communication between components. To start with you can break down the application into multiple smaller services and deploy a message queue for communication between these services.  A simple example of this would be to have your application’s Web instances put messages into a queue which are then picked by worker instances.

Such a mechanism is very effective for processing user requests that are time consuming, resource intensive, or depend on remote services that may not always be available.
Decoupling using a message queue

Keeping the communication asynchronous allows the system to scale much farther than if all components were closely coupled together.  By increasing compute resources for the workers, larger workloads can be disposed of quickly and in parallel. From a user experience standpoint, there are no interruptions or delays in the activity flow improving the user's perception of the product.
Most importantly, this allows the web tier of the application to be lightweight and therefore support more users per instance of compute that runs in the cloud.

2. Be Stateless

In order to scale horizontally, it is important to design the application back-end such that it is stateless. Statelessness means that the server that services a request does not retain any memory or information of prior requests. This enables multiple servers to serve the same client. By being stateless (or offloading state information to the client), services can be duplicated and scaled horizontally, improving availability and catering to spikes in traffic by scaling automatically.

REST or Representational State Transfer is a good way to architect back-end services. REST lends itself nicely to a stateless protocol and adhering to the principles of REST provides rich integration possibilities as it is based on the well-known HTTP protocol. Good quality clients to use the protocol are available in all languages, making access to the API simple and open regardless of client language preference.

3. Cache aggressively

Caching improves performance by storing recently used data for immediate reuse. Application throughput and latency are typically bound by how quickly data and context can be retrieved, shared and updated. Add cache at every level from the browser, your network, application servers and databases. You can implement caching by leveraging Content Delivery Networks (CDN) especially for static data or by using caching solutions like Redis and Memcached. Cache aggressively in order to significantly increase the application’s performance and its ability to scale.

A fast, in-memory cache speeds up look-ups

Before adding elements to a cache, assess the following 2 things:

  • How often is this data accessed ?
  • How often does this data change ?

If data is accessed very often and changes rarely, then this type of data is a prime target to be cached.

4. Expect Transient failures

A large cloud application spans many compute containers (VMs), Messaging services (Queues), Databases and other services. Although many managed services on the cloud provide excellent availability, it is still possible that failures may happen.

Transient failures may manifest themselves as temporary service outages (e.g unable to write to db) or a slowdown in performance (throttling).

Failing operations due to transient failures is not a good option.  Instead, implement a strategy that retries the operation.  Do not retry immediately because outages in most cases may last for a few minutes or more.  Implement a strategy that delays the retry thus allowing the resource to recover. Implementing such smart retry policies will help handle busy signals or outright failures without compromising user experience.

5. Don't be chatty

Since a cloud application is spread across multiple component services, every interaction might involve connection overhead. This means going back and forth many times can be detrimental to performance and expensive in the long run.

Pay attention to how data is accessed and consumed. Wherever possible, process data in batches to minimize excessive communication. For user interfaces, try and cache objects that may need frequent access, or better still, query most data needed for a view upfront and cache on the browser side to improve performance and reduce round-trips.

When it comes to user interfaces, a pure object-based REST API may not be sufficient to provide all the data needed for screens (especially dashboards), upfront. Avoid the temptation to make multiple calls to the back-end. Instead, build an aggregation API whose data can be cached to improve performance.

Remember, on the cloud, innocuous looking functions may end up making a web-service call, adding overhead to your functionality.


Use these design principles while building your cloud application. These principles will help you build applications that can scale to meet demand by leveraging auto-scaling. By recognizing the pitfalls of the cloud environment and designing around it, you can build fault-tolerant apps that users are happy with.

Building scalable applications (Part 1): Leverage the cloud

+ 1 comment
This is part 1 of our series of blog posts targeting building scalable applications on the cloud. In this post, Rahul (rahul dot amodkar at indexnine dot com) points out the advantages to using the cloud to build scalable applications. In future posts, we will cover reference architectures and examine different components of a cloud-application.

A friend of mine tells me a story repeatedly. Back in 2003/4, he co-wrote an online news aggregation web app when RSS feeds were first taking off. The app had every social news application feature that you could think of. Imagine Digg + StumbleUpon + Google News all in one (and in 2003 !). Things were going great guns until the time a popular blogger chose to write about the site. A couple of hours later, the site crashed in the middle of the night. Next morning, the negative reviews started. Large number of users reported the site was down or slow to respond. My friend was never able to capitalize on a viral uptake in user base.

Consumer facing applications can go viral very quickly. If an application is accessible via a smartphone interface, the power of social media can fuel exponential growth of your user-base as well as data.

Monolithic applications fail to scale up to the demands of a growing business. The failure of these applications in responsiveness and scalability results in customer dissatisfaction leading to loss of brand value and of course revenue.
Airbnb user base uptake. Source :
The above graph shows Airbnb’s user base expand virally between May and August 2011. They added close to 500,000 users in the space of 3 months. A similar but more recent example is Twitter's Periscope app which racked up more than 1 million users just 10 days after launch. It would be extremely difficult to take care of such swift scale in traffic without a cloud deployment.

In this series of blog entries, we will take a look at some of the aspects that help us build a scalable cloud-deployed application. In this first part, we will look at the advantages of building scalable applications that leverage the cloud.

Unlimited Resources

The cloud provides the ability to scale on demand and with almost no practical limits. It is possible for infrastructure resources like CPU, memory and storage to be provisioned in a matter of minutes as opposed to hosting and operating those locally. In traditional on-premise data-centers, growth in demand has to be catered to by procuring new hardware. This is not only expensive, but may lead to over-provisioning of resources, not to mention the lead time required to setup and run.


As traffic spikes, cloud based applications can scale on-demand. You can even configure automatic scaling such that as the traffic spikes, resources are allocated and as the traffic declines the resources are relinquished. (We will look at Auto-scaling in details in future posts of this series). This helps the application use excess capacity only when needed.

Low Costs

Cloud adoption cuts down your Capex costs in a big way. It does not require a significant up-front capital expenditure and requires considerably few internal resources to maintain and administer. Operating costs in a cloud environment can be monitored and controlled as per your budgets.

Pay As You Go

Cloud platforms do not need any up-front commitment by users. Most cloud platforms allow you to pay for actual resources consumed. Resources like compute (CPU) are usually billed on a per-hour usage model whereas other resources like network or storage are billed on bandwidth and storage consumed. This approach takes away a large component of any upfront payments for businesses. It also levels the playing field, allowing smaller companies or even individuals to start cloud businesses that succeed.

Time to Market

Be it an established business house or an individual entrepreneur, one advantage of the cloud that excites everyone, is the ability to push products to the market faster. Traditionally it takes anywhere between 6-8 weeks for IT server provisioning. Compare this to the cloud that can provision the IT resources in a matter of hours if not minutes.

High Availability

Cloud platforms have presence in multiple regions. Each region is a distinct geographic area which in turn has multiple, isolated data centers. Applications hosted in cloud can be deployed on multiple data centers in order to make them highly available. Since the application is not dependent on one physical location, outages such as loss of network or electricity do not translate to application downtime.

Disaster Recovery

To protect against natural disasters and calamities, applications can also be hosted in multiple regions using data replication and DNS failover. This allows applications to survive outages of complete regions, by restoring business operations quickly with minimal downtime. Providing disaster recovery solutions using traditional means was reserved for the deepest pockets before the cloud arrived.


Cloud adoption has allowed organisations to scale faster to the needs of a growing business without the headache of managing infrastructure. It is no wonder that cloud has become an integral part of any scalable application. Coupled with “pay as you go” pricing and economies of scale, cloud vendors provide incredible value for computing needs that cannot be ignored.

Architecture for applications on the cloud is a different ball-game and involves various components in tiers that need to be put together to create a functioning application. In our next post, we will delve into the design aspects of a scalable cloud application. Stay tuned !