Cloud Computing really Simplifies your life - Read this Why?

In the age of Cloud Computing, one of the fundamental questions that always bother my mind was, Why Cloud Computing is very successful in spite of all the myths around?

I believe the answer is “Simplified computing paradigm ".

Let’s take a look at various things happens in our corporate life today. (Just about Infra, nothing else.) :-).

As a developer, I still remember my old days (Before Cloud computing that is even before the age of VMware (or equivalent virtualization technologies), I spent almost all my time (70% of project time) acquiring new servers, installing the OS, configuring the network (sometimes even the network elements like router & Switches, especially if you were a start-up) web, app & DB server...etc that were related to my projects.  The complexity of the infrastructure made sure that, we pray to the god every day for its availability without any major issues. Especially if your machine crashes, you spend almost the equivalent time that you spent initially to get back to its original state. And things became more complex if you have not documented your setup procedures.

I hear what you saying, “I wish my project manager reports to me for one day as a developer”. :-)

On the other hand, infra setup takes time and that becomes an advantage for us (developers), isn't?  We put the blame on our IT folks stating that the Infra is not provided yet. Hence, the work could not be started. And if it is a critical one, the IT folks are really in trouble to make sure they get everything you needed, yesterday. :-)

Let's hear our IT folks, and they say, we have exceeded our usage limits in our current IT capacity. The purchase request has been sent to our finance and waiting for their approval. May be, it will take minimum of 2 weeks to get everything setup what you requested. Now, the project manager trapped in trouble, answering the customer :-(

I believe even today most of the big corporate are still having these issues. And that is why; most of the CIO's have hundreds of IT folks for supporting their enterprise infrastructure to keep their SLA of 90 - 95% of not having major outages. Unfortunately not every CIO can hire great minds. Finally, for any failures, we either blame the process or people (who are involved) for not planning properly or executing them. :-)

All of these are going to vanish soon with the help of Cloud Computing and people are going to be more agile and responsible for what they do. In one way, Cloud Computing technology is not just going to change the way we compute, it is also going to bring discipline within people.

Happy Cloud Computing!
“Cloud computing removes frequent mist encircling day to day IT  problems”.

Migrating Existing Apps to AWS Cloud

Background

Few weeks back, I was part of a Panel discussion in the Chennai Cloud User Group un- conference.

Almost all the members of the group had one question, how do you make your existing web application run in the Cloud Environment? I am sure, all who wants to migrate to the Cloud for the first time, would be asking this question to themselves. Though it is a basic one, I believe there needs to be a systematic approach towards achieving this.

I am sure you all know about Amazon Cloud, but for those who do not know about AWS, please visit http://aws.amazon.com website and create an account using your credit card. You will not be charged for anything upfront. :-)

Some of the useful tools that are available to manage the AWS Cloud are Hybridfox, CloudBuddy Personal, Elasticfox, S3fox...etc. These tools have more features than any of the tools available in the market.

Well, In general to move your applications to the Cloud, you don't really need to do any changes. All you need to know is how the Cloud works. Cloud languages like Machine Image (EBS or S3 based), instance, Security Group, user data, Availability Zones, CloudFront, SQS, RDS..etc This can be easily picked up if you spend about a day or two in AWS website. But, if you want leverage the inherence advantages of Cloud, you need to really have a solid design in place, and that’s what I have described here with our own experience of moving an existing application to the AWS Cloud.

Approach

Examine the Current Application Deployment Architecture

Before moving your apps to Cloud, you must be familiar with your current application deployment architecture and its non functional requirements (NFR's) such as availability, scalability, security, performance...etc. This would enable you to clearly map the Cloud building blocks to your existing application architecture.

Let me show you this via an example,

In my organization, we had a CRM which was serving for about 100 concurrent requests per second, hosted in our own data center. While, we decided to move it to the Cloud, it looked like very simple thing to do. But, while we analyzed, we came up with the following non functional requirements that are needs to be addressed. I have detailed our solution to each of the issues listed below.

1. Application provisioning & configuration

2. Backup of the database required to be automated

3. Caching services using Memcache to be implemented

4. Single Sing On, the server is available in the enterprise data center and we had to authenticate from the Cloud

5. Emailing from the Cloud - Since it is a CRM, the main way of communicating to our customers are via email

6. Shared file system for storing files, images...etc

7. Secured Access using SSL

8. Payment gateway integration

9. SLA for application performance (every request should be served with a response, within 3 Sec of originally receiving it)

10. Auto scaling to support additional spiking need

Solution - Design your Cloud Deployment

Our Cloud Architects have designed the architecture to satisfy the NFR's listed above and they came up with almost completely, a new deployment architecture, due to the nature of AWS Cloud.

Application Provisioning & configuration

An application in the local data center is provisioned & configured once by the administrators. But our Architects proposed a new approach to do the same installation by automating the provisioning & configuration. The reason behind this is, when you want to automatically scale your applications to handle more loads, you may not be able to perform the following steps manually in the Cloud environment. Because doing this manually requires someone to monitor the performance of the Application, based on the load, he may need to decide to launch an another supporting instance to handle to load, when the application server instance boots up, the server engineer may need configure the required parameters such as Database instance IP address, Memcache server IP address, SSL certs deployment, changes in your web application configurations...etc manually. This is a tedious process and the human intervention required to do this can be completely avoided. Secondly, in the AWS Cloud though they have SLA's for the availability of their data centers in various regions, they do not guarantee for the life time of the instance. Which means at any point in time, your instances may fail( very rare) but, we need to accommodate the failure as well. "Design for failure", so that nothing fails, is the key for successful Cloud App. you can also, take a look at Jinesh's presentation on slideshare (http://www.slideshare.net/AmazonWebServices/aws-architectingjvariafinal)

Backup of your database

In general backup's are pointed to a local drive and the backup tools that you use can extract the content and store it in a local hard drive. Now in the AWS Cloud, you may use either S3 (http://aws.amazon.com/s3/) or EBS (http://aws.amazon.com/ebs/). Our architects have chosen to go with S3 due to its new reliability support at the API level. And we used CloudBuddy WEBDEV (you can use s3fuse, if it is a linux) to mount the S3 backup buckets to the EC2 instances running our CRM application database to completely automated the backup.

Memcache for caching information

Memcache is kind of a straight forward setup, but we had to configure each instance running the Memcache. We have decided to use CloudSmart to configure the Memcache instances.

Single Sign On

Single Sign on residing on our local data center premise provides domain verification mechanism and we used that feature to authenticate the caller machine from the Cloud before proceeding to the user authentication process.

Emailing from the Cloud

Emailing from the Cloud is kind of restricted beyond certain number of emails, which requires a special permission from AWS and we obtained the permission from AWS by submitting a form. You need to have a domain name associated with the instance to send emails to the outside world.

Shared file System

Shared file system can be achieved using SAMBA, NFS, network share...etc. Due to the nature of Cloud, it is a best practice to store the files in S3 for reliability, security & availability reasons. We have used CloudBuddy WebDEV to mount S3 bucket as a drive in both Windows & Linux operation systems. Alternatively you can also use S3FUSE to mount buckets as a drive for your linux operation system.

SSL Implementation

SSL was required to be installed in all the front ending machines in Apache, CloudSmart is used to automate the installation at every end point.

Payment Gateway Integration

Payment gateway integration requires Level 1 PCI compliance as per our policy, so we decided to keep the payment processing engine at our data center premise and redirect from the CRM hosted on the Cloud.

 Performance of the Cloud Application

The performance of the CRM is measured using our CloudTestGo platform and our Cloud Architect's have come up with the following recommendations to satisfy the 3Secs serving time requirement. This design is expected to serve 100 concurrent hits at any point of given time with a TTL for every request is 3 Secs.

          - Use CloudFront to serve the static content such as HTML, images, Javascript...etc

          - 3 Medium instances behind the load balancers, to handle SSL transformation to non-SSL requests

          - 2 Large instance running IIS with windows 2008 64-bit

          - 2 large instance to host the SQL Server database which is clustered

 Auto scaling was achieved for all the three layers such as web, application & DB layer.

Web Layer (Apache Servers) - If the web server instance gets more than 70% network utilized, the Auto scaling mechanism of AWS will launch an another instance of the same image. This will allow the installed CloudSmart to do the necessary configuration in the AWS environment such as adding the machine to the Auto Scaling group, copying the new httpd.conf...etc

Application Layer ( IIS) - When the IIS instance gets more than 70% network utilized, the Auto scaling mechanism of AWS will launch an another instance of the same image. This will allow the installed CloudSmart to do the necessary configuration in the AWS environment such as adding the machine to the Auto Scaling group, Copying the ASP.NET application, mounting EBS volume as a drive..etc

DB Layer (MS SQL Server 2008) - All the calls to the database was distributed by the SQL Server Cluster itself and the load on the individual cluster node will automatically launch an instance and create necessary configuration to join the Cluster group.

Conclusion

It is understood that moving apps to Cloud is straight forward but the key factor is to use the advantages of the Cloud, which is nothing but automating every aspects of your application deployment. And that can be very well achieved through the available open source technologies itself.

To become a Cloud Developer

What is Cloud Computing?

Cloud Computing is nothing but providing Information Technology (IT) components such as computing, storage, network bandwidth,security, platforms and software as a service. It is considered to be a cost effective method, because you need not pay upfront a hefty amount and you will pay for what you use.

Classification of Cloud Computing Services

There are three types in Cloud Computing Services .

1. IaaS - Infrastructure As a Service

The base IT infrastructure components such as Machines for computing, Storage, Network is provided as a service. Example : Amazon AWS, Rackspace Computing Cloud & Microsoft Windows Azure.

2. PaaS - Platform As a Service

A solid platform in which you can develop, test, deploy & offer your applications to your end customers. Mostly your applications are offered as a Web & Web Services technologies. Example : Microsoft Azure, Google App Engine, force.com from SalesForce.

3. SaaS - Software As a Service

Software applications provided as a service, As a Cloud Developer, mostly you will consume the services to enhance or enrich their existing offerings. Example : Google Apps, SalesForce.

Simple Analogy to Cloud Computing

I am sure we are not new to the idea of pay per use, ie, utility model, we have been using electricity, telephone, mobile phones in the utility mode and the government & telecoms providers have invested a lot in building the infrastructure to provide best services to you. In the similar way, Cloud Providers like Amazon, Microsoft Azure, Gogrid, Google have invested a lot and created the infrastructure to be used by end users like us.

What does Cloud Computing Solves?

Cloud Computing is going to change the way we compute soon. It is important for the developers to understand that many problems that we currently face like application not scaling, database reached maximum no of concurrent connections, single server deployment...etc are being addressed by Cloud Computing.

I am a Developer, Why should I worry about this?

In a traditional way, If you are a web developer you would have learned either Java, .NET, PHP...etc along with one or more RDBMS technologies and frameworks related to your technologies. Especially , if you are first timer, you would have spend most of your time configuring the related software in your machine rather than developing the code that you like.

Cloud Computing brings the flexibility for you to quickly start using the configured machine instances rather than you setting up things on your own. in Amazon terms, it is refered as Machine Images. A Machine Image is nothing but a package comprises of Operating System, required softwares and its related configurations. This package can be used to while launching your instances. Most of the traditional Software Providers such as Microsoft, IBM, Oracle, Redhat has already started providing their software packages as machine images in Amazon Cloud.

Cloud Computing provides various application building blocks that are commonly used in large scale application development & deployment such as message queues, databases, application containers and extendable storage for backup as a services and you can develop your applications to leverage those building blocks rather than breaking your head to deploy those components on your own. As a Cloud Developer you should be familiar with all these building blocks and their provider specific API's.

In a nutshell, If you want to be Cloud Developer, you must understand all these building blocks and their API's to develop applications for Cloud Technologies.

Where do I Start?

"Practice Makes the Man and Women Perfect", you should start using the Cloud Computing technologies to get a feel of it.

Before you start, you must decide on a provider first, aligning yourself to a provider, makes you an expert in that platform too. When I started my learning way back 2006, I have chosen AWS platform. I would suggest, if you are Microsoft developer, you can start with Microsoft Azure by learning SQL Azure, Windows Azure...etc. If you are Java & Python developer, start using Google App Engine. If you are a CRM/ERP app developer, you can choose SalesForce.com.

Welcome to Cloud For Developers Blog

If you are a developer or a software architect, irrespective of the technologies, you must have heard about Cloud Computing. I have been receiving lot of questions from Developers of all technologies about Cloud Computing and how it is going to change their life in the future.

I am sure this question will be there in most of the developers & architects mind. This blog is dedicated for developers & software architects who wants to become Cloud Developers & Cloud Architects!

Stay tuned.