In my previous post about Azure Services Platform I summarized many of David Chappell’s points in his talk last week as well as Clemens Vasters intro to .Net Services. One of the things David mentioned was other cloud providers that exist today.

One that I find particularly interesting is Amazon. In this article I will cover the Amazon Web Services and how they map to Azure Services Platform. Amazon is my starting point, so I will not talk about the services that Microsoft have that Amazon don’t, unless they are tightly connected to one of Amazon’s services. One thing to note though is that Amazon has been offering cloud services for a while and some of their services are already out of Beta. MS however is just getting started so it’s not easy to know where MS is going to end up compared to Amazon. In other words; this article might be out of date pretty soon! :-) Saying that, some design decisions needs to be taken early and some decisions can’t easily be reverted, so many of the differences and similarities between the two providers will probably be around for a while. And when people from Microsoft are going to Amazon, maybe the similarities between the two will increase even further… Actually, talking about mappings, lets get the first mapping out of the way: Amazon Web Services maps to Azure Services Platform.

Amazon EC2
Amazon Elastic Compute Cloud (EC2) is a virtual computing environment that let you create virtual servers in the cloud. Either Linux or Windows based. Windows based OS’s (Windows Server 2003) are in Beta and only available at the datacenters in the US, but I’m expecting Europe to provide this service soon. By creating or using a predefined Amazon Machine Image (AMI) and adding your application to the same image, you get your application into the cloud. By using the EC2 web services or any of the configuration applications out there (like Elasticfox) you can configure your virtual images, security, network access and more. To do actual work on the image (like installing applications) you just RDP into that server and do your install or configuration.

Amazon EC2 vs. Azure Services Platform
EC2 can be compared with Windows Azure. What Amazon let’s you do which Microsoft don’t, is have direct access to the virtual image. You can RDP (Remote Desktop) into the server and do whatever you want and install whatever. Windows Azure gives you an API where you can configure stuff like logging and alerts. According to MS they will provide more services to interact with Windows Azure later, but I don’t think you will get access to the actual virtual image(s). To add applications to Windows Azure you need to use either the Web Role or the Worker Role. However, windows Azure maintains the OS for you (updates and such) while on EC2 you have to do this yourself. Amazon today gives you a Windows 2003 Server while on Windows Azure it’s Windows 2008 Server. On Windows Azure you can configure how many instances of your Web/Worker role you want for scaling, while on EC2 you need to use the provided web services to spool up new instances.

One thing Microsoft provide that Amazon don’t is the ability to have a local environment of “your cloud”. This allow you to do development on your local computer in a simulated cloud environment, which again will let you do things like debugging. You can of course to this on EC2 as well by installing Visual Studio :-)

A common requirement for both is that the EC2 image and your Web/Worker roles need to be stateless. If not, scaling would just not work the way it’s indented.

For more information: Amazon Elastic Compute Cloud (EC2) and Windows Azure

Amazon S3
Amazon Simple Storage Service (S3) is where you persist your data/files. Here you can store anything from files you need for your server to databases, downloadable files, or any files you want to store in the cloud. S3 exposes a web service you use to write, read or delete files (objects) up to 5 gig each. The number of files you can store is unlimited, but you of course have to pay for storage. The underlying structure is flat using a key/value pair to write and read (or delete). They use names like Buckets, Keys and Objects. A Bucket is similar to a file folder, a Key is the unique identifier for your object within a Bucket and the Object has metadata about the object you have stored (like created date, modified date, content type etc).

Amazon S3 vs. Azure Services Platform
Comparing this with Microsoft’s services is not a one-to-one mapping. To explain this I need to give you some details about the data services provided by Microsoft. First there are two main services:

1. The data storage facility in Windows Azure
2. Microsoft SQL Data Services (SDS)

In Windows Azure you have three storing options:

1. Blob
2. Table
3. Queue

And to make the story short: Windows Azure with its Blob service is similar to S3 in my opinion.

The ability to query data on S3 is limited compared to Windows Azure, but Amazon provide this service in their SimpleDB instead which integrates with S3 (more on this later). In Windows Azure you can use a LINQ like query language, but on S3 you’re limited to using a pre-known bucket and key. Both do however have similar url access:

S3 example: http://torresdal.s3.amazonaws.com/photos/me.jpg

Windows Azure example : http://torresdal.blob.core.windows.net/photos/me.jpg 

In the S3 example; photos = Bucket, me.jpg = Key

For more information: Amazon Simple Storage Service (S3) and Microsoft SQL Data Services (SDS)

Amazon EBS
Amazon Elastic Block Store allow you to do two things:

• Create volumes
• Create snapshots

Volumes are storage volumes from 1GB to 1TB that can be mounted as a disk on EC2. They’re also placed in a Availability Zone, meaning you can only attach volumes to EC2 instances that run in the same zone.

A snapshot is a copy of the complete volume at the time you took the snapshot. If a snapshot already exist, only the diff will be saved to save storage. All snapshots is stored in S3 for later retrieval. You can then use your snapshots to create new volumes based on one of your snapshots.

Public Data Sets is free snapshots everybody can use. They contain big databases for things like public DNA data, transportation data etc. Typical data used for research. Though I’m not in that type of business or research, I see the perfect match for the service they’re providing.

If you want to use your own database server (like SQL Server) you would use an EBS volume to store the database files to persist data. This will make sure that all your data are persisted in case the server goes down for some reason or another.

As a side note I have to admit every time I read Elastic Block Store, my mind reads Elastic Book Store, which kind of makes sense considering who provides the service :-)

Amazon EBS vs. Azure Services Platform
As far as I know there is nothing equivalent on the Microsoft platform. Since they don’t have an open interface giving you direct access to the OS there is no need.

For more information: Amazon Elastic Block Store (EBS)

Amazon SimpleDB (Beta)
Amazon SimpleDB lets you store, process and query data in the cloud. It is designed to scale and developers use provided web services to integrate with the service. There is also a connection between SimpleDB and S3 allowing you to query data in S3. SimpleDB is structured into domains which contains items which again contains attribute-value pairs, resulting in a flat and highly optimized data structure. Querying data is done through either the Select API or the Query API.

Amazon SimpleDB vs. Azure Services Platform
Comparing SimpleDB to MS is not strait forward. As mentioned previously when I talked about S3, Microsoft provide several services in this area. You have the data storage in Windows Azure with its Blob and Table data structure, and you have Microsoft SQL Data Services. All of these can be compared to SimpleDB, though they’re not an exact match. As the word Simple is suggesting, Amazon is providing a simple interface for storing data. Microsoft is giving you more options to how your data is going to be stored and structured, but all in they all have a flat structure similar to Amazon’s SimpleDB. Microsoft is however building on top of their existing database investments of SQL Server and provides similar services that we know from that platform in the cloud.

For more information: Amazon SimpleDB, Windows Azure Tables and Blobs, and Microsoft SQL Data Services

Amazon SQS
Amazon Simple Queue Service (SQS) is, you got it, a queue. As with the other services they provide this is highly scalable and they provide unlimited numbers of queues to use as well as unlimited number of messages to add to a queue. The body of the message is limited to 8KB and a massage can be stored up to 4 days before it gets deleted. Though the message body is limited, you can use the body to reference data stored in e.g. S3 or SimpleDB. They provide you with four API’s: CreateQueue, SendMessage, ReceiveMessage, and DeleteMessage, and are exposed as web services.

Amazon SQS vs. Azure Services Platform
Compared to Microsoft this is the queue storage type in Windows Azure as I mentioned above (in the S3 section). But queues are often used in conjunction with workflows, and here Microsoft provide its own service. Amazon however does not have a separate service for workflows, but encourage the use of SQS for this. The Windows Azure Queue as described now has a main purpose of providing a communication facility between the WebRole and the WorkerRole. As with SQS also the Azure Queue have the same flexibility and limitation (unlimited queues, unlimited messages and a message size of 8KB). I could not find any information about how long MS stores messages in a queue, though I expect there to be one :-)

For more information: Amazon Simple Queue (SQS), Windows Azure Queue and Microsoft .NET Workflow Service

Amazon CloudFront (Beta)
CloudFront is a service for distributing files. All files are stored in S3 but CloudFront is using a network of edge locations, so that users downloading/accessing files will get it from a location close to him or her. When a user request a file/object CloudFront will automatically route to the nearest edge location for best performance.

Amazon CloudFront vs. Azure Services Platform
Currently I cannot see a similar service from MS. There has been some speculation that Microsoft’s cache solution codenamed “Velocity” is their service in relation to CloudFrount, but I don’t see the match.

Conclusion
The similarities between the services provided by Amazon and Microsoft are striking. There are a lot of similarities on how they store, manage and query data. Especially that they both have chosen a flat data structure with no relational tables, which is the only sane thing to do when you want to scale I guess.

The major differences are around how they expose their operation systems to the developer. Amazon gives you access via RDP to the virtual server, while MS adds services on top of their virtual servers that you can integrate with. Also since Amazon is not an OS or application manufacturer they are freer to chose which technologies to provide. Here Microsoft is more restricted to their own platforms and technologies, though they promise to support other programming platforms than .Net.

So the final question to all of this have to be:

Which one would you choose?

Based on what I know about the two service providers now, I would go for the consultant answer… It depends :-) and answer with a new question:

What are you going to use it for?

If you have an existing application built around SQL Server that does not need to scale very much (like an enterprise app), Amazon seams like a good solution right now. If you’re starting from scratch and writing a new application that suits the cloud model, then I’m not that sure. Technology or programming platform will probably impact your decision. Looking into the future and if your a .Net developer choosing MS is probably a safe bet, because you would get great service support on their platform as well as a local version of Windows Azure on your desktop. On the other hand, Amazon’s solution is quite open and does not limit you to any programming language. The race is open and I’m excited to see where they are in a year or so…

A few years ago I did some presentations on memory leaks in .Net. I pointed out a few things you needed to be aware of even though you have a garbage collector. Like how you manage your events in .Net. Anyway, back then Rico Mariani was a great source for me (or actually his blog was)with all his knowledge about memory and performance stuff.

Summer 2007 Rico became Chief Architect of Visual Studio which gave me hope :-) I think we already start to see his impact on the product. If you have been following Rico’s blog you know that he has been working with performance for almost 30 years!

Anyway, on his blog he have a few articles about where VS 2010 is heading and what we can or cannot expect. He also talks about the overall plans for Visual Studio (and not only 2010) which I found to be interesting reading. Here’s a couple of links from his blog:

http://blogs.msdn.com/ricom/archive/2008/11/18/the-visual-studio-tech-roadmap-starring-visual-studio-2010.aspx

http://blogs.msdn.com/ricom/archive/2008/11/25/visual-studio-dialogue-with-wpf-performance-emphasis.aspx

One of the things that have come up is the use of WPF. WPF in Visual Studio? Is that a good idea? Well, as Rico says: Do you really think GDI is the last word in computer graphics for the next 10 years?

Check this video out where he more or less summarize what he said on his blog about VS: http://channel9.msdn.com/shows/Going+Deep/Rico-Mariani-Visual-Studio-Today-Tomorrow-and-Beyond/

Below is a couple of other interviews he have done lately:

http://channel9.msdn.com/posts/VisualStudio/Paramesh-Vaidyanathan-and-Rico-Mariani-The-Future-of-Visual-Studio-Extensibility/

http://deepfriedbytes.com/podcast/episode-21-talking-performance-with-performance-preacher-rico-mariani/

Back in April last year I configured Google Apps for my family domain. Email was goal #1, but I also had a plan to start using and sharing Google Calendar with my family. At the time however there was poor tooling support for syncing Outlook with Google Calendar. Since I’m using Outlook as my main source of event planning, it was natural for this to be my primary source with a 2 way sync to Google, so I could share this info with e.g. my wife. I actually started on a small dev project to get this done, but I soon got tired thinking Google or somebody else would create something for this soon.

It turns out I was right, only I didn’t pay attention :-) Long story short: in March this year Google released Google Calendar Sync which allow you to do 2-way synchronization between Outlook and Google Calendar. I know this is old news, but it slipped my radar and it might have slipped yours as well…

In my previous post about Azure Services Platform I linked to where you could find David Chappell’s presentation that I attended. Now Børge Hansen have made Clemens Vasters slide decks available too. You can find them all here:

David Chappell (Azure Services Platform): https://cid-88b5769d9bbdf846.skydrive.live.com/self.aspx/Offentlig/Azure%20Services%20Platform–Chappell.pdf

Clemens Vasters (.Net Services overview): https://cid-88b5769d9bbdf846.skydrive.live.com/self.aspx/Offentlig/NETServicesOverview-v2.pptx

Clemens Vasters (.Net Services drilldown): https://cid-88b5769d9bbdf846.skydrive.live.com/self.aspx/Offentlig/NETServicesDrilldown-v2.pptx

Last week I attended 2 sessions about the new Microsoft platform in the cloud: Microsoft Azure Services Platform. The first was held by David Chappell and the second by Clemens Vasters. Two great speakers. Unfortunately I didn’t get to see Clemens complete talk, because I had to catch a flight back home, but hopefully I got the main content of his talk.

In this blog post I’m going to ask and answer some questions by the help of these two sessions + the additional study I’ve done around this topic.

Disclaimer: The answers I provide below is based on my current knowledge of this platform and might not be correct now or in the future, though I’ve spent a fair amount of time making sure it is.

What is it?

It’s a platform where you can deploy and manage new services or even complete web applications into the “cloud”, in addition to make use of existing services already provided by the platform. This is often referred to as cloud computing. Today the Azure platform contains the following services: Windows Azure (the platform), Microsoft .Net Services, Microsoft SQL Services and Live Services. Microsoft SharePoint Services and Microsoft Dynamics CRM Services are also mentioned in relation to the platform. All of these services make up the Microsoft Azure Services Platform. By reading these names you may see the potential naming confusion? When you say Azure, do you mean Windows Azure or Azure Services Platform?

Now you may ask, what is it really? From a simplified developers perspective it’s a platform on the internet (the cloud). Instead of hosting your web application/service on your local server, you publish it into the cloud. In addition it gives you access to the services you need to build a multi-tier architecture. From another perspective it’s big monster of a datacenter providing you with a lot of data power (virtual Win 2008 servers) to do your stuff, giving you the option to scale up and down at will or dynamically.

How committed are Microsoft to this platform?

As committed as they’ve never ever been before with anything as far as I can remember! They invest heavily in the technology and infrastructure needed to support this platform. By heavily I mean they’ve told the marked that this is such a huge investment that it will highly impact Microsoft’s money bin. As an example of this, here’s a short description of what they’re doing with their datacenters needed to host this beast:

Every datacenter is set up of a grid of physical and virtual computers. These computers are delivered by HP and Dell and some in preconfigured shipping containers ready to be plugged in. I don’t mean the computers are ready to be plugged in, but the containers! What Microsoft then does is put a bunch of these containers close to a power plant (for obvious reasons), plug all the containers in and they’re up and running. At least that’s how it was described to me :-) The applications, services, data etc you have put into this cloud is distributed across many VM’s, meaning if a VM or server (hosting VM’s) goes down, it’s not a problem. There is always some VM on some server holding your data. Actually the shipping containers I mentioned are remotely managed and MS never enters any of these containers for maintenance. If a computer goes down and does not come up again, it just stays down. If this happens to a certain number of servers within one container, the whole container is disconnected from the grid and sent back to HP or Dell to be fixed, without affecting any of the applications or services! This little story told by Clemens Vasters really says a lot of the scale on this monster!

Can I take my existing application and deploy it to the cloud using Azure Services Platform?

The short answer is: no. The longer answer is: you might in the future. First of all the platform is still in Beta, but beside from that David Chappell says: “Azure Services Platform today targets the next Facebook”, meaning it’s highly scalable and can support a lot of users. To support this level of scalability there are some compromises to be made. On Azure this is (among other things) how you store and access data. Using a plain SQL server with relational tables and ADO.net would just not scale/perform. Thereby they decided to implement a different solution where the data are stored hierarchically and data is exposed via a RESTful interface using a LINQ like query language. Unless your application have this type of data implementation today, you would have to rewrite your data access and data layer. But don’t take my (or David’s) word for it, check out this video about table storage (which you would think is normal relational tables, but it’s not) from MS and you can make up your own mind: http://wm.microsoft.com/ms/msdn/azure/scalablestoragetables.wmv. Note though that in this video they’re using Windows Azure Storage and not SQL Services, but the same is true for SQL Services.

If I can’t deploy my existing app to Azure Services today, what CAN I do?

The obvious thing is to wait and hope Microsoft will add support for relational tables allowing you to use existing SQL Server data implementations, but the Azure Services Platform is much more than a place to have your application and data. Your (local) application might want to interact with services in the cloud to enhance your existing application. I find the Service Bus to be especially interesting, or even the Access Control Service. This is only from my perspective and within the enterprise domain where I work. If you work with a more consumer oriented domain, you might find the Live Services interesting. Note that you don’t have to use the complete platform to make use of Azure Services. You can of course select the ones giving you value for you particular domain.

What’s the difference between Microsoft SQL Services and Windows Azure Storage?

First of all these two services looks very similar today and one might think why have both? My impression though is that Windows Azure Storage is going to remain much the same, but SQL Services are going to expand quite a bit (reporting, analytics, synchronization etc). Clemens offered this simplified analogy: “Think of Azure Storage as the registry on Windows, and think of SQL Services as your SQL Server.”. Another important difference is that Azure Storage is not built on existing SQL Server technology, but SQL Services is. Another interesting thing pointed out by David is that none of these services use SQL!? They use a RESTful interface with a LINQ like query language. He offered one explanation to this: SQL within MS means SQL Server :-)

What’s the Access Control Service and is it limited to only Azure apps?

The Access Control Service is “Microsoft’s humble attempt” to support as many access control providers as possible, so that this service will become a common place to perform identification. They state that “No one is willing to take this responsibility, so we have to” :-) Just think of it. How many providers of access control do we have today? Google, Microsoft, Amazon all have their own personal identity services on the internet. In addition there are a lot of others like Active Directory, Novell eDirectory, Apple Open Directory etc. What if this service would support the majority of these? Then you could use this service to do the access control for your system being in the cloud or not, and the end user could use his provider of choice. For more information about the Access Control Service check out the MSDN site: http://msdn.microsoft.com/en-gb/library/dd129876.aspx (note that this url might be quickly outdated).

By choosing the Azure Services Platform, will I be locked in by that platform?

By “locked in” I’m here going to assume you mean if you can move your cloud application away from this platform and on to another if you want to. Taking into account the current providers of cloud platforms we have today and the current services they provide, my impression is that you will be locked in. Just because of the simple fact that no one else is supporting .Net like Microsoft do. Hey, wait a minute… What if I use Amazon’s EC2 which gives me a Windows Server 2008 and a SQL Server, can’t I then move to that platform? That environment support .Net just fine. Well, it depends… :-) If you’re using the Azure Storage or SQL Services, you can’t cause it’s just not a plain SQL Server. It seams like the common thing to do among cloud services providers is to create their own data access solutions (or even programming languages like SaleForce.com have done), meaning you will be locked in on that. If you’re not using the data storage solutions you might not be.

Why should I be afraid of being locked in anyway? I’ve already chosen Microsoft as my technology provider. Yea, but now you’re also locking onto your host. If your host (in this case MS) decides that the solution you’re running in their environment should be increased by a cost of 500% and you can’t afford that, what do you do? Find another cloud provider?

However, one interesting thing to note about Azure Services Platform is what MS says about programming languages. They want (though they don’t do today) to support many other languages like Eclipse, Ruby, PHP and Python. Google only support Python, SalesForce only support their own language Apex, Amazon basically provide you with VM’s (Linux and Windows) leaving you to choose. Another interesting aspect is that MS seams to be very open about what and how they’re doing things. 

People are talking about the Web Role and the Worker Role. What’s that all about?

The Web Role and the Worker Role is part of the Windows Azure fabric (see separate section for more fabric info). Microsoft says:

A web role is an ASP.NET web application that may be accessed via an HTTP or HTTPS endpoint. Web roles run in integrated pipeline mode on a hosted IIS7 web core. 
..and/or:
A web role is a web application accessible via an HTTP and/or an HTTPS endpoint. A web role is hosted in an environment designed to support a subset of ASP.NET and Windows Communication Foundation technologies.

A worker role is a background processing component. It does not accept inbound network connections.
..and/or:
A worker role is a background processing application. A worker role may communicate with storage services and with other Internet-based services. It does not expose any external endpoints. A worker role can read requests from a queue defined in the Queue storage service.

A service may consist of a single web role, a single worker role, or one of each.

When you create Azure projects in Visual Studio, you select which Role you want your project to be (or both). As described above the main difference between the two is that the web role allow incoming traffic, while the the worker role does not.

What’s “The Fabric”?

The Fabric, or more accurately Windows Azure fabric, is the operating system that the services run on. Again from MS:

The Windows Azure fabric offers an Internet-scale hosting environment built on geographically distributed data centers. This hosting environment provides a runtime execution environment for managed code. The fabric handles load balancing and resource management and automatically manages the life cycle of a service based on requirements established by the owner of the service. The developer includes with the service code specifications for the service topology, the number of instances to deploy, and any configuration settings. The fabric deploys the service and manages upgrades and failures to maintain availability.

To try to simplify (in my words): Below the fabric you’ll find a lot of VM’s. On top of the fabric you’ll find Compute and Storage. The fabric host .Net and take care of networking stuff.

What if I’m not going to build “The next FaceBook”, is this still something for me?

Absolutely. Even though the architecture behind it all is built so that it can handle the next Facebook, it does not mean you can’t use it for an enterprise solution which have less than 1000 users or an application you want to share with your friends. It will be interesting though to see if MS will provide us with different data models, so that we don’t have to use a really scalable one, when we will never need it, also allowing us to port existing solutions onto their platform.

What’s it going to cost?

No one knows. At least not outside of Microsoft. We do however know some thoughts they have around the pricing model(s). It does not look like they’re going for a “simple” price model as e.g. Amazon which charge per hour. They will probably use a combination of CPU, network, storage and transactions(data) to calculate the price. More details can be found here: http://www.microsoft.com/azure/pricing.mspx (note that this url might quickly become outdated).

Who else out there offer cloud services?

According to David Chappell; Amazon, Google and SalesForce.com are the leading providers today. All of these provide different solutions and pricing models.

Will everybody now deploy their apps to the cloud?

Not now, but I defiantly see this as a very attractive hosting model in the future, and for some it is already today with services provided by e.g. Amazon. I would think it’s especially interesting for startups that does not have the resources (money and knowledge) to set up a datacenter for hosting their solution.

Hi, this is your mom. Do I need to know what this is?

No. You together with the rest of the world not running an IT company or doing programming can happily not care and just use whatever services that will be provided through this platform without you knowing. Btw mom, you don’t have to read my blog if you don’t want to ;-)

Where’s Hello Cloud?

It’s here: http://wm.microsoft.com/ms/msdn/azure/hellocloud.wmv
..and implemented here: http://hellocloud.cloudapp.net/

Are any of the material from the sessions you attended made available for the public?

You can find David Chappell’s presentation here: https://cid-88b5769d9bbdf846.skydrive.live.com/self.aspx/Offentlig/Azure%20Services%20Platform–Chappell.pdf