No-Click Web Deployment – Part 1
Update: Part 2 is now available, covering Web Deploy (a.k.a. msdeploy)
Getting code deployed should be as easy as open up a web site (only taking slightly longer :-)). Devs or IT people should not spend manual labor (with the possibility of mishaps) on getting files from one place to another, making changes to IIS (or whatever you’re using), restarting servers, copy/changing web.config files etc. That’s the job for scripts and automation tools. Not to mention the cost savings of not needing IT people to do deployment. I bet you there’s no one manual step in the process of deployment that cannot be automated. Saying that, you have to consider how many times you deploy per day/week/month or year before going for full automation. However, if you’re serious about being Agile/Lean, you can’t do without a auto deployment scheme.
In the coming blog posts I’ll walk you through the steps I went through to automate our deployment, and hopefully you’ll find it interesting and even suggest improvements.
Overview
Below is an overview of the environments we’re deploying to:
One Load Balancer (LB) in each environment, two web servers in Dev and Test, and 3 in Prod. The actual numbers might or might not be true ;-), but that doesn’t really matter. In addition there’s SQL Servers, but I will not cover that here.
Why have a LB in Dev? Reason number one is to catch any possible LB issues in Dev before going to Test and Prod, and have the exact same environment in Dev as in Prod. It’s also useful to try out new stuff, like having the LB do caching etc.
Deployment Frequency
For Dev we auto deploy every night (part of nightly build) and at will during day. For Test, 2-4 times per week and to Prod 1-2 times every 2nd week. That was yesterday! :-) Today we can do it when the sun comes out of the clouds (not often in Bergen), every time I refill my coffee cup or whenever we feel like. The point being: we are no longer constrained by how often we can deploy.
Why All These Environments?
You can read about that here, but for us:
- Dev is where we try out things without physically hurting users, but still being in a real server environment avoiding the “works on my machine” issue.
- Test is as close to Prod as we can get (at external hosting provider, different network, firewalls etc) and where we make sure things run smoothly before going to Prod.
- Prod is Prod
Tech Details
Here’s the tech stuff we use which might be relevant:
- All servers are running Windows Server 2008 R2
- Web servers are running on IIS 7.5 (since we’re on R2)
- Application Request Routing in IIS is used as Load Balancer and runs on 2008 R2 Server Core (if you like, check out my previous post about setting up and configuring ARR)
- TFS 2010 for builds
- Team City for CI
Also note that we have access to the actual subnet where Test and Prod lives. This does however not mean we have access to all servers and features in all environments, it just means we can be given access to certain things not recommended through external firewalls, like PowerShell Remoting. This is where your environments might be different from ours.
Some General Advice
Consider Using a LB Even If You Don’t Need One For Performance Reasons
Load Balancers are useful for other things than load balancing. The biggest benefit (except from its core task), is that you can do upgrades and maintenance on servers without taking the whole site offline, by always leaving at least one server online.
Consider Turning Off IIS Recycling
Do you know that IIS automatically recycle your applications every 1740 minute, effectively restarting them? Are your web sites free from memory leaks or do you want to know if you have memory leaks? Why not turn off recycling? This is too big of a topic to cover here, but go Google: IIS7 recycle.
Consider Using Windows Server 2008 R2 Server CORE
This should get you slightly better performance, but for me it’s more about scripting. Most of the things that needs to be done on server core, must be performed from command line, forcing you to create scripts.
Why Is Deployment Difficult?
First of all because every environment is different and there are no really good tools to automate the whole process. The challenge is to find the right tools to solve the problems your organization is facing, and have the tools work for you to get to the final goal.
What’s The Challenges?
For us it was about:
- How can we safely move files from a build server to Dev, Test and Prod?
- How can we automate the process of taking a node out of an LB cluster?
- How can we safely execute an upgrade on a server in Dev, Test or Prod and get feedback of progress, errors, and abort and roll back on failure?
- How can we remotely make changes to IIS?
- How can we avoid all manual tasks? (like adding a virtual directory in IIS or copy a web.config file)
Safety and automation is two keywords that sticks out. Where safe means no-one else than the intended persons or services should be able to perform the specified tasks. Automation meaning no manual operations should ever be needed in either Dev, Test or Prod except from IT maintenance like hardware upgrades, windows update etc.
What tool options have we?
Copy files:
- Secure FTP in IIS 7 on a non public/available IP
- or PowerShell with BITS
- or WebDeploy
Taking LB nodes offline/online:
- Use PowerShell Remoting to execute PowerShell scripts on ARR server
- or the Web Farm Framework
Safely execute an upgrade:
- Use PowerShell Remoting to execute PowerShell scripts on web servers
- or WebDeploy
Avoid manual tasks:
- Script all tasks, so they can be repeated
The Build/Deploy Process 
What About MSI’s?
If you read my blog you know I’ve done quite a bit of MS Installer stuff and WiX in particular. MSI’s are perfect for deploying to multiple places where you have no control. The drawback is that most developers don’t know how to customize MSI’s and often end up with a versioning problem and leaving lots of old stuff behind on the server after upgrades. If you have people skilled in Windows Installer, please feel free to use MSI, but I personally find XCopy to be very easy and is what I recommend if you’re not an ISV. With MSI’s you still have to install them remotely, which could be done with WMI or PowerShell.
Notes on WebDeploy
I’m currently looking at using Web Deploy to simplify/reduce the amount of scripts needed. WebDeploy would replace the FTP and deploy steps, but first impression is that it’s too generic, making it really hard to do simple things without spending quite a bit of time learning the tool, it’s underlying package schema and IIS schemas. Hopefully one day Web Deploy will be the only tool I’ll need to execute the whole deployment process.
What’s Coming?
In future blog posts I’ll walk you through step-by-step how to accomplish the above solution. While I’m writing this I’m not 100% sure if it will be a solution using PowerShell (which I have in production) or a slightly modified version using Web Deploy. It all depends on which one is easiest and which has the potential of being maintained by other people than me in the long run.
Hopefully this will give you the input you need to fully automate your deployment process as well.
