Rob's Ramblings

Deploy Your Web Application by Checking in Code

I’ve always wanted to do automated deployments by checking in code.  I feel strongly that any files on a webserver should be in source control.  So, why not take it one step further and let the code check-in do the deployment as well.  The methods below are in use at the company I work for.  I’ve been using this method for 6 week or so now and have had close to 100 automated deployments with this method.  Although its not perfect, it’s a very good start.

Components

I am using the following components.  By no means do you have to use the exact same technology here.  This post is meant to promote concepts and not necessary technology.  As long as the technology of your choice is capable of these concepts, you’ll be up and running in no time.

This is a list of the technology/software that I am using, but as mentioned above, these concepts don’t require the same technology, just technology that can do these things.

Source Control
Starting with the basics is source control.  In my case, I am using Git but really, any source control system that has fairly lightweight branching is completely fine. What I mean by this, is that you need to be able to merge changes from one branch to another very easily.  Git and Mercurial are perfect for this.

Branch Structure
I am using braches to manage my deployment environments.  So, I have 4 branches that I manage for this on top of my feature branches that I create.

Master/trunk/head – this is the branch I use when I’m done with a feature and ready to share it with my team.  Its suppose to be bleeding edge code.  You may or may not have this mapped to a deployment environment.  In our case, I do not simply because I want this branch to be an easy way to share code with my team, but it may not be ready for deployment to a web server.

Dev – this branch is mapped to our Dev server.  So, when I move code into this branch, I typically want someone to test out a feature before I’m ready to push it out to beta/QA.  A good example of the usage of this is that I want a co-worker to play with a new rollover menu that I’ve created and want to see their interaction with it.  I watch how it works for them, and go back and make changes.

Beta – this branch is mapped to our Beta/QA server.

Production – this branch is mapped to our production environment

Shared Repository
I am using Unfuddle for a variety of things, one of those being our shared source control.  Unfuddle provides SVN and Git hosting (to date).  I have all my origin’s setup to Unfuddle to make things a little simpler.  This really just needs to be any place that multiple users can get to the code.  SVN by default as this, Git does not as it’s a distributed SCM.  This could just as easily be a file share on our internal network.

CI/Build Server
We are using TeamCity as our build server.  I chose TeamCity as it seemed to fit our needs the best.  I’ve used CruiseControl.net in the past but I hate XML files so I didn’t want to use that again, however CruiseControl.net would work just the same.  See this post on how I’ve hooked up Git/Unfuddle and TeamCity

Build Scripts
I’ve played with a couple of build scripts (MSBuild, Nant, UppercuT, psake, and Albacore).  I chose Albacore for a number of reasons but mainly I like Ruby.  If you are a Powershell guy, go with psake.  If you like Nant but don’t like XML, go with UppercuT.  The point here is not the technology but the fact that you have script-enabled builds.  If your project doesn’t have this, CREATE THEM and get those scripts in SOURCE CONTROL!

Deployment
Up until this past year, my deployments were done by either, FTP, xcopy or drag and drop with Windows Explorer, and in a lot of cases, a combination of all of these.  This was the case until Web Deploy entered the picture.  If you fit into the FTP/XCopy group, please watch this presentation by Scott Hanselman.  Trust me, this will make your life much better.

The Good Stuff

Okay, enough of the definitions and documentation, show me the code already!

My folder structure:

image

The build scripts are in, you guessed it, Build folder.  Here I have a sub folder for each build/deployment environment:

image

You must have build configurations in your Visual Studio Projects/Solution that match in some way (they don’t need to be named the same).

image

Build Scripts

image

All this really does it call msdeploy.exe with parameters (that’s what is in my .bat files).  I let msdeploy do the heavy lifting of syncing and deploying to the web server.

My common folder has assemblyinfo.yml and build.yml to define common properties (e.g. path to solution, etc.) for all builds.

Build Server

TeamCity configuration I’ve setup to use Rake and the build runner.

image

Trigger Deployment

image

The big thing here is that you simply merge changes in from master or wherever you want.  Also, if you need to make fixes in Production for example, make those fixes on that branch and simply pull those changes back to the master.

The final step that I do is have TeamCity tag the repository on every successful build.This way, I’ve got the branch tagged for every deployment. This is helpful if I need to rollback a changeset.

That’s about it.  Once you’ve got this setup, you’ll never regret it.

December 31, 2010 Posted by | Programming | , , , , , | Leave a comment

Migrate your repository from Team Foundation Server to Git

My company is in the process of decommissioning our TFS server and migrating our source to Git. I wanted to preserve the commit history so I couldn’t just create an empty Git repository and move the files into it.  Instead I am using a few tools to keep the history.

The first is called SvnBridge.  This is an open source project that allows you to Subversion clients with TFS.  This only gets me half way however.  To finish the process and create my Git repository, I simply use the git-svn commands.

In my case, I am using the SvnBridge-Client application so I don’t have to try and get the server version installed on our TFS server.  Simply download the latest Client from CodePlex and run it.  This will start the server locally.  You will see the icon in your notification area.

You can specify the port number.

image

Next I enter in my Git command by simply doing:

git svn clone http://localhost:8080/<tfsserver>/<tfsproject&gt;

image

That’s it.  You now have a Git repository with all of the history from TFS.

November 2, 2010 Posted by | Programming | , , | Leave a comment

Setting Up a Callback Url from Unfuddle to TeamCity

I’ve been trying to get our Unfuddle repositories to be smarter by notifying the CI server when something has been committed.  To date, Unfuddle only supports there Callback URL functionality and not any sort of Git hooks.

Based on the specification from Unfuddle, they are doing a POST to the URL specified with data regarding the commit.  Looking at the TeamCity specification, the URL they provide is wanting a GET.  Since I’m not sure if TeamCity as actually looking for a GET or only accepting a GET, I decided to investigate further.

After opening up one of my favorite debugging tools, Fiddler, I looked at the POST request from clicking the “Run” button on the TeamCity interface.  Upon doing so, I built the URL:

http://uname:password@ciserver.mydomain.com/httpAuth/runCustomBuild.html?includedChanges=LATEST&buildComment=Callback%20from%20Unfuddle&buildTypeId=bt4

This tells TeamCity to run a build from the latest changes and add the comment: “Callback from Unfuddle”.  The build type parameter is simply a reference to the build specification in TeamCity.

So, to test this theory out, I installed a Firefox add-on called HTTP Resource Test to test out actually a POST to this URL.  This worked out perfectly.  By build was triggered without a hitch including my build comments.

All that is left is to put my URL in the Unfuddle repository settings and away we go.  I then made some commits on the local Git repository and then pushed those changes to Unfuddle.  Waiting with bated breath, nothing happened.

Ok, what went wrong.  I contacted Unfuddle support and it turns out that they aren’t passing QueryString parameters but the Callback URL was firing.  So, back to square one.

At this point, I think my only option is to setup a proxy server to work with both parties. I looked at the Git hooks and there doesn’t seem to be a post-push (out of the box, there maybe an add-on/extension) that I can filter by pushing to Unfuddle to then run a script locally to do the GET or POST to the build server. Post-receive won’t work because again, Unfuddle can’t run hooks.

Anyway, when I get more time, I may investigate further and blog again about my solution.

October 13, 2010 Posted by | Programming | , , , , | 2 Comments

ASP.NET Security Update Owes Me Dinner

Let me preface this with saying PEBKAC.  I was the problem not ASP.NET. In fact, ASP.NET did exactly what it was suppose to do.  So with that, here is my story…

I have spent the entire day trying to figure out why I’m no longer able to login to my website that I’m currently developing.  I was able to login before (last week), what has changed.  Time to follow the white rabbit down the rabbit hole.

Context Please

My web application uses a common login site to authenticate me.  In this case, I was using lets say login.mydomain.com.  login.mywebsite.com is setup with a specified machine key. That way, I can login there and get a session cookie that can be consumed by thecoolapp.mydomain.com.  This affectively provides me with a Single
Sign On Service (SSO).

The Hole

I spent all morning trying to figure out why my version of ‘thecoolapp’ was unable to accept the cookie from login.mydomain.com.  I re-cloned my code down to a temp folder and updated IIS to point to the new test site so see if there was an issue with the web application or possibly the application pool.  No go.  A team member did the exact same thing and his worked fine.  At that point (and since we are using Git for our source control – hashed commits), I knew that it couldn’t be the source code since we were running the exact same code.

This can only mean one thing… its my machine.  What the heck have I done (installed, setting change, etc.) since last week that may have caused this problem.

So, I uninstalled IIS and reinstalled thinking it would reset something.  It didn’t.  When that didn’t work I decided to take IIS out of the picture and try to run it using Cassini (Visual Studio Web Development Server).  Oh wait, that can’t work because of my fancy login cookie stuff.  See, login.mydomain.com sets a wildcard domain wide cookie and Cassini will only work with localhost.  However, I added localhost.mydomain.com to my hosts file to point to 127.0.0.1.  Let’s try this.  Crap, it behaves just like IIS and DOESN’T WORK!

Lightbulb

This led me to one conclusion.  Windows Updates.  I knew about the ASP.NET security update that Scott Guthrie mentioned herehere, and here. Maybe one more time would have done the trick ;).
Looking at the history, Friday evening, I applied KB2416471 and KB2416472.  And then it hit me!  Duh! What an ID10T!  Of course it won’t work.  login.mydomain.com doesn’t have the patch! (don’t worry, this isn’t a production URL and is currently in an intranet zone).

So, uninstall the patches, reboot and violas, I can login again.  So, if you’re reading this, I hope you didn’t make the same mistakes that I did and remember what you installed on a Friday night.

October 12, 2010 Posted by | Programming | , | Leave a comment

Setting up TeamCity CI Server with Git and Unfuddle

I’ve been struggling trying to figure out exactly what settings I needed to use when setting up my TeamCity project using Git as the VCS/repository type and Unfuddle as the repository source.

I have my TeamCity server and build agent on one machine right now.  Unfuddle currently requires me to use SSH to work with so I need to setup a key and register it with an Unfuddle account.

For anyone else having troubles with this, here are the settings that I used to get it working.

Obviously, select Git as the VCS Type

image

The General Settings section is basically the default settings from TeamCity.  One thing to note is using the git@ forcing SSH.  Be sure to have Git installed on your TeamCity server (I installed msysgit) and have port 22 outbound opened up to connect to Unfuddle.

image

The Advanced Setting is what took me a while to figure out.  The things to note:

  • I needed to use a Private Key for Authentication to connect to Unfuddle.
  • Leave the user name blank
  • Create a key and put it on the server.  In my experience, you have to put a passphrase in the key.  If you leave it blank, it didn’t seem to work.

image

I setup polling every 5 minutes.  I’m not sure what the polling policy for Unfuddle is so you may want to change as needed.

image

That’s it.  Click the Test Connect button and everything should work as expected.

October 11, 2010 Posted by | Programming | , , , , | 1 Comment

Calling Stop-Process on a remote machine using Powershell

I decided that I wanted to update the application that I’ve been working on at work so that I can deploy it to the server much easier than copy and pasting using Explorer.  Since I have not done anything using Powershell, I thought that this would be a good opportunity.  Upon finishing my deploy scripts, I found that some files I was attempting to override or delete were locked.  After further investigation, I found that those files were being currently used.  I also found that because of th numerous times debugging, running, etc, there were hundreds of instances running (this is normal because of the nature of the application, this happens each time it runs).  So, since there is no way (that I know of) to kill lots of processes using some UI tool (Sys Internals Proc Exp or Windows Task Manager) I decided to use Powershell to do the dirty work.

So, I terminal serviced onto the server, fired up Powershell and ran the command

Stop-Process -processname "PName"

Worked great. Killed all my processes running. But I started thinking, that it would be great if my deploy script did this automatically (or at least prompted me first). Since I was running my deploy script locally but wanted to run the command remotely, I started looking into how to do this using Powershell being the newb that I am.

My first attempt was to use:

Enter-PSSession machineName

And

Exit-PSSession

I fired up the Powershell console locally and did:

Enter-PSSession machineName
Stop-Process -processname "PName"
Exit-PSSession

It worked! So, I copied this code into my .ps1 file and attempted to run it again. However, each time I ran the script, it was not finding my process when after further investigation, the process was clearly running on the remote machine. That makes no sense! How can I run it via console and then run it in a script and get 2 different results. So, I started doing some digging and came across a TechNet Forum Thread which clearly explained that using Enter-PSSession and Exit-PSSession are only for interactive use.

I knew that some Powershell commands had the -computerName parameter like Get-Process. Looking at the documentation for the Stop-Process command, there was no option to specify -computerName so I didn’t know how to proceed.

Since I can’t let things be, even though my deploy scripts work fine without doing this, I knew I had to find a way to get this to work. So after doing some more digging, I found Invoke-Command with the -computerName parameter.

To run any Powershell command on a remote machine simply do the following as an example:

Invoke-Command -computername machineName {Stop-Process -processname "ProcessName"}

Happy Coding!

February 17, 2010 Posted by | Programming | , , | 1 Comment

   

Follow

Get every new post delivered to your Inbox.