NuGet for the Enterprise: NuGet in a Continuous Integration Automated Build System

I had the pleasure of speaking at TechEd 2011 North America last week in Atlanta. You can see ALL the videos of all the sessions on Channel 9. As an aside, you might notice that they are in the process of organizing video archives of ALL Microsoft developer events at http://channel9.msdn.com/Events. You can even see PDC 1999 if you like or see sessions by Speaker at http://channel9.msdn.com/Events/Speakers. Here are all my talks with a horrible headshot that I plan on asking Duncan to swap out ASAP.

My favorite talk was NuGet: Microsoft .NET Package Management for the Enterprise. I talked about NuGet, like I did in The Netherlands a few weeks ago, except the TechEd talk was focused much more on how NuGet fits into the software development lifecycle in a diverse Enterprise (or big boring company, if you prefer) environment.

Here's the video downloads, or you can click the slide at the right.

• Just the Slides (useless)
• MP4 (iPod, Zune HD)
• Mid Quality WMV (Lo-band, Mobile)
• High Quality WMV (PC, XBox, MCE)

At my last company, we used Subversion for source control and CruiseControl for Continuous Integration (CI). I thought it'd be nice to setup a similar system using the latest free (and mostly free) tools. Note, you can do all this with TFS as well for both Source and Build. I'll do a post on that later. For now, I give you:

Setting up NuGet to build using Mercurial for Source Control and JetBrains TeamCity for Continuous Integration while pushing to a local Orchard NuGet Gallery Server

Oh yes, that's a long H3 right there but it's descriptive, right? Here's the general idea.

This of course, is not unique to NuGet, as NuGet is just a build artifact. At my last company we had several things that popped out of the build. Not just the DLLs, but also a ZIP file, MSI installer and even a complete configured and prepped Virtual Machine for sales people to pick up and give demos with our latest bits. You can setup your Continuous Integration system to be as awesome or as simple as you like. You should be thinking about which libraries and parts will create NuGet packages.

Another thing to think about is daily (or every build) packages vs. stable or release packages. There are some discussions on the NuGet site around a "-beta" switch and baked-in support for understanding stable vs. volatile builds. For now, consider two locations for your builds, one for each build and one for "blessed." For some, this might mean a folder for dailies and only blessed go to a server.

Here's the demo I did. You can change what you like and swap out for your favorite tools. I'll point out some gotchas and issues that hit me and might hit you. It's not perfect, but we're moving in the right direction.

Step 0 - Prep Stuff

Some of these steps are "make sure x is setup" type steps and can happen whenever, so don't take the ordering of the steps as totally crucial. Here's what I used and installed.

• Get NuGet for VS if you like, but at least the NuGet.exe command line and put it in your PATH.
• Get whatever SCM (Source Control Management) system you want. I'm using Mercurial and TortoiseHg because it feels like Subversion which I'm used to.
• Install the TeamCity Continuous Integration Server. They have a free version for small teams.
• You can use a network share or local folder for a "poor man's NuGet Server" or you can install a full-on Orchard-based NuGet Gallery
  • The full (draft) instructions for setting up an Orcgard NuGet Gallery are up on docs.nuget.org. We'll be updating them with graphics and more explicit detail soon. If you get stuck here, don't worry, just use a local folder, or even a basic NuGet.Server installation like the one Phil shows you how to create.

Step 1 - Make sure your project builds and you can make a NuGet package (nupkg)

I'm doing all this on one laptop, but you might have things spread out at your company. Do what makes you happy.

I made a trivial little class library called TechEdLibrary and confirmed it builds with MSBuild like this:

C:\dev\techedlibrary\TechEdLibrary>msbuild TechEdLibrary.csproj
Microsoft (R) Build Engine Version 4.0.30319.1
[Microsoft .NET Framework, Version 4.0.30319.225]
Copyright (C) Microsoft Corporation 2007. All rights reserved.

Build started 5/24/2011 4:52:54 PM.
..snip...
Done Building Project "C:\dev\techedlibrary\TechEdLibrary\TechEdLibrary.csproj"

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:00.13

The most important part is to make sure that your AssemblyInfo.cs is actually filed out and not just the defaults. This is because we'll want to update the NuGet's specification file using the values from the DLL and project. Basically we want the metadata of the project to drive the NuGet package (or we have to update it all manually.)

I need a spec file. I can do this a few ways. I can do it manually with just "nuget spec," I can build it off the resulting DLL with "nuget spec -assemblyPath bin\debug\techedlibrary.dll" or off the csproject with "nuget spec techedlibrary.csproj."

If I create it off the csproj, the NuGet spec file will be created with some $tokens$ that will be filled out at packaging time:

C:\dev\techedlibrary\TechEdLibrary>nuget spec TechEdLibrary.csproj
Created 'TechEdLibrary.nuspec' successfully.

In the talk at TechEd I mistakenly build it off the DLL and ended up hard-coding the versions. In a continuous integration system you'll want to update the version in your spec as the build versions. You can either do it with tokens like I will in this post, or you can pass in the version (often from an environment variable) into the command line like "NuGet Pack -version 1.0.1.0."

You can also  use the UpdateVersion.exe that Matt Griffith and I updated SO many years ago to change your AssemblyInfo.cs, then the NuGet package can pick it up. Again, there is no right answer, the point is to have a strategy. What drives the version and how does NuGet find out about it?

My TechEdLibrary.nuspec I just made looks like this:

<?xml version="1.0"?>
<package xmlns="http://schemas.microsoft.com/packaging/2010/07/nuspec.xsd">
  <metadata>
    <id>$id$</id>
    <version>$version$</version>
    <authors>$author$</authors>
    <owners>$author$</owners>
    <iconUrl>http://www.hanselman.com/images/nugeticon.png
</iconUrl>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <description>$description$</description>
  </metadata>
</package>

And my AssemblyInfo.cs (abridged) is like this. The attributes from the DLL will get copied into the $tokens$ and packaged.

[assembly: AssemblyTitle("TechEdLibrary")]
[assembly: AssemblyDescription("This is my cool library")]
[assembly: AssemblyCompany("Scott Hanselman")]
[assembly: AssemblyProduct("TechEdLibrary")]
[assembly: AssemblyVersion("0.9.*")] //The * means I'll get an automatic version bump
//and more...

Now I can pack it up with NuGet Pack TechEdLibrary.csproj. Note the command line output find the spec and does the token/metadata replacement?

C:\dev\techedlibrary\TechEdLibrary>nuget pack TechEdLibrary.csproj
Attempting to build package from 'TechEdLibrary.csproj'.
Building project for target framework '.NETFramework,Version=v4.0'.
Packing files from 'C:\dev\techedlibrary\TechEdLibrary\bin\Release'.
Using 'TechEdLibrary.nuspec' for metadata.
Successfully created package 'C:\dev\techedlibrary\TechEdLibrary\TechEdLibrary.0.9.4161.28882.nupkg'.

Now I can just open the .nupkg into the NuGet Package Explorer and see the version, author, ID (from Title) description and version are all there, brought in from the attributes and combined with my NuSpec. I can edit the NuSpec to taste as long as I maintain the $tokens$ I want.

Now if I build the library again I'll get a new version from the .NET build system and that will cause a new NuPkg to be built with a new version. I make a change to my application's source, build again, then pack again. Note the results in my directory after I make a small change, build and pack.

C:\dev\techedlibrary\TechEdLibrary>msbuild TechEdLibrary.csproj 
Microsoft (R) Build Engine Version 4.0.30319.1
Build started 5/24/2011 5:07:58 PM.
blah build blah 
Done Building Project "C:\dev\techedlibrary\TechEdLibrary\TechEdLibrary.csproj" (rebuild target(s))

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:00.32

C:\dev\techedlibrary\TechEdLibrary>nuget pack TechEdLibrary.csproj
Attempting to build package from 'TechEdLibrary.csproj'.
Building project for target framework '.NETFramework,Version=v4.0'.
Packing files from 'C:\dev\techedlibrary\TechEdLibrary\bin\Release'.
Using 'TechEdLibrary.nuspec' for metadata.
Successfully created package 'C:\dev\techedlibrary\TechEdLibrary\TechEdLibrary.0.9.4161.29041.nupkg'
.

C:\dev\techedlibrary\TechEdLibrary>dir *.nupkg

05/24/2011  05:07 PM             4,770 TechEdLibrary.0.9.4161.28882.nupkg
05/24/2011  05:08 PM             4,763 TechEdLibrary.0.9.4161.29041.nupkg
               2 File(s)          9,533 bytes

Without settings really anything up but the versioning plan, spec file and checking if packing works, I've got a little system here and hopefully you can see how it'll work.

WEIRDNESS NOTE: We do have a double-build going on. NuGet.exe, for some weird reason, is running MSBuild again for us. That's lame, and a known bug. Surprisingly in a large number of CI systems in both the .NET and Java worlds "double builds" are common. Weaksauce, but common. Still, no excuse. That'll be fixed in NuGet.exe soon.

Step 2 - Initial Source Check-in

I like using BitBucket for small private projects and CodePlex for public Open Source projects. Both support Hg (the elemental symbol for "Mercury" as in Mercurial) and CodePlex supports TFS for both Source and Work Items. Since my demo is private and BitBucket supports unlimited private projects it was a good fit.

I cloned my project URL from BitBucket into a folder, then added, commited and pushed my bits to the BitBucket Site:

C:\dev\something>hg clone https://shanselman@bitbucket.org/shanselman/techedlibrary
...move my files in...
C:\dev\something>hg add
...yada yada yada...
C:\dev\something>hg commit
...yada yada yada...
C:\dev\something>hg push
...yada yada yada...

It's useful then to make sure I can get my source code into another folder and still build it. It's common to miss a file or two. Since the CI Server will be getting the code into a temporary folder you really need to make sure the source will build as it is, retrieved fresh from your SCM.

Step 3 - Setup your Build Server

I'm running TeamCity locally on http://localhost:8111/ with a "build agent" (there can be many) on the same machine. Visit Administration and make a new project, then a new Build Configuration (like Debug or Release).

You'll need to make a VCS root (Version Control System - there are like 60 different acronyms for version control systems. If you see a TLA (Three Letter Acronym) out there that you don't recognize, it's probably something that means "Source Control.") in TeamCity. Note that I selected Mercurial and set the HG Command Path with the full path to HG.EXE as the CI will call that to check out the code from BitBucket. Note also that I put the path without my name and password for the "pull changes from" as I put the name and password lower down.

Back in Administration, select Edit under your project. Note the nice lists of steps on the right.

We want two steps, one for the MSBuild and one "custom command line" for the NuGet package step. The first step is easy, we call MSBuild on our TechEdLibrary.csproj.

The second step is a temporary hack. It's temporary because JetBrains TeamCity is building NuGet support in directly (screenshots of their internal build below!)

The custom script is basically a BATCH file that looks like this:

del *.nupkg 

NuGet pack techedlibrary.csproj

forfiles /m *.nupkg /c "cmd /c NuGet.exe push @FILE yourapikey -source http://localhost:81/

There's a few interesting things going on here.

First, I delete all the nupkg files in the CI folder as it's all temporary and we don't want to accidentally push old stuff.

Second, I pack up the NuGet package like we saw before.

Finally, I don't know the name of the newly created *.nupkg file, so I cheat by making a DOS BATCH "for loop" that has only one item in it, the newly created .nupkg file! Then I call cmd.exe execute NuGet.exe pack with that new file as a parameter. Make sure you have the trailing backslash.

NOTE: You can also save the API Key in local storage with NuGet SetApiKey yourapikey -source http://localhost:81 and it'll be saved on a per-source basis.

Because I have an Orchard NuGet Gallery running locally, I have an API key for that server. I'm running my gallery server (it holds the packages and serves the OData Feed) locally on port 81 and the Orchard Site itself is on port 80.

I set the build to check Source Control every 60 seconds, if a change is detected, the source is retrieved, build, then NuGet packed, and published to my local server (or the public one).

TeamCity is something I like to have running on extra hardware I've got lying around. You can have up to 20 projects with their free version, so when combined with BitBucket and CodePlex where I keep my projects I can setup my own Continuous Integration System in just a few hours and have better confidence in my code. You can even use Amazon EC2 images to run your builds or just be a build agent.

Sneak Peak - TeamCity with NuGet Integration

I'm continuing to talk to companies and software vendors who are jazzed about NuGet. If you are one, watch my talk for ideas on what you can do as a commercial software entity and get into the discussion on CodePlex.

Here's some mockups of what JetBrains is planning for TeamCity. These are just ideas, and they aren't available, so don't stress or pressure them. ;)

This is a mockup of NuGet as a possible Build Step within TeamCity.

Here's a mockup of a Build Feature where TeamCity downloads specific packages that will be needed by the build.

Here's a screenshot of some early work that Martin Woodward from the TFS team is doing to make sure NuGet and Team Foundation Server work well together in a Continuous Integration environment. Feel free to contact Martin via his blog and continue the discussion on http://nuget.codeplex.com.

Hope this helps you integrate NuGet into your company, however you like it.

Related Links

• Session Video: NuGet In Depth: Empowering Open Source on the .NET Platform
• Session Video: Coding for Fun with Scott Hanselman - Silliness!
• Session Video: MVC 3 – 101 - Good beginner video!

---

© 2011 Scott Hanselman. All rights reserved.