Continuous Integration (CI) using TFS, Visual Studio and NuGet with Artifactory

Part 2 – “Is the sky cracking”?

In the first part of this blog, we covered the downloading of the sample projects, the installation of Artifactory PRO, the building of the MyLogger library, and its storage into Artifactory. In the previous blog we covered the ability of Artifactory to proxy build dependencies from a remote cloud repository and how to setup of the Artifactory.

This second part will cover the consumption of the MyLogger library by three projects found within one Visual Studio solution. This multi-example project builds the api, core, and service libraries using the latest mylogger.dll from the CI build as well as requiring other remote dependencies that will be cached by Artifactory. All build artifacts will be hosted in Artifactory. How to promote a build artifact from an Artifactory staging repository to a release repository will also be covered. The final part of this blog will show how one may connect the artifacts stored in Artifactory back to Visual Studio and the TFS database.

All source code needed should already be available from the Part 1 of this blog. Artifactory Pro setup with the appropriate repositories is assumed to be complete. Visual Studio/TFS knowledge is assumed.

Creating the Multi-Example Build Definition

Continuing, we create a new multi-example build definition in Visual Studio to build the multi-example solution.  Customizations and the relevant parts of this build definition are highlighted below. There is also a brief explanation of each customization below the image. The outcome of this build will create the three libraries, package up each into a NuGet package, and push these packages to the Artifactory server.

NuGetExampleMultiProjectsExamplemulti-example.sln is the solution that will build and store the api.dll, core.dll, and the services.dll.

p:DistribPackages=true — This is a custom MSBuild task created inside Nuget.targets file. The DistribPackages task is mean to “push” the Nuget packages from the build site to the local Artifacotry repository.

When NuGet Management is added to a solution in TFS it adds a NuGet.exe, a NuGet.config, and a NuGet.targets file to the Solution File. In this project we check in the NuGet.config and the Nuget.targets file, but not the NuGet.exe.  The NuGet.targets file is customized to contain the MSBuild targets needed to deploy to Artifactory.

ProcessTemplatesScriptsUpdateNuGetPackages.ps1 – This is a custom “pre-build” powershell script that is run prior to the compile. The main purpose of this script is to make sure that the latest package of MyLogger and other required binaries are included in the build.

$(BuildDefinitionName)_1.0.3$(Rev:.r) – This will allow the user to set the Major.Minor.Build number.

Build Multi-Example Project with MyLogger from Artifactory

The multi-example.sln project contains three projects – one for the api, the core, and the services. All three consume the mylogger.dll as well as other third party libraries. After the NuGet integration with the multi-example.sln file, we need to add the Newtonsoft.json, the MyLogger.dll, and the Ninject.dll to the installed packages. Although these are found on external repositories, they will all be cached locally in Artifactory and be accessed using the virtual ‘repo‘ repository instead of having to go outside.

Below is the NuGet Package Manager screen of installed packages for this solution and a screen shot the packages.config dialogue which can be updated for all or some of the projects within the solution.

During the build of the multi-example.sln, we run a UpdateNuGetPackages.ps1 script to make sure we have the latest packages needed for each of the three builds. There is a RestorePackages in the NuGet.targets file, but this only restores what is missing in the packages.config files of each project.

The packages.config is read at the beginning of the build and the version numbers are set at that time. If a new NuGet package has been released the MSBuild will not get the latest because the package.config has not been updated yet. For this reason, we had to use a small pre-build powershell script to update the packages.config files prior to the MSBuild loading these files.  This issue may be resolved in future releases of the build template.

After the multi-example project completes, all the artifacts from the three projects are packaged with NuGet and “pushed” into the nuget-staging-local repository in Artifactory. These newly stored artifacts can be pulled from Artifactory and deployed. Any promotion scheme for the MyLogger project artifacts, may also apply to these artifacts.

Promoting from Staging to Release

After a package has been deployed and tested it can then be approved for promotion to a “release” status. In Artifactory we use a release repository (e.g. nuget-release-local) to store NuGet packages that are ready for production release. Access control for each repository is maintained by changing group and user security settings for each repository.

The move from the local staging repository to the local release repository is a “cheap copy” and does not take up time or disk space – “it happens only in the database”, much like placing a tag on a group of files.

Once the files are in the release repository, other developers who need to work with the latest “released” package, can pull their dependencies from the nuget-release-local repository to ensure they are working with a library that has been fully tested and ready for production.

Below is a sample of the outcome of such a promotion action within Artifactory. The promotions can be made from the GUI or scripted. If desired the scripts can be connected with a test results or an approval signature workflow. By using promotions, one can in the prevent the DevOps group from deploying code from the staging CI builds repository (code that has not been approved by QA) instead of a release repository.

Connecting from Artifactory back to TFS

After the NuGet packages are stored inside of Artifactory, we can return to the build information stored in TFS by doing the following:

   Go to Artifactory -> Artifacts -> NuPkg Info –> click on MyLogger under the General tab

From here, you can review the original build, read the diagnostics logs, review the test that have been run or just review other build details inside of TFS database.

From Artifactory one can have a direct link back to the TFS project and the build that was used to actually pushed into the NuGet package into Artifactory. Below one can see more of the details for this build.

If you wish to see the contents in the TFS drop site for the build found in a NuGet package inside of Artifactory, there is a URL link from the TFS Web Client directly into the drop site.

In addition to the checksums for the NuGet packages stored inside of Artifactory, other properties like the Project URL and the Release Notes URL can also be found.

This integration did not make modifications to the build process templates, did not rely on custom activities, and did not require any custom csharp code. The Powershell script used is a lightweight script that may be deprecated or expanded as the user so desires.

What is the Outcome of all this effort?

Artifactory Pro currently, with little customization, allows one store artifacts (binaries) built using Visual Studio and TFS. Artifactory currently allows developers to host remote build dependencies locally with an on-premise server controlled by the developers. Libraries built for the consumption by other internal development groups can be hosted on the same Artifactory server. These artifacts can then be promoted through a CI pipeline and deployed from an Artifactory release-approved repository into production or to the customer.

For this example, we have modified the standard nuget.target file, added a simple powershell script, and customized the repositories in Artifactory. The same or similar results can be accomplished by using either MSBuild tasks, TFS Build Workflow customizations, powershell scripts, Visual Studio plug-ins or a combination of all of these.

The integration of all these tools will hopefully improve as the demand rises.