Continuous Integration( Part three) Things That can be done from the Build Server Side

Note : If you are interested in installing jenking build server for .Net applications I found an interesting post which has a good step by step explanation. 

Before going through this blog post I would recommend you to go through following blog post which gives you knowledge which is useful to understand the things mention in this post.

CI - Part One

CI-Part Two

As I mentioned in above CI-Part Two we can create a IIS deploy-able packages with different configurations on the built server machine.

 Requirement : 

 "Automate the deployment process on IIS location for every successful  build.Moreover I need to automate multiple deployment with different configurations."

If I breakdown above requirement further, it is obvious that my first task is to create multiple packages with different configurations if and only if the build process is successful. In this case I may need some support of build server side to trigger me after a successful build. Once build server trigger me I have to think of creating packages with different configs. First question arise is as follows.

Me : Hey Jenkings, Do you have a plugin which you can trigger me after a successful build ? 

Jenking : Yes. I do have many which you can get your job done.

Me : Thank god. Then I will be surviving..... :) 

There is a good plugin to get this job done which is called Jenkins post built task plugin. By using above file I can execute any command line operation by checking the status of the build server build report. So what can I do is searching through the build report and find the build status. And then if it successful then execute a batch file or simple command line operation for the deployment. By ticking on the post built task you can get the following post build task. (In this example it runs the script if the build is failed)

So now we have a way to trigger a batch command if the build is successful.Then the next task is to find a way to do deployment automation.First I need to think about a way of creating packages with different configurations.I found following possible approaches of doing the packaging.

• Assign Multiple jobs to jenkins server for different configurations :

In project configuration section you may find the build task. Here you can explicitly tell jenkins build server the configuration you want to build and package.(Simple it accepts MSBuild external parameters)

In this comamnd line argument section you can pass /t:package /p:buildsever and then it package web project which has the build server configuration. Similarly what you can do is assign new job to the jenkings server and in that job in the build section pass other configuration which you want to create the package.(eg: Live Server Configurations)

• Build the package with multiple configuration by using a single Jenkings job:

Only change is in the command line argument section you can add two configuration which separated as follows.

              /t:package /p:buildsever;liveServer

But in order to get this job done you need some changes in the sln/csproj file which you are referring in the build server. This post will helps you to get this project file configuration done. 

     
At this point we are almost done with the packaging process.Next remaining thing to be done is deployment process of multiple package.Here I would suggest following approaches to to do it.

• Use XCOPY command to copy the newly build package in to IIS location :

In this approach what you need to do is execute XCOPY command in Script section in post build plugin section.
 cd to project location in build server          

 XCOPY Views \\ES-WEBSRV01\Library\Views /e /y /i

 XCOPY packages.config \\ES-WEBSRV01\Library /y  

Similarly you need to copy all required folders in to IIS location.

\\ES-WEBSRV01\Library is the place where we deploy our IIS application.If your IIS location is on a another machine rather than the build server machine you need to share the location with the build server machine as above.But here we have a tricky problem. As i mentioned in my previous blog post there is a issue with the web.config file.So what we need to do is go to the project\obj\release\TransformWebConfig\transformed in the build server and get the transformed web.config file and then XCOPY it in to IIS location.(here release means you configuration)

Note : You need to do this for all packages built by the build server

• Deploy using msdeploy command line tool ;

You can find your installed msdeply.exe in C:\Program Files\IIS\Microsoft Web Deploy

Once build server created the web package you can see a windows batch file named projectName.deploy  in Project\obj\release\Package. In the post built plugin script section what you need to do is run this batch file and it does the deployment.For that you need to go to the file where you can find msdeploy.exe and then run that batch file as follows.

C:\Program Files\IIS\Microsoft Web                               Deploy>H:project\obj\release\Package\MVC-Client.deploy.cmd /T 

It will deploy you web application in to the IIS location with the correct configurations which you want to deploy.

Note : You need to do this for all packages built by the build server

Continuous Integration( Part Two) :Web Config Transformation

Note : In this post I will use Microsoft based technologies. But the main concept describes in a general manner.

In my previous post I mentioned that a build server should run the tests with its configurations(eg: data base connection strings) and if the test are successful then it should deploy working packages in to many web container location according to your requirement.For an example it should deploy a working package in to productions server as well as in to the live server with their specific configurations.(both application are pointing their own data bases)

production server => production database with tests data for QA guys

Live Server => live data base with the actual data 

In order to do this, build server has to create several packages with different configurations and deploy them in to required locations.There are many packaging tools which facilitate configuration transformation functionality.In this post I will use MSBuild command line tool to create IIS deployable packages.Before Moving on we need to know a bit detail knowledge about web config transformation.

If you have a close look in to your web.config file which created by VS2010 you will notice that there are two config files which named as web.Debug.config and web.release.config. Those two files default files which offer by VS2010.If you want your own you can add since VS2010 provides you an extensible configuration setting environment.

                          Build =>Configuration Manager  => Add your own configuration 

Lets assume we have created configuration for our production server which I mentioned above.Till you cant see it your latest configuration setting appears on web.config file. In order to set it you should do following step.

                         right click the original web.config file and click the context menu command “Add Config Transforms” 

In the Web.BuildServer.config file you can write specific settings which is specific to the build Server settings. Following is the way we do the we web.config transformation. 

You can see here we have used XML Transformation Engine which does the actual task when we package our project with build Server configurations.Followings are the some of functionality offer by transformation engine.

Other than the xdt:Transform tag you can see we have used a locator which refer the original web.config configurations. 

And following is the MSBuild command which we can transform web.config which has the build server configurations. 

                   /t:TransformWebConfig /p:Configuration=BuildServer" 

Also you can create your web deployment package with the build server settings by following MSBuild command.

           "ProjectName.csproj" /T:Package/P:Configuration=BuildServer;

It creates a ZIP file which has batch file which can be executed from MSDepoly command line tool.And also we can see transformed and original web.config files. Here we have a small problem.That is if we manually unzip this created zip file and then manually copy to the IIS location . then build server specific settings are not applying to it.Since in web.config file refers the transform elements from a external XML file which is used by the MSDeploy command and it is the task of MSDeploy tool to read the external XML property file and replace the matching tokens. So if you do the deployment manually I recommend you to copy the transformed web.config and replace the existing web.config file in the package.Then you can manually copy  package in to the IIS location and your app will work since now you have the build server configurations.

Ok.Now we should back to our main topic again which is Continuous Integration. If we can use above concept in the build server side (through a build server job) we can create a package in build server machine with its specific configurations and then deploy it. Also we can use same strategy for multiple deployment with many configurations.And my next post will describe how we can achieve this task. I hope to use jenkins build server and some of its plugins to do it. 

Continuous Integration( Part One) :Build server

why we need a build server ? 

Before moving in to the today's topic I would like to explains the typical scenario of the software development with out using continuous integration. Suppose that your project team is using a source controller where you save all source files in a shared location for the collaborative development process. How it works is simply a developer implements a feature of your product and then do the testing on developer's environment. If the testing  is successful he/she push the new changes in to the source controller. Following diagram explains it briefly.

Here the developer must do the modification such that the latest version is build on every other developers environments. Here what I am trying to notice is that it is not sufficient the latest version build successful in one machine.Lets say another developer get the latest update and it is not working in his machine.Reason is there may be some configuration which you forget to commit and it leads to errors in the latest version of the software in other's machines. Those are things that the software engineers had to spend much time to fix which reduce the productivity of your team.

Solution for this issue is to introduce a third party built environment which is isolated from developers' specific setting. It should be an environment which user its own configurations (eg : database configurations)

Simple you can configure your build server such that it runs in an isolated environment and runs all unit testing and UI testing(Selenium). If it fails also build server can configure such that it sends a mail to developers with the build error message.

Following are few of available build servers in today's software industry.
http://hudson-ci.org/
http://jenkins-ci.org/
In my next blog posts I will use jenkins server for explanations.

Followings are the basic tasks which we expect from the build server side to make a end to end Continuous Integration.

• Get the latest updates from source controller and run all unit tests/UI tests(Selenium)
• If the tests are successful then automate the deployment process. There can be multiple deployments take place.Eg: Live deployment,Production deployment
• Create a package which can deploy on your web container and save it with the version.Simple its like keeping working copies of software packages for each working changes.

Part Two