Introduction to Continuous Integration Testing

What is Continuous Integration Testing?

Continuous Integration (CI) is an approach within software development in which the developer pushes code into a repository, such as Git or SVN, several times daily during the development phase. Each push is followed by a build, which will then be automated using a set of tools and will also most probably have a set of automated tests run against the build. This will ensure new changes are well integrated into the project code base so any problems can be found very early.

How Continuous Integration is Done?

There are tools available (both licensed and open source) for you to set up CI with your own project. Some commonly used tools are Visual Studio Team Services, TeamCity, Jenkins, and CruiseControl. Using CI tools, simultaneous builds can be run on multiple platforms. Some licensed tools come with the option of ‘on the fly’ test result reporting plus state of the art test reordering. These tools also support multiple technologies and platforms, such as Java, .NET, PHP and others, and support the creation of better quality code by finding duplicates within the code.

Advantage of Continuous Integration

Most CI tools act as both continuous integration server and deployment server. Once initiated, they can build your solution, run unit as well as functional tests, and automatically deploy your application to a server. Some of the benefits of CI are:

1. They are a great time saver as the entire process is automated.
2. It ensures full integrity across code and build before deployment.
3. They are easy to use, even for those who are non-technical.
4. Faster builds are produced through few, easy steps.
5. Real time reporting of the build process.
6. Simple setup and configuration.

Continuous Integration concepts

A CI tool consists mainly of a build server and build agents. It’s a good idea to understand the basic concepts before embarking on continuous integration, using any CI tools. The points below describe a few concepts on continuous integration, and CI tools specifically.

Build Agents:
The build agent is responsible for executing the automated build process. It’s not essential that both build agent and build server are installed on the same OS (build server and build agent can work individually on separate machines). You can set up as many build agents as you want (or as your CI tool license permits), and each can run on separate platforms, on which you want to build your software project.

Build Server:
Don’t be mistaken into thinking that the servers actually run the automated builds. The server is for monitoring the build agents, which are connected with the server, provision the build queues to the build agents on the basis of its compatibility, and log and report build results. All information on the build results, VCS changes, build queue, agents, user accounts etc. will be saved in a db.

Build Project:
This is a working project that is set up on the CI tool for software that is under development, and on which build automation is planned. It is basically a set of configurations for your build.

Build Configuration:
Build configuration is a basic range of settings for the build process, such as build source path, build step and build trigger, which will be discussed below.

Build Source path:
This refers to the version control URL and its credentials, that will define how the tool should communicate to this version control system, so it can monitor code changes and fetch the source for your build.

Build step:
This represents a task in a build process. Build steps are carried out by the build runner, which provisions the integration with specific build tools (like Ant, Maven, MSBuild etc.) or testing frameworks (TestNG, NUnit etc.). Therefore, in each build you will be able to run several tasks (build steps) which will invoke testing tools (TestNG, for example), build tools and any specific commands such as batch or powershell, sequentially.

Build trigger:
This is a rule that invokes a build process. For example, you can set when you actually want to trigger the build process, such as whenever there is a commit to the code repo, or scheduled builds like EOD every day, or twice a week, for example.

Build Queue:
These are a set of build jobs that are waiting to be executed, one after the other.

Build Artefact:
These are the output files that are available for download, once the build process is complete. For example log files, JAR etc.

Build Runner:
Build runner is responsible for providing integration with specific build tools such as MSbuild (for .net projects), Maven (Java) etc.

Common CI functional flow

The below steps show how CI works in order to run an automated build process:

1. Build server will periodically initiate poll jobs to detect any changes in VCS (providing build trigger is set up specifically for that). Changes, if any, will be sent to the db.
2. Build trigger will pick up these changes in the db and will add a build job to the build queue.
3. Build server will then find a build agent that is ready, and will assign this build (which is next in queue) to this agent.
4. Build agent will execute the Build Step, which has already been set up. When the build is in process, the agent will report the build process to the server by sending logs, test reports etc. on the go, so that you can view and verify the build process in real time.
5. Once the build process is finished, the agent will send build artifacts over to a server where the software is to be deployed.

What tool to choose: Open source vs Licensed

Every tester and organisation will favour different tools depending on their requirements and the size of budget they have available. To help you decide, we’ve put together our list of the top continuous integration tools currently available.