So what exactly happens during each of the three phases of CD?
What is Continuous Build?
Continuous Build (CB) has been a standard practice in software development for many years.
The aims of Continuous Build
Every developer is familiar with routine builds, and almost all integrated development environments (IDEs) are designed to support building code directly. But not every developer builds and rebuilds their code with the frequency (one might say fervour) that is the norm in Agile teams.
If you are not already used to frequent builds, this is as good a starting place as any for Continuous Delivery. In fact without this one step, there will be no beginning at all.
The purpose of Continuous Build (CB) is to ensure that:
- Code is always good enough to at least compile.
- Build-related defects are detected early, ensuring that they are usually relatively small, simple and easily fixed.
- The code on which subsequent development is based is healthy and clean.
- The distance between code in development and potentially shippable working software is kept to a minimum.
CB is now so much the norm within Agile that it is often applied after every single change of any significance. The process is quick, automated and informative.
From the point of view of creating a true CD solution, however, the key issue is not whether individual developers are running a Continuous Build system, but whether the entire team is doing so, in an harmonised and integrated manner.
Even at the start, all team members should builds their code at last daily, so that the overnight build will confirm that their code hangs together as expected – or if it doesn’t, it makes sure they have something to do first thing the next day.
Once that routine has been established, modern development tools mean that it is probably not possible to build too frequently. Executing a build after every change is perfectly normal. So is building a parallel suite of automated tests, including not only adopting test-driven development but also running a basic external test suite too. By progressively assembling such a suite, a longer term objective is also served: the automation of testing as a whole.
One purpose of automating the build is to accelerate it – especially the build that commits the code back to the mainline – and so to get rapid feedback on how well your most recent code works. The goal should be in to complete the build in minutes (XP aims at getting under ten), but you are only likely to achieve this early on if you are also careful to restructure your codebase to eliminate dependencies that hamper build speed. You should also simplify and perhaps reduce your tests to a minimal subset, replacing slow external resources such as databases with stubs, and other rationalisations.
But every such change also presents a risk that the build will not in fact be valid and that others’ code will be infected by your mistakes, so it should only be used when speed is more important than absolute quality. But you can also mitigate such risks by running more exhaustive and realistic but also slow tests immediately after the code has been committed, just in case a defect or two has crept in after all. But don’t expect the full set of tests to be finished in minutes, and don’t imagine that you don’t have to fix any bugs they throw up exactly as urgently as any other.
Of course, this assumes that a range of tools and repositories are available, including automated code control, unit testing and version control, but such a level of support is relatively easily achieved, so it doesn’t present a significant obstacle to the adoption of CD as a whole.
The basic Continuous Build process
This is what the basic CB process looks like:
- Clone a working ‘branch’ (or ‘fork’) copy from the latest ‘mainline’ source.
- Write or update the code as necessary, including both application code and (ideally) automated tests (in Agile, using test-driven development techniques).
- Refresh the local copy of the mainline to obtain the latest versions of libraries, etc.
- Compile and link.
- Verify the code:
What is Continuous Integration?
A successful build tells you that your code works. It does not, however, tell you whether it will still work once it has been combined with the code your fellow developers are building, or it is interfaced with the required databases, files, APIs, etc., or it is simply sitting on the right server. So Continuous Build should always be followed Continuous Integration.
The aims of Continuous Integration
Although it had been circulating for some time, the term ‘continuous integration’ first emerged in the 1990s as one of the core practices of Extreme Programming (XP). Essentially, Continuous Integration (CI) aims at:
- Ensuring that the team’s work is regularly and actively maintained as a unified whole, by frequently integrating local developments with the source code master (the ‘mainline’).
- Accelerating, simplifying and de-risking development.
- Ensuring that the team can always quickly deploy a working, release-quality version of the code into production.
Conversely, failing to integrate continuously has a clear price:
- The less frequently integration is done the greater the gap between the developer’s ‘branch’ copy of the code and the mainline grows.
- So the greater the errors and wasted effort that integration will eventually expose and the rework that will be needed.
- Delaying integration makes it harder to predict:
- How long the ‘final’ integration will take.
- How long development itself will be delayed.
- What the eventual outcome – success or failure – will be.
Eventually integration and the resulting rework – often accompanied by substantial redesigns and even architectural changes to compensate for local decisions, plus major increases in test budgets, and so on – will start to take longer than the original development. This state is correctly referred to as ‘merge hell’.
Continuous Integration and Agile
Like Continuous Build, Continuous Integration is increasingly widely used in software development as a whole. However, the significance of CI is different for Agile than for traditional approaches.
In Agile, CI is the standard approach, and frequent integration of code is a basic method of ensuring both that quality is the team’s first priority and that the team as a whole is kept aligned. So CI has an profound impact on team organisation and methods:
- If any single developer fails to integrate their code at regular intervals, the risk grows that their team mates are developing their code based on false assumptions about what ‘works’.
- Likewise, if a team member spots an issue in the integration but does not correct it, it quickly becomes a problem for everyone.
- So a strongly collaborative approach is absolutely fundamental to Continuous Integration.
So, CI means more than simply agreeing to integrate with other team members who are working on related items or code. Such an approach would only be worthwhile if it was possible to predict exactly where potential conflicts would appear – a situation that is practically never real. Instead:
- Everyone should integrate.
- Everyone should integrate continuously.
- Everyone should integrate continuously with the mainline.
Typically this means every developer needs to check their code in at least once a day.
In a way, CI is like continuously testing software not by setting up formal tests – or, TDD notwithstanding, any tests at all – but by simply throwing it at the other code and other system components with which it needs to work and seeing if it still works. It’s a simple – and, if everyone else is following the same process, an inherently valid – test!
The basic Continuous Integration process
From the developer’s point of view, the manual element of Continuous Integration is simple. Just commit their successfully built and tested code on the integration server. Behind the scenes – and this is the main point of CI – the integration server then carries out a number of critical tasks automatically:
- Monitoring the repository for changes.
- Application code, supporting files and databases, support services, third-party components, etc.
- Creating a new build of the system as a whole, typically in response to every commit.
- Running unit and integration tests.
- Releases deployable artefacts for testing
- Version control.
- Notifying the team of successful – and failed – builds.
Note that and effective CI process assumes some basic conditions:
- All code branches comes from a single repository of source code.
- All commits take place on a shared integration server.
- All defects are dealt with as quickly as possible.
What is Continuous Deployment?
Continuous Deployment, the final phase of Continuous Delivery, takes the standard Agile aim of ‘potentially shippable software’ and turns it into the real thing: the direct delivery of working applications to user machines.
Why is Continuous Deployment hard?
Although complicated, Continuous Build and Continuous Integration are both controlled by the development team. The problem with Continuous Deployment is that it requires the team to cooperate with a variety of other groups.
- Each of these groups may (and usually will) have completely different priorities, timescales, agendas, processes, and organisation.
- Many of these groups will know little of Agile, and may disapprove of it as a matter of principle.
- These groups may also feel threatened by the changes Continuous Deployment entails, which will generally reduce their authority and control over the delivery process as a whole.
- Even if they are eager to adopt Continuous Deployment, the effort required to implement Continuous Deployment and the disruption it could cause will be considerable.
- Although the changes needed to implement Continuous Build and Continuous Integration are basically technical in nature, Continuous Deployment demands a substantial change management programme, affecting organisation, process, systems and tools, governance, culture, politics, and a good deal else.
- So creating Continuous Deployment – and therefore CD as a whole – is a far more radical and difficult task than Continuous Build and Continuous Integration.
The traditional deployment process
The difference between the CB, CI and Continuous Deployment processes is made very clear by the conventional deployment process:
- Create a final, fully integrated release build.
- Prepare the various stakeholders for its acceptance:
- Train user and operational acceptance testers.
- Brief impacted managers and operations teams.
- Set up operational controls, help desks, etc.
- Agree a deployment plan with the relevant managers and authorities.
- Agree the deployment process.
- Set up governance over the deployment process.
- Define details deployment steps.
- Agree and schedule deployment resources.
- Agree warranty arrangements.
- Set up organisation-specific systems, controls, responsibilities, resources, approvals and arrangements.
- Brief and train members of the deployment team.
- Perform User Acceptance (UAT).
- Promote the integrated system to the UAT environment.
- Perform UAT.
- Evaluate, approve and authorise for use.
- Perform Operational Acceptance (OAT)
- Promote the integrated system to the OAT environment.
- Perform OAT.
- Evaluate, approve and authorise for deployment.
- Apply other controls:
- Internal compliance.
- Etc., etc.
- Deploy to production.
- Set up production controls, environments, etc.
- Execute the deployment plan.
- Obtain final approvals.
- Cut over.
- Validate the implementation.
All this takes place through the (usually manual or semi-manual) transfer of code, data and other materials between a succession of staging environments (UAT, OAT, pre-production, etc.), until finally it arrives on the users’ machines.
And even this ignores the number of times the whole process will go offline while higher-level and external reviews are conducted, signatures gathered, and so on. And, depending on your environment, maybe a good deal more.
Plainly, this makes deployment a long, complicated process demanding lengthy planning, multiple agreements, careful scheduling and intense scrutiny during execution. The risks, costs and opportunities for error and omission are equally great.
Assuming that Continuous Build and Integration are already in place, Continuous Deployment is rather different:
- Press large red button on release lead’s desktop.
Continuous Deployment and Agile
In many ways, Continuous Deployment is the final step in Agile. Agile’s ultimate goal is to deliver working software to its users, and Continuous Deployment does exactly that in exactly the way Agile would ideally do everything:
- At low risk.
Almost every element of the traditional deployment process makes this aim difficult to achieve. So moving to Continuous Deployment is an important step in implementing Agile.
The basic Continuous Deployment process
The tasks Continuous Deployment automates follow much the same sequence as traditional deployment, but the way they carry them out quite different, and resembles CB and CI closely:
- Controls are automated, based on the automatic generation of data and its automatic verification and approval.
- The actual movement of code, data and other components is handled by automated tools.
- It eliminates overt decision and approval points.
Not that Continuous Deployment ignores or overrides the authority and responsibilities of user, governance or operations. Rather:
- The control points in deployment are identified with stakeholders.
- Note that automation may make it possible or necessary to change some existing controls or make them redundant.
- The pass/fail criteria for each control point are defined.
- The information and decision-making that normally drive these controls are then built into the tools, including appropriate decision nodes and authorisation steps.
- Methods for final visibility and reporting are included.
In most respects Continuous Deployment is no different from Continuous Build or Continuous Integration. They were also made possible by better tools, but they could not be implemented fully (i.e., made truly continuous) until the process, controls, decision-making rules and so on were all agreed by the various development teams.
Continuous Deployment and Devops
Another way of understanding Continuous Deployment is in terms of ‘devops’. Devops is a rapidly maturing approach to redefining and re-integrating all the disciplines that traditionally span the boundaries between development and operations. This, of course, is a large part of what Continuous Deployment is trying to achieve, so a high degree of synergy between these two strategies should be expected.