For something that always seems so fresh and maybe even still trying to find its feet, Agile has a remarkably long history.
Rapid Application Development (RAD) emerged from the work of Barry Boehm and James Martin in the 1980s, and Dynamic Systems Development Method (DSDM) in 1994. It’s 20 or so years since Scrum emerged into the light of day and 17 since the Agile Manifesto was published.
In all that time a lot has happened, a lot has been learned and a lot has changed. So why are Scrum and XP, for example, taught largely as they always have been? Why do the core components of Agile generally still look much as they did then? Conversely, if we were to create Agile and Scrum today, what would we produce?
Whatever the detail, the answer to that final question can be summarised briefly: Agile 2.0. That is, an update to Agile that serves the same purposes of the original Agile Manifesto and the other core components of Agile but which reflects and consolidates the experience of the last two decades.
This ‘Strategy for Agile 2.0’ outlines a first draft approach to creating Agile 2.0. In that respect it addresses only the first 2 of the key 3 issues facing Agile generally:
- What is stopping Agile from moving forward?
- How could we put Agile into a state where it can move forward?
- Where exactly should Agile move forward to?
What is the problem?
Where are we now?
Agile is now entering its 3rd decade. It has been immensely successful. Yet in many ways it has barely moved on. What has happened? Well…
- We’ve learned a lot, but not incorporated all this experience into Agile itself.
- The original flavours – Scrum, XP, Lean, etc. – remain separate.
- New flavours of Agile have emerged.
- New techniques have been developed.
- Agilists routinely face problems that could be solved by an expanded version of Agile.
- Top-to-bottom, end-to-end delivery process.
- The world has changed since the early days of Scrum and Agile.
- Its audience not longer consists of innovators & enthusiasts, and the kind of model we need has changed.
- It has been assimilated by corporate organisations with different needs (e.g., scaling).
- So Agile now has to do the hard work of growing up.
- Responding to the changing world.
- But without losing all the things that made Agile revolutionary.
- So – Agile 2.0.
In short, when it was introduced, Agile was revolutionary. But have its core definitions developed in parallel with the Agile revolution? Unfortunately not.
6 basic questions for Agile
So the crucial problem with Agile in its present form is not (despite interminable articles entitled ‘Agile is dead’) that it fails to perform as required. It is rather that the last two decades have seen significant changes on the situation in which Agile finds itself. This naturally raises 6 basic questions:
|1. How have the many original sources of Agile evolved since then?|
|2. What have we learned since Scrum was invented and the Agile Manifesto was written?|
|3. What problems that Agilists routinely have are not being solved by core descriptions of Agile (e.g., Scrum)?|
|4. Does Agile (as it is generally presented) map closely to its intended scope?|
|5. What guidance does Agile give for supporting technologies?|
|6. How have the conditions in which Agile operates changed since the early days of Agile?|
The state of Agile
So what does each of these summary In other words… conclusions mean?
Agile is fragmented
- Agile began life in an explosion of new ideas.
- Scrum, Agile Manifesto, Lean, XP, etc.
- Although users generally take elements from many sources, Agile itself remains a collection of independent packages.
- They overlap but are not integrated.
- With different groups of owners, authorities, enthusiasts, purists, etc.
- This ensures that Agile as a whole remains versatile & flexible.
- Though there’d be no problem embedding optional pathways in Agile itself.
- But it is an issue because:
- There are multiple Agile ‘industries’, which perpetuate this fragmentation.
- Individuals, teams & organisations have to master each source separately – multiple courses, multiple sources, etc.
- Teams & organisations then have to create their own lifecycle out of various scattered components – a skill they seldom possess.
- Inexperienced Agilists will adopt partial, mismatched or inappropriate practices & tools.
- Time & effort are wasted on a task that needn’t even exist.
- So any ‘real’ implementation of Agile:
- Should build on these sources,
- But should also strive to unify them.
Agile has moved on
- New disciplines, tools and techniques have arisen since Agile first emerged.
- They are widely used.
- But they’re not integrated into Agile.
- So like the individual flavours of Agile, they have to be discovered, mastered & adopted by each individual Agilist, team & organisation – over & over again!
- This creates real problems:
- For Agile teams & organisations.
- For mobile individuals.
- For teams & organisations looking for improvements.
- For consultants, trainers & coaches.
- For tool developers & vendors.
- In addition, there is extensive research on which Agile methods, tools & techniques:
- are really used, and
- really work.
- This research should be reviewed and its results incorporated.
Agile is too narrow
- Agile began at team level, but it can’t stay there.
- There are too many things that an Agile organisation/team does that Agile itself does not support well.
- Agile needs to explain how it aligns with all these areas.
- In other words, Agile needs to cover the true scope of delivery, which looks more like this:
- Agile itself does not need to do all these things, but it should provide hooks allowing Agile work to be integrated into the wider organisation.
- It’s not only important to migrate Agile to all these areas but also to integrate the groups and variants this migration creates into a unified strategic approach – in other words, to scale
- This creates a number of basic requirements:
- The scalability of Agile’s elements should be validated.
- Scaling methods should be built into the core model.
- However, because Agile needs to be equally usable to small organisations too, the core model should clearly differentiate scalability and non-scalability elements.
- Ideally, Agile supports delivery in general, not just IT.
- So it makes sense to teach/certify each element of Agile (e.g., Scrum, XP, Lean) separately.
- But business, managerial, technical & operational elements can’t be understood or used in isolation, so:
- Teaching them separately makes little sense.
- Isolating the elements deprives Agile’s users of a coherent, consistent & complete approach.
- Conversely, each such sector (HR, finance, IT, etc.) would benefit from its developing own model.
- We should incorporate the many lessons and innovations from related areas (notably Scrum, XP and Lean) directly into a single model.
- In fact we should aim to make these more ‘local’ variants redundant.
- This would not prevent Agile from incorporating variants.
- E.g., package implementation, support, various X-as-a-Service (XaaS) models, etc.
What about tools?
- Although Agile itself is agnostic about tools, they are extremely pervasive. Although not indispensable for Agile, they are:
- An inescapable reality of Agile environments.
- Realistic requirements for large-scale work programmes;
- Essential for integration with other corporate information flows and decision-making;
- Vital for many distributed teams.
- However, individual Agile tools do not present a consistent model of what the underlying Agile process is.
- This is only partly because a model like Agile is open to many different interpretations.
- Mostly it’s because it’s impossible to say with any clarity what Agile is.
- The scope of Agile is only weakly defined.
- Agile’s functions, tasks, tools and techniques are only weakly defined.
- So no single tool could support ‘Agile’ as a whole, even if it were designed perfectly.
- So tool developers and vendors are left to their own devices about what their tools should do.
- This is true even for core Agile functions and techniques.
- Likewise, selecting tools is problematic:
- When choosing Agile tools, it is hard to define what functionality is needed – or available.
- Moving between teams & organisations creates a steep learning curve for individuals.
- Changing tools creates a steep learning curve for organisations & teams.
- So Agile 2.0 should remain agnostic about the use of tools (and perhaps actively minimise it where tools create a risk of bureaucratising the workflow), but also address the needs of tool makers and users.
Agile’s audience has changed
Here’s a familiar model for adoption of changes.
The issue for Agile is, where are we now, and what does it signify for Agile?
- The old predominance of Agile enthusiasts – Innovators & Early Adopters – has passed.
- We are now in the era of the Early Majority.
- Or maybe even the Late Majority.
- This segment of Agile’s audience is not highly motivated by or curious about Agile as such.
- So Agile needs to provide a solution to their needs.
- This needs Agile to:
- Provide an end-to-end model.
- Drill down a level.
- That is, they need a unified Operating Model rather than a collection of frameworks.
Given the bad name it had in pre-Agile days, it’s worth recalling the Agile Manifesto’s views of methodology:
The Agile movement is not anti-methodology, in fact, many of us want to restore credibility to the word methodology.
Agile2.o is how we to that.
Frankly, Agile is getting middle-aged. The problem is that, like many middle-aged individuals, it refuses to let go of its uproarious youth.
It needs to be emphasised that the pragmatic attitude this segment is likely to adopt is not compatible with Agile itself.
- Agile cannot survive with a much higher degree of engagement, transparency and empowerment than many organisations currently encourage or permit.
- However, that is not to say that they need to become battle-hardened warriors for Agile itself. Their primary interest is in delivery, not in debating the approach to delivery, Agile or otherwise.
- So, although it is impossible to ‘do’ Agile without also ‘being’ Agile, that need not exclude the possibility that a great deal can be done to support this group, without falling back into old bureaucratic and hierarchical ‘command-and-control’ management strategies.
- Agile 2.0 should support these users, by coming much closer to a classic operating model solution than the high-level frameworks to which the major flavours, subsets and variants of Agile are currently limited.
So Agile now has to do the hard work of growing up.
Responding to the changing world, taking advantage of 3 decades of development, but without losing all the things that made Agile revolutionary, we need to rebuild all these elements into a single, integrated model that still refers back to the Agile Manifesto, but supersedes Agile’s existing limitations.
In other words, we need Agile 2.0.
The business case for Agile 2.0
Agile isn’t a public standard. It isn’t even a open source project – there are so many flavours and subsets of Agile that the best you could say is that it’s a tribe of open source projects. So it’s not very surprising that it is so tribal.
And of course this very tribalism explains a little of why Agile has not really come together since it first stuttered into existence – these tribes have their own interests, to the point that not only are they have little intellectual or professional interest in Agile as a whole but many have strong business and financial interests in making their fragment of Agile as separate as possible and to resist its incorporation into a largely, freely available whole.
So what would motivate any group to take part in creating Agile 2.0? The next few sections summarise why various generally speaking in the Agile universe should positively want Agile 2.0 to come into existence.
The Agile community
From the point of view of the Agile community as a whole – all its users as well as its enthusiasts and creators – Agile 2.0 would offer many benefits:
- It finally provides a unified definition of Agile as a whole.
- No need to master multiple sources.
- No more conflicts between flavours of Agile.
- It provides a operating model for Agile that is:
- Free to use.
- Built & maintained by Agile experts.
- It minimises the effort of planning & designing a roadmap
- For getting to Agile.
- For using Agile.
- It creates a shared definition of the entire delivery process across multiple products, teams, locations & organisations.
- It minimises the problem of recruiting & inducting new team members.
- It provides a thorough basis for planning professional development.
- It creates a platform for assimilating new developments in Agile.
- Also, performed by an independent team of Agile specialists.
- This makes it relatively simple to ‘sell’ Agile 2.0 to Agile’s existing users, and so we gain acceptance for Agile 2.0 across Agile as a whole:
- It simplifies Agile.
- It reduces the number of models any given team or organisation must master.
- It covers the entire delivery process.
- It offers everyone a massive free resource.
- Because is takes the form of a navigable website, it makes Agile easier to understand, implement & use.
- So it’s good for beginners.
- … and should accelerate the further development of Agile as a whole.
- It is an answer to many long-standing criticisms of Agile’s recent direction.
- Not least from many of Agile’s founders.
- Like other Agile models, it is totally optional, so it threatens no one directly.
- It provides a consistent definition against which professional certification can be mapped.
- As an open-source project:
- Anyone can contribute.
- The definition of Agile will be continuously maintained for the indefinite future.
Agile service providers
- Agile service providers – trainers, coaches, consultants – have reason to be wary of Agile 2.0. Why?
- Agile 2.0 changes the structure of the market in Agile services.
- So a determined & creative effort will be demanded – just to keep up.
- A comprehensive Agile ‘operating model’ competes directly with their services.
- Agile 2.0 offers a free alternative to paid services.
- An end-to-end Agile operating model makes many services redundant.
- 90% of process definition.
- Operationalising Agile.
- Bridging different areas of Agile.
- Integrating Agile into the wider organisation, etc.
- But Agile 2.0 will benefit Agile service providers too.
- Early adopters will be first to market.
- … and establish a dominant position.
- Agile 2.0 creates new opportunities.
- Agile 2.0 itself.
- g., products & services.
- g., marketing USPs & differentiators.
- Agile 2.0 provides a basis for alliances between service providers.
- e., no need to negotiate the shared ‘Big Picture’.
- A simple basis for demarcating services.
- Agile 2.0 provides a reference point for explaining – and so also marketing – their services more clearly.
- Agile 2.0 takes key elements of Agile away from large consultancies.
- g., modelling the ‘Big Picture’, operating model definition, end-to-end implementation model, etc.
- Levelling the playing field with smaller providers.
- Opening up larger organisations to smaller service providers.
- Contributors to Agile 2.0
- Because of the way we hope to build Agile 2.0, the direct contributors have a special interest in its development:
- Agile 2.0’s founders will be in an especially strong position to market their services.
- They will have a head-start on how Agile 2.0 is evolving.
- And an enduring authority as original contributors.
- Agile 2.0 will provide:
- A core project around which to organise contributor’s own future ideas and innovations.
- A shared product framework to which specialised products can be created in alliance.
- This will make their offerings more coherent & more scalable:
- For larger, more diverse & strategic engagements.
- For larger, more lucrative clients.
- Agile 2.0 offers a platform & driver for coordinating their services among contributors.
- And so provides a basis for business alliances.
- Principles for Agile 2.0
- Defining Agile 2.0
- Here are some basic precepts for defining Agile 2.0:
- Agile 2.0 should present itself explicitly as a tool and knowledge-base at its users’ disposal.
- Not a mandatory & fixed system for processing work or controlling people.
- Agile 2.0 should adhere to the principles set out in the Agile Manifesto.
- Though the Manifesto itself should be expanded, especially to reflect the corporate scale of software development.
- Agile 2.0 should be a single unified Operating Model, not a loose collection of frameworks.
- Agile 2.0 should define Agile to as low a level as is consistent with defining a shared approach.
- Potentially to the point of (optional) step-by-step procedures.
- Agile 2.0 should be suitable for (or at least adaptable to) any plausible IT delivery environment.
- Agile 2.0 should be published as a navigable tool for direct use by delivery teams.
- E.g., a website.
- Building Agile 2.0
- Here are some basic precepts for building Agile 2.0:
- Where mature models of Agile already exist, Agile 2.0 should retain as much as possible.
- Why > what > how.
- Agile 2.0 should be managed as a product.
- Agile 2.0 should be open source.
- Agile 2.0’s conceptual underpinnings & language should be fully generic.
- E.g., not ‘Scrum 2.0’ or any other language with a high level of commitment.
- This may require some restructuring of current language and concepts.
- Agile 2.0 should be subject to continuous product development.
- It should not a one-off deliverable.
- It should have a Core Team, Extended Team & wider constituency.
- Core capabilities
- It is also important to define the capabilities Agile 2.0 needs to support. Agile 2.0 should:
- Help users understand Agile as a whole
- Help users carry out their work
- Help teams and organisations adapt Agile to their local conditions and needs
- These capabilities can be thought of as some of Agile 2.0’s functional requirements.
- Help users understand Agile as a whole
- To help users understand Agile as a whole, Agile 2.0 should define:
- Agile 2.0’s values and principles.
- Ownership, strategy and management within the organisation and team.
- The delivery process.
- Boundaries and external interfaces.
- Top to bottom – from product vision to coding and testing.
- Start to finish.
- From need to outcomes.
- From product backlog/plan and pipeline management to desktop implementation and support.
- Components and their inter-relationships.
- Stepwise models of tasks.
- Organisation, roles and responsibilities.
- Including a well-defined entry and exit points for each participant in each activity.
- Responsibilities, knowledge and skills, daily routines and ‘the Right Person’.
- Core disciplines, tools and techniques.
- A lifecycle/methodology mapping the relationships between these elements.
- Help users carry out their work
- Above all else Agile 2.0 should be a tool to help users carry out their work. To be this, it needs to:
- Identify a broad set of processes & products for defining & planning releases.
- Model all tasks required to carry out this process, & so explain what the delivery process does.
- Model basic organisations, roles & responsibilities.
- Define each task to a common standard, showing how the process works.
- Output, steps, inputs.
- Supporting methods, tools & techniques.
- Roles & responsibilities.
- Common issues and risks.
- Enable users to:
- Understand their specific responsibilities & tasks.
- Know where to start.
- Hand off to/from neighbours.
- Include a navigable tool for direct use by delivery teams.
- g., a website.
- Support adaptation to local conditions and needs
- Though always rigorous, Agile is never rigid. To ensure that Agile 2.0 upholds this standard. It should make it easy for teams and organisations to adapt Agile to local conditions and needs. In particular it should:
- Provide guidance on adaptation.
- Adopt object-oriented principles to define and develop Agile 2.0 itself.
- Apply abstraction, encapsulation, modularity, interfaces, etc. to all elements of Agile 2.0.
- Identify the risks and issues associate with adaptation.
- What should Agile 2.0 look like?
- Like the product of any other Agile development, it is impossible to specify what Agile 2.0 will look like in advance. However, to give a sense of the type of product Agile 2.0 is likely to be, the following sections outline a hypothetical architecture.
- Structure of Agile 2.0
- As already mentioned, Agile 2.0 should be delivered in the form of a highly structured, highly navigable website, capable of immediate practical use by users.
- Ideally this site would be configurable by individual users:
- To hide non-required elements of Agile 2.0.
- To add links to their own additional or replacement elements.
- To configure and link to compatible tools.
- Content of Agile 2.0
- The content of Agile 2.0 should consist of an operating model covering the full scope of Agile 2.0 as described in above.
- To Agile 2.0.
- To the Agile 2.0 site.
- Agile meta-model.
- Research on Agile.
- How Agile is really used.
- What really works.
- Agile strategy.
- Business/IT alignment.
- Product management.
- Scaling Agile
- Delivery at scale.
- Scaled organisation.
- Scaling disciplines, tools and techniques.
- Product management.
- Outcomes and outputs.
- Task flow, including options.
- Associated disciplines, tools and techniques.
- Release management.
- As for Product Management.
- Iteration management.
- As for Product Management.
- Agile organisation.
- End-to-end delivery team.
- Development team.
- Corporate stakeholders.
- Agile tools and techniques.
- Scope: All tools and techniques required to support Agile delivery.
- For each item:
- User instructions (for use within Agile 2.0).
- Variants & advanced versions.
- Access to Agile 2.0
- The site should be open to any self-registered user.
- Agile remains one of the mostly powerful innovations in delivery since the invention of the Gantt chart. But it has its limits – many of them reflections of Agile’s rather hesitant early history, compounded by the way it has split into often belligerent tribes, co-opted by consultancies and tool vendors, and anatomised by online critics and specialist consultants and coaches amplifying minor difference and promoting fringe techniques.
- Creating Agile 2.0 will not eliminate all (or perhaps any) of these problems altogether. However, it would provide a refreshed definition of Agile that might allow new practitioners and established experts alike to consolidate their view of Agile and move forward. Hopefully this article provides a rationale, a model and a way forward around all those who wish Agile well can group.