Since the authors of the “Manifesto for Agile Software Development” developed “an alternative to documentation-driven, heavyweight software development processes,” project management practitioners have adopted elements of Agile frameworks, but many projects don’t really apply Agile in its pure sense as originally intended. This results in confusion, setting the wrong expectations with project stakeholders, and disagreements about scope and deliverables.
The Agile Manifesto outlined a set of values and principles which emphasize responding to change over following a plan, delivering smaller increments of working software within a short timescale, and actively involving the customer in the development cycle. The Agile Manifesto did not define a specific way of applying these principles in practice. Before Agile, Waterfall was the common approach used in software development projects until the 1990’s when some organizations and teams started adopting other approaches such as Extreme Programming (XP), Scrum, and Kanban that are more closely aligned with Agile principles. One of the primary objectives behind Agile is to allow for and actually embrace change while building the software product. Everyone knows from experience that things will change as soon as you start building the product, no matter how much time you spend up-front talking about requirements and analyzing the problem. Agile assumes that the scope is uncertain and empowers teams to clarify requirements and define scope while building the product, not before starting to build it.
Using elements of Agile
If you are tasked with determining things up-front and delivering a fixed scope within a specific schedule and budget, then your project is most likely not really Agile. It doesn’t matter if you are using Sprints, Kanban boards, Product Backlogs, Story Points, or Daily Standups. A project that truly follows Agile principles is one where the scope is defined, prioritized, and managed during product development. In order for this to work effectively, the business (represented by the Product Owner in the Scrum framework, for example) must be actively involved throughout the product development cycle and not only at the beginning and end of the project. The Product Owner is responsible for clarifying requirements and determining priorities based on the capacity and velocity of the development team (e.g., Scrum team) and business needs, not based on pre-determined scope and schedule. If this flexibility is not granted to the project team and they are constrained by delivering a predefined scope within a certain budget or timeline beyond the current increment, then the project is most likely not really Agile.
Setting the wrong expectations
Many organizations start applying Agile at the project level without a broader organizational understanding, alignment, or commitment to Agile principles. As a result, they end up using elements of Agile frameworks (Scrum, Kanban, etc.) and following ceremonies, artifacts, and roles defined by these frameworks, but not really adhering to the true intent of the Agile Manifesto. In order to provide executives with longer-term visibility and commitment, some organizations conduct an up-front discovery phase before starting to build the product. While this informs planning, it doesn’t fully solve the scope commitment problem as a short discovery phase only produces high-level scope, and more detail will need to be flushed out during the build phase. This puts the project team under significant pressure by committing to deliver some scope, which hasn’t been fully clarified yet, within a fixed timeline and budget, while at the same time providing the business with the flexibility to define their requirements in more detail and even change these requirements during the product development cycle. This sets the wrong expectations, and if not managed really well, results in disagreements about scope and cost, missing deadlines, and disappointed stakeholders.
Embracing Agile at the organizational level
Agile is a fundamental change for many organizations. Therefore, Agile concepts should be understood and adopted at the organizational level, starting at the top, before trying to use Agile in projects. Effective application of Agile starts with educating everyone involved, especially executives and middle management, about what Agile really is and understanding key differences from other ways of software development, related benefits, and potential risks.
Business stakeholders must understand and agree up-front that scope is uncertain and subject to change during the product development phase while, at the same time, be willing to commit to a fixed budget and timeline. Business representatives must be actively involved throughout the product development cycle and guide the prioritization and value that the business expects to be delivered. Additionally, the organization needs to embrace the idea of following an iterative, incremental approach, self-organizing teams, and active engagement of business users and sponsors in the development process throughout the project. These are basic tenants of the Agile software development approach. Organizations that are able to make this mind shift are better positioned to implement Agile properly and reap its benefits in the long run.
This article is not meant to be for Agile or against it. Projects can still succeed whether they use Agile, Waterfall, or something in-between because Agile is not necessarily the most suitable approach for all projects, and there are many other factors that determine project success or failure. However, your project is either following pure Agile principles, or it is not. And that’s ok, but you have to be very clear on the approach you are following to manage your project and make sure the project team and business stakeholders fully understand it and know what to expect.
For more information on this topic, see my article: Why the confusion around Agile in the software development community?