Since the Agile Alliance formed and met at The Lodge at Snowbird Ski
Resort in February of 2001 to explore better ways of developing software, an apparent
love affair has developed with agile practices in the Software Engineering
Community. Yet it is particularly strange that there is little in the way of
critical analysis and evaluation of agile methods and whether the net result of
the adoption of agile brings those adopters any measurable benefit. Could agile be little more than a shell game
and sleight of hand? Regardless, we are concerned that the lack of serious
critical scrutiny of agile methods may be contributing to far greater
challenges among Software Engineering organizations throughout the world than
they started with. To be certain, there
has been to date a dearth of critical academic scrutiny of Agile or the
multitude of self-proclaimed Agile Methodologies and the practice of Agile within
the profession. Such a critical perspective is necessary. An objective perspective is needed. The authors do not intend to simply take a
contrary perspective to those of the various proponents of Agile. Instead, we wish to provide a critical assessment
of the Agile Software movement and report a purely objective assessment of
Agile Software Engineering in concept and in practice.
1. Introduction
The
Software Engineering world is rather in love with the concept of Agile
Development. We cite and reference Dingsøyr,
et al.[1]
who published a paper trumpeting A Decade of Agile Methodologies, though the
intent of this paper is to laud the success of Agile Methodologies versus
conducting critical research as to the advantages and disadvantages of Agile
Methodologies. There is a large community of Software Engineers who see Agile
as a magic bullet that will solve every problem under the sun.
We will proceed on the Converse assumption. First, we will examine the Agile Manifesto itself and look for fallacies or pitfalls in the
original document. Second, we examine the most prevalent
approaches claiming to be agile and examine the strengths and weaknesses of
each approach and attempt to distill whether these agile methods are producing
measurable results that differ meaningfully from the results teams achieved
using more conventional methods. Finally,
we conclude our critical analysis of Agile Software Development and summarize
our findings.
2. The Agile Manifesto – Strengths and weaknesses
Clearly, the team that produced the
Agile Manifesto vetted their concepts in a rather formal manner. They attempted to provide a set of principles
-- in the Aristotelian sense -- that Software Developers would employ and that
would improve the way that they worked.
Each member of the group assembled to draft this document brought with
them a set of experiences and a vision of what they believed constituted best
practices in Software Engineering. In
the end, they published a document not unlike the United States Constitution or
the Communist Manifesto. Clearly, the
Agile Alliance was keen on starting a revolution against traditional methods. What
is not clear is whether Agile in practice actually solves any real problems or
if it results in a superior outcome.
Working software is not a guarantee of any material results. There is an awful lot of working software in
the world that never gets used or which is used incorrectly thus negating the
overall benefit of the software. The
Agile Alliance missed the point with the focus on working software. It is far more important that the software be
intuitive, adaptable and maintainable over the full product lifecycle.
One post on “agilemanifesto.org” reports the
history of the team, self-named “The Agile Alliance” that drafted the Agile
Manifesto on February 11th-13th of 2001, and states, “…a
bigger gathering of organizational anarchists would be hard to find, so what
emerged from this meeting was symbolic – a Manifesto for Agile Software
Development – signed by all participants”
The result has been somewhat striking in that there is scarcely a
software engineering team throughout the United States that does not aspire to be an Agile Team. It is not
absolutely clear why Software Engineering Teams are striving to evolve their
methods from conventional techniques toward agile methods. Nor is there any empirical evidence that
Agile methods improve software engineering in a tangible manner such as reduced
overall cost. On the contrary, it would
appear by the enormous size of a typical agile software engineering
organization that Agile has accelerated more than the software development
process, it appears that Agile is vastly superior to conventional methodologies
in using up whatever financial resources are available. It is also unclear how agile
techniques are evolving the role of the Software Engineer within the profession
and whether the outcome is actually desirable.
3. Strengths of the Agile Manifesto
The Agile Manifesto is a brief document stating that the
authors were uncovering better ways of developing software by doing it and by
helping others to do it. Clearly the 4 core values that they focused on are
intriguing. We will explore these first:
1)
Valuing individuals and interactions over
processes and tools
2)
Valuing working software over comprehensive
documentation
3)
Valuing Customer collaboration over contract
negotiations
4)
Valuing the ability to respond to change over
following a plan
Objectively, there doesn’t appear
to be anything particularly subversive about these modest principles. Clearly there is notable good in valuing the
individuals and their interactions over the processes and tools. To wit, some organizations, especially large
organizations had, at that time, become so overwhelmingly stagnated by process
and governance that it became prohibitively difficult to get any real software
developed at all. We applaud the work of
the signatories and admit that the emphasis on process that was the norm in the
25 years prior to the writing of the Agile Manifesto had reached a threshold of
being stifling and often impeded the development of software products.
Valuing working software over
comprehensive documentation sounds pretty good, but we must pause to assess the
implications, namely: Is this a good principle?
We have doubts which we will examine in the next section.
Valuing Customer Collaboration
over Contract Negotiations also sounds good insofar as we feel better about
Collaborating with a Customer versus engaging in Contract Negotiations with
that Customer, but at this too requires further analysis.
We also agree that it is important
in many situations to be able to deviate from a plan in order to respond to
changes that were not anticipated at the outset of a software project. Such changes are ultimately guaranteed to
occur in all but the most trivial software engineering endeavors.
Thus, on the surface the Agile
Manifesto appears to be a rather intriguing document with generally good intentions.
4. Pitfalls of the Agile Manifesto
There are several fallacies and pitfalls associated with
Agile Development in practice today. We trace these pitfalls to central
omissions in the Manifesto:
1) Agile methods turn over much of the
systems analysis, design and technical architecture to software
engineers/developers who may not have the proper skill set to analyze, design
or architect such a solution. More often than not these individuals are
sequestered from the users of the software and even the key stakeholders who
are on point for the system requirements thus there is rarely any means to
achieving effective requirements analysis and reaching a clear understanding of
the solution domain. The Agile Manifesto
does not discuss the vitally important concern of systems architecture.
- Agile practitioners tend to take the quintessential square peg of solution requirements analysis and stuff it into the round hole of a 2-week sprint. For the most trivial of information technology problems where the teams have access to the system owners and key stakeholders, the time-frame may be acceptable, but for more interesting problems, it is not. Examples where two-week sprints work in opposition to the business need include
- A quoting system for insurance products, which demands extensive information gathering, and which is governed by complex actuarial tables and regulations is a far-fetched goal for a 2-week investigation.
- The solution requirements for the guidance and anti-collision systems for a self-driving automobile.
- The analysis of a system that uses human genome mapping to aid in the development of medications for human diseases also should not be packaged into even a series of 2-week iterations – this is an entirely unreasonable expectation.
- Agile also tends to bracket engineer learning cycles into these two-week intervals. In reality, some learning takes more time than 2 weeks.
- Agile is not an architectural discipline even though it clearly requires tremendous architectural discipline to implement the requisite continuous integration processes which are at the core of most agile practice and embodied in the statement by the signatories, “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.”
- In fact, Agile practitioners have widely learned that Agile methods in practice violate the first and over-riding principle of the Agile Manifesto, namely: that of valuing “Individuals and interactions over processes and tools.” If we are to be honest with ourselves, Agile techniques the world over are emphasizing processes and tools even more than conventional project philosophies and methodologies. An important maxim is suggested: When adopting agile, avoid the adoption of complex processes and tooling like the plague!
- Too, it is foolish to believe the second principle of the agile Manifesto: that of valuing “Working software over comprehensive documentation” Teams that forego comprehensive documentation are doomed to a vicious cycle of software artifacts that need to be rewritten far too soon after they are delivered, as they will have failed to capture important knowledge relating to the modules that were developed. Moreover, Testing, even exhaustive testing is not a replacement for comprehensive system documentation, nor is it a substitute for a thoughtful and carefully considered systems architecture that takes into account the functional and non-functional requirements of the solution. Poorly documented products are almost always used incorrectly!
In the best-case scenario, Agile practices place solution
architecture responsibilities are in the hands of a software architect who is secluded from the software engineers, and the
software engineers must blindly trust that the analysis of the solution requirements
was performed correctly. Assuming so, they can proceed to build a
solution that fits the need. Otherwise, the developers are routinely foisted
into the role of back-slope architects cleaning up the mess made by the Software architect whose work is not vetted. This is most often the crux of the problem:
1) Companies frequently do not employ software architects and do not
consider the importance of preparing a sustainable software architecture
2) Companies
place the responsibility for defining the software architecture in the hands of
individuals who are either poorly suited to the role or who lack access to the
sponsors and key stakeholders of the solution.
3) Companies
are relying increasingly on Software engineers on agile teams to back in to a
solution architecture that will satisfy the needs of the business and we see
this as fraught with disaster.
5. Dominant Agile Methods
There are several dominant agile approaches that have been
widely adopted within Software Engineering teams over the past 16 years:
- eXtreme Programming (XP)
- Scrum
- Lean Software Development
- Crystal Methodologies
- Feature-Driven Development
We will examine each of these methodologies in detail in the
following sections.
Extreme Programming
This methodology is predominantly focused on pair programming;
however, eXtreme Programming is also underscored by a formal methodology that
emphasizes ceremony and checkpoints referred to as feedback loops common to
most traditional software processes. The
process diagram for Extreme Programming (below) seems to be quite busy, and
seems to add pair programming solely as an afterthought
Scrum
Scrum
and its derivatives which include SAFe Agile (an acronym for Structured Agile
Framework for Enterprise) are among the most process- and ceremony-heavy of
the agile methods. While, most Scrum
teams disdain comprehensive documentation, they employ a broad complement of
tooling including tools for continuous integration, test driven development
tooling and with increasing frequency they choose to implement convoluted
release management and branching strategies which would seem to be entirely
contrary to the first principle of the Agile Manifesto, it makes us wonder if
Scrum methods are even considered “Agile” by the members of the Agile Alliance.
Lean Software Development
The
Lean Software Development concept was developed by Mary Poppendieck and Tom
Poppendieck. Their
approach presents traditional lean principles as typically found
in manufacturing, in a modified form, as well as a set of 22 tools and compares the tools to agile
practices. The Poppendiecks' involvement in the Agile software development community, including
talks at several Agile conferences has
resulted in such concepts being more widely accepted within the Agile
community. Clearly the Poppendiecks are
interested in marketing their methodology more than they are interested working
software or any of the other principles of the Agile Manifesto. It seems most process heavy, and therefore,
contrary to the first principle of The Agile Manifesto to require the
employment of 22 tools to achieve agile practices. Lean Software development
would be intriguing if not for these 22 tools needed to employ the methodology.
And the authors have not observed any teams actually adopting this methodology
in practice
Crystal Methodology
The Crystal Methodology is an encompassing collection of
eight different project lifecycles or methods; however each reflects 7 common
characteristics:
1.
Frequent delivery
2.
Reflective improvement
3.
Close or osmotic
communication
4.
Personal safety
5.
Focus
6.
Easy access to expert
users
7.
Technical environment
with automated tests, configuration management, and frequent integration
The
focus on frequent delivery and frequent integration in Crystal Methodology is a
concern as we have observed that all approaches that focus on Continuous
Delivery tend to suffer from the inevitable complexity that Continuous Delivery
implies. Notwithstanding, the emphasis that Crystal Methodology places on reflective
improvement and personal safety are admirable.
Nevertheless, we have observed in practice that when Personal Safety and
open communication are valued, there is the occasion for highly negative
influences to monopolize the time of the team thus contributing to bottlenecks
and reduced team throughput and performance. Nevertheless the value placed on
easy access to expert users is a mitigating factor that is likely to further
advance projects employing Crystal Methodology.
We have employed technical environments with automated tests and
configuration management and frequent integration and release cycles. What we conclude is that each software
product must be architected with these activities in mind and when software is
architected to support such an environment and such processes and is founded in
stable architectural patterns, there is no better approach to building quality software
Feature Driven Development
After examining the remaining approaches to agile we finally
arrive at Feature-Driven Development.
FDD is founded in Peter Coad’s concept of Software Features, where a feature
similar to a use case in the context of the Rational Unified Process (RUP). The major activities of FDD include:
1)
Develop an Overall Model
2)
Build a features List
3)
Plan by feature
4)
Design by feature
5)
Build by feature
Each
of these steps or activities produces a defined deliverable. Most important the Feature List is the
product of step 2 and is a governing document for the entire methodology. FDD unlike the other methodologies reviewed
here actually incorporates a formal Requirements Analysis process (termed
Requirements Envisioning) as part of its Iteration 0: Envisioning, which
consists of Initial Requirements Envisioning and Initial Architectural
Envisioning the team then transitions to Iteration Modeling followed by Model
Storming and Test-Driven Development, which comprise the parts of an Interation
1: Development activity. Though FDD also
has substantially more process and ceremony than indicated by the first
principle of the Agile Manifesto, it is clear that the concept of FDD provides
a more transparent process than the remaining dominant Agile approaches. Moreover, it appears that FDD is rooted in
the Coad and Yourdon methodology of Object-Oriented Design and incorporates
practices that we view as similar to Rapid Application Development with FDD’s
focus on Requirements Envisioning (reminiscent of JRD) and architectural
modeling as well as joint design activities (reminiscent of JAD). The emphasis
on Test-Driven Development (TDD) is excellent, but in the end analysis FDD,
while a self-proclaimed agile methodology is difficult to classify as agile,
due to its notable departures from the first principle of the Agile Manifesto. Still the approach is compelling due to the
fact that it appears that the knowledge gained over the course of developing
the software is documented, if only informally, thus the knowledge is not lost
which seems a cornerstone deficiency in the other methods which we studied.
The purpose of this critical
perspective of Agile Software Development is not to say that there are no
benefits and nothing to be learned from agile software practices and methods.
Rather, we must be realistic in terms of what we expect from the adoption of
agile methods. There are some good ideas in the agile manifesto, but there
are also ideas that are intrinsically in conflict with the reality of business
and software engineering principles. Good engineering principles should
not in practice be shunned solely due to what amounts to religious
fervor. There are no magic bullets in Software engineering and as
scientists, we should take exception when our leaders parade through the main
thoroughfare of our profession without their knickers, asserting that Agile
Software Development is the most beautiful paradigm they’ve ever seen.
In short, before adopting
Agile, it is vital to frame the issue carefully and ask, “What problem are we
attempting to solve by changing our fundamental approach to how we develop
software within our organization? The
authors have observed some organizations that employ the ceremonial aspects of
agile – such as the daily scrum -- while keeping much of their traditional
waterfall process intact. The end result
is the ability to more closely manage the time-sensitive topics within a
project while providing a higher level of communication within the team. The authors are also familiar with certain CTOs
who are strongly opposed to Agile methods in any form as they assert that they
have been burned by Agile and will not accept such notions within their
organization. One team in working under
a particular CTO was already practicing a lean approach to software engineering,
which was not, strictly-speaking, Agile.
The team simply reported that they were practicing lean software
development and truthfully testified that it was not “Agile”. The reality of this team's culture was as follows:
- They did not lack documentation of the knowledge gained throughout the development effort
- They were not over-burdened with either process, tools or ceremony
- They worked together harmoniously
- They had tremendous knowledge of the domain in which their products were targeted
- They understood their mission and purpose with great clarity
Leaders and leading researchers in the Software Engineering discipline must strive to keep in front of the trends in the profession and caution practitioners from dealings with snake-oil purveyers and hair tonic salespeople. The tremendous flocking to agile that we observe throughout the profession should be a cause for alarm. There has been no legitimate vetting of Agile practices and no metrics to help practitioners to make an informed decision whether Agile offers and tangible benefits in terms of cost or time to market. Without such data points there is no way for software engineering teams to make any informed decisions about the advantages or disadvantages of adopting Agile methods.
1Dingsøyr, Nerur, Balijepally and Moe - A decade
of agile methodologies: Towards explaining agile software development, Journal
of Systems and Software V85, Issue 6, June 2012 Pages 213 – 221
(http://www.sciencedirect.com/science/article/pii/S0164121212000532)
No comments:
Post a Comment