Hoodwinked

The Agile Deception: a New Path for Software Engineering

For over two decades, Agile advocates have hoodwinked the software engineering community. We have attempted to force the square peg of Agile methods into the round hole of disciplined engineering, blinded by a fervor that prevents us from imagining what comes next.

The reality missing from the "Agile Transformation" hype is:

Software does not Become Agile Simply Because we Proclaim it so. 

Short iterations alone do not guarantee flexibility. True agility requires a master plan—a blueprint for how software is architected, designed, assembled, and coded.

The Case for Software Stability

Frameworks have long empowered engineers to build stable, sustainable systems. A promising path forward is Software Stability, a model pioneered by Dr. Mohammed Fayad. This approach posits that software can only be agile if it is designed for stability, scalability, and extensibility from the outset.

The promise of Software Stability is an evolution from "brute force" coding to the configuration of stable modules. This aligns seamlessly with modern AI initiatives. By using a foundation grammar closer to natural language than traditional code, Software Stability Models allow AI "bots" to pair with developers or independently generate test suites. We need technologies for the future, rather than continues dictates imposed on legacy systems that cannot support them.

The "New Boss" Fallacy

Agile began in 2001 at the Snowbird resort in Utah. The mission was to find an alternative to heavyweight "Waterfall" methods. While the critique of Waterfall’s inflexibility was valid, the industry essentially installed a "new boss" that was the same as the old one. We swapped one set of challenges for another, often creating new problems in the process.

The fundamental issue is that software ages. It becomes obsolete as technologies fail to keep up with modern tools. Academic literature suggests most software must be rewritten every five years. However, Agile’s disdain for "bloat" often results in a total lack of documentation.

Without documentation or a comprehensive test suite, how can a team achieve a necessary rewrite? Reverse-engineering a system is prohibitively expensive, yet Agilists decry documentation as unnecessary, leaving teams to wander through legacy code without a map.

The Fatal Fallacy: NBUA

We observe the focus on NBUA (No Big Upfront Anything) as a fatal flaw. This "fail-fast" maxim results in "local-minimum" designs—solutions that work for a specific sprint but lack the integrity required for durable, supportable software.

This has led to a growing "Anti-Agile" community born from the ashes of failed projects. Executives, frustrated by the skyrocketing costs of building and managing software, are sold "Agile Transformations" as a cure-all. When these transformations fail to lower costs, the blame is shifted to the engineers. Layoffs follow, but this is a fool's paradise; no one wants to admit they purchased snake oil.

The "Blind Leading the Blind"

In large organizations, Agile—particularly Scrum and Kanban—often results in the "blind leading the blind." By persuading executives to "fire all the project managers," Agilists have left projects rudderless.

• Abusive Timelines: Scrum Masters, often accountable only to cadence rather than technical requirements, push engineers toward arbitrary deadlines.

• The Scalability Wall: While Agile may function in small teams, it struggles in organizations that have grown beyond the SMB threshold.

• Governance Gaps: Overall architectural integrity is sacrificed at the altar of the sprint.

Conclusion: Beyond the Fallacy of Extremes

We have fallen prey to a fallacy of extremes. Waterfall was too inflexible, but under an Agile regime, no one knows when the software is broken until it is too late. We have traded long-term sustainability for short-term velocity.

Agile is short-sighted; it fails to address the critical lifecycle considerations of software and obscures the need for a path toward the inevitable rewrite. To move forward, we must stop imposing dictates on legacy technologies and return to a "right-sized" design and architecture—one that values stability as much as speed.

 

You are Dead to Me

For the reader who has read my "Dirty Little Secret" essay on this site, you will not be surprised to hear me say this:  

Enterprise Agile is dead to me!

Agile marketeers have been hocking agile to large scale enterprises for more than a quarter of a century.  What has it yielded?  I'll wait...

To reiterate, Agile is fine for startups and green-field development, but even then, it can be argued that the cost of spinning up an Agile practice is not economically sound..  Agile never promised a plethora of tools that would be required to preside over an agile engineering discipline.  In fact, the Agile Alliance abundantly rejected tools in favor of personal interactions.  Yet in practice most companies adopting Agile quickly outfit their Agile practice with a large acquisition of tools:  It's an unmanageable proposition, especially when the cost of the tools compel the enterprise to fire  all of the project managers in the interest of balancing the financials, after opening the coffers to buy all of the tools.

1. Tools for CI/CD aka pipelines, with not less than 1/2 dozen tools required.

2. Tools for Agile Management (is this even a  legitimate term - Agile Management?),We rarely see any integrated management in enterprise Agile rollouts. Instead we have clueless Scrum Masters who know nothing about traditional project management, and typically  have even less knowledge or experience with agile management)

  a. We get JIRA, Confluence, Service Now, and others....

3. Tools built on top of CI/CD tooling to address legacy concerns that traditional CI/CD tooling does not handle. Worse still, everything happens at the same time, so there is no ability to assess impact or incorporate any planning into the  "Agile Transformation" - a term that I am quite certain means "Suicide" in The Oxford English Dictionary.

The Tooling required does come with not only up-front costs and maintenance and leasing costs, but there are also integration costs that are not typically well understood.  These often get bundled into the term DevOps, but these DevOps concerns are often manifested in roadblocks, which cannot be easily quantified, nor circumnavigated in highly controlled or even regulated enterprise environments. The costs add up, but no one is responsible for tracking the overall costs of engineering around the requirements imposed by Enterprise Agile and it's requisite tooling.

Given these findings.  I can easily say, that Enterprise Agile is dead to me.  I hope that the reader finds this information of benefit.

Isnt it time?

Agile is getting old, yet it hasn't matured.  Isn't it time for a facelift for Agile? 

Or, in the alternative, isn't it time for smart engineers start to look beyond agile?  

Facing the facts, it is clear - Agile is not the holy Grail of software engineering discipline. Instead of feeding the beast that is Agile, shouldn't we be seeking to feed our Customers and users once and for all, and shake off the technology monkey that  has taken a ride on the backs of software engineers at no small cost?  AI should fill many gaps.  AI can be our DevOps if employed strategically to automate the DevOps activities, simplifying CI/CD pipelines and take software engineers out of that tedious and time-consuming activity.

A Dirty Little Secret

The world over, companies are embracing the Agile Revolution.  All software engineering is relegated to agile practices. Sounds good, right?

Wrong.  Agile is not a one-size fits all solution.  It is not even a methodology.  Agile is only a set of guidelines.  Most companies embracing agile have no appreciation for the pitfals that come with agile.  Invariably, many companies will embrace the SCRUM methodology in some form or other.  Unfortunately, SCRUM is not really a methodology either.  Rather, it is a further set of guidelines and rituals that may be adopted.

There is a dirty little secret about SCRUM and Agile.  The only promise that the agile alliance makes is that there is an emphasis on working software.  This is a good thing, right? Not exactly.  

In practice, SCRUM and Agile are pitched to corporate executives with the promise of greater efficience (read: cost savings).  The dirty little secret is that there are virtually no successful agile implementations of Agile methods in the corporate world.  Large corporations come replete with a set of challenges that Agile is not equipped to address. Namely, large corporations have the challenge of large amounts of legacy and commercial software  products which may be highly customized and integrated into a vast web of complexity.  Furthermore, corporate software may have evolved over years, or even decades such that the underlying business rules which prompted the development of various features of the applications are not well understood.  These software modules are typically very poorly documented and it is prohibitive to reverse engineer the software, and the time and budget to do this is typically astronomical.

The bottom line, is that the "experts" pitching agile to corporations are selling snake oil.  There is literally not one large corporation the world over that can point to any successful agile implementation at corporate scale with the possible exceptions of Amazon and Ali Baba, both are arguably green-field, startups, which is the sweet-spot for agile approaches.  When you have no legacy software, no installed base, and no contractual commitments whatsoever, agile can work brilliantly, but established corporations are being blind-sided by agilists who promise the world only to deliver nothing of value.