The Battle of the Sexes

The Battle of the Sexes

I was eleven years old in 1973 when the whole nation watched "The Battle of the Sexes" play out on television. Never mind that a movie was made of the subject in 2017.  The original competition between Bobby Riggs and Billie Jean King was touted as the battle to end all battles. But the hype was far more interesting than the tennis match.

And years later, we are left with all of the hype and very little substance.  The battle of the sexes has played out on many levels in US culture and society.Women's liberation continues to fight for equal rights and against the male-dominated establishment, when, in fact, the male-dominated establishment retired decades ago.  Unfortunately, the gender wars have raged on. We have women in the C Suite, Women in Congress and in the Senate. Still, the equal rights amendment hasn't been passed, and some women are sincerely P-O'ed and feel victimized by a society that seemingly fails to recognize Women as equals.

I disagree.  I think that women have had a far better outcome due to the unsettled nature of the battle of the sexes.  Moreover, I believe that feminists would rather not talk about the changing roles of men and women in society.  Women have enjoyed a thoroughly free ride in the past five decades.

While American Society has evolved to the point where it is nearly impossible for a family to survive on a single income, many women still hold on to the fantasy that they can aspire to be housewives, sitting at home eating cherry Bon-Bons a la Kate Bundy, while their husbands toil away frantically trying to make ends meet, pay the bills and save some money so that the family can enjoy just one annual vacation each year -- or perhaps every other year, and so that the children might go to college one day and become self-sufficient. No thought or dollar is invested in the thought that the bread-winner himself might actually retire from work at least a few years before he's 6 feet under!

What does this all mean?  I have uncovered a blockbuster finding:  Despite all of the pomp and circumstance surrounding the infamous "Battle of the Sexes".  Nothing at all was settled.  The fact of the matter is that our nation has never had a candid dialogue pertaining to gender roles in the age that followed the battle of the sexes.  Absent that we have fumbled along quite clumsily attempting to achieve some semblance of balance in an otherwise unbalanced world.

The end result is that we have a tenuous battle for a woman's right to choose versus a fetus' right to survive.

We also have an ongoing war of words between feminists who have utterly failed to engage in a long overdue dialogue as to what gender roles look like after the equal rights amendment and after the famed battle of the sexes is finally settled.  Surprisingly, the alleged male-dominated establishment does not especially embrace the concept of Women's liberation as defined by the National Organization for Women or embraced by more militant feminist factions.  But it all comes down to the failure of the Women's Rights movement to engage in a constructive dialogue about how society functions after the battle of the sexes -- as if it were the duty of men to sit them down and ask if they might condescend to even entertain such a dialogue.

I don't pretend to know what the solution is exactly, but it won't get solved until the dialogue is happening in earnest.

Agile pitfalls to avoid ... At all Cost

Agile Pitfalls to Avoid 

...At all Cost

Copyright © Thought Rising 2020 All rights reserved.

Working in Agile Teams over the course of more than 15 years.  We have observed some distinct benefits; however, there are some formidable pitfalls and bad habits that can doom agile teams.  This paper, classifies these pitfalls and bad habits that must be avoided at all cost to ensure a successful Agile experience for the engineering team and for the product owners and end customers

In our experience, we have observed one or more specific pitfalls or fallacies corresponding to each of the 4 principles (or Values) delineated in the Agile Manifesto as detailed in the following table:

Principle/Value	Pitfall/Anti-Principle
Individuals and Interactions over Processes and Tools	Agile is a Gateway Drug The Rat's Maze / The Swarm Instinct.
Working Software over Comprehensive Documentation	NBUA - No Big Upfront Anything Non-Agile Software Architectures
Customer Collaboration over Contract Negotiation	Agile Obesity The Fallacy of "Yes" - Over-committing leading to Under-delivery High Software Defects and Brittleness Excessive Rework Cycles.
Responding to Change over Following a Plan	Inadequate Testing

1) Agile is a Gateway Drug

The first agile  principle values Individuals and Interactions over Processes and Tools; however, the need to automate routine processes often leads to massive tool propagation. In this respect, Agile itself is a Gateway Drug, such that ever more tools are needed to orchestrate the agile experience.  For example, we see the following tools employed within agile teams the world over:

1) JIRA for issue tracking

2) Confluence for Just-in-time documentation

3) Junit for Java Unit Testing.

4) DB Unit for database unit testing.

5) Mockito for injection of Mock objects into JUnit Tests.

4) Selenium for Application Test Automation and regression testing.

3) GITHUB for source code management

4) Artefactory for code library management

5) Jenkins for build management

6) UDeploy for deployment automation

7) Splunk for Application Logging and log analysis.

And on and on the list grows.  It is important to note that we have not listed all of the various tools that are needed within the production support team for the overall support and trouble-shooting of applications. It might be argued that many of the tools listed above were always needed, but this is a fallacy in itself.  One might ask, "how did we build software before these tools were available?" Simply, we found ways of solving the technical needs for building software by building one-off tools and thus we were building tools from scratch over and over again.  Perhaps these tools can save us time? Yet there is a cost to all of this tooling, and in the end analysis we were supposed to value individuals and interactions over processes and tools.  Where did Agile lose it's way? It seems that Agile was always destined to become a gateway drug for so many processes and tools.  We conclude that it's a fair question to ask the founders of the Agile Alliance, "Where did we go wrong?"

2) The Rats Maze / The Swarm Instinct

The value of Individuals and Interactions over processes and tools can be undermined when the Individuals and Interactions are not governed by some structure.  Since Agile is also predicated on self-organizing teams.  There is a tendency for disarray to reign within agile teams.  Everyone is on an individual mission to find a piece of cheese within a virtual rats maze.  Thus Agile Team members may find themselves wasting cycles searching the virtual rats maze for answers to technical questions whereas someone else has already found the virtual cheese.

Another analogy is the "Swarm Instinct".  In competitive soccer, we learn that inexperienced players tend to swarm wherever the ball may be on the field of play.  Whereas experienced players, teams and coaches will exploit the swarm  instinct by playing positional play which will allow a more experienced team to exploit the inexperience of the swarming opponent.  Grady Booch famously asserted that Software Engineering is a team sport; however, there is clearly a distinction between haphazard, self-organizing teams and a team that employs a fundamentally rich game-plan, where each team member has a defined role and position on the virtual field of play.  While there is no direct opposing team  per se  within  the Agile project environment, the takeaway is that a team working to a rich gameplan will achieve far greater success and far more reliable and repeatable results versus a team that is entirely self-organizing.

3) NBUA - No Big Up-front Anything

The NBUA pitfall was first detailed by Bertrand Meyer.  Meyer's assertion is that agile projects are destined to fail when agile teams reject all forms of up-frront documentation, in consideration of the assumption that all of the documentation relating to the software will ultimately become outdated.  This is not a legitimate rationale for decrying all up-front documentation.  An important consideration is to invest the appropriate level of effort upfront in any software engineering effort to design a solution that is built for test-ability and that embodies an Agile Software Architecture.  These points are treated in more detail in the following section. But it is sufficient that the fallacy of NBUA can be the root cause of many Agile failures.

4) Non-Agile Software Architecture

Agile software development works best when there is a high degree of repeat-ability.  Software architectures that are cumbersome do not lend themselves to agile software development.  There are two flavors of the Non-Agile Software Architecture Pitfall:

a)  The Square-Peg/Round-Hole Fallacy

b)  Design without Test-ability Fallacy

The Square-Peg/Round-Hole Fallacy occurs when an existing software that was created in a non-agile ecosystem is force-fitted into an agile world.  A similar fallacy occurs when the software architecture fails to take into consideration software life-cycle concerns, and fails to employ patterns that tend to produce a durable, resilient and flexible solution.  This fallacy can quickly doom any team to fail, and experience shows that the end result is an unruly mess.

A similar concern pertaining to software architecture relates to the all important mandate to design for test-ability.  When software is not designed for test-ability from the start, it is most challenging to achieve the level of test-ability needed to ensure product quality.  In addition, it is difficult to create the comprehensive unit test suite that is necessary to support the iterative development cycles that are typical of Agile development.  Without these comprehensive unit tests, the software quickly becomes brittle, defects begin to appear with regularity, and software releases become increasingly unreliable.  This fallacy can be realized in any software that is not designed from the ground up to support agile methods.

5) Agile obesity

Agile teams may loose sight of the principles laid out in the Agile Manifesto.  Agile obesity results from various lapses in judgment.

a) The Fallacy of "Yes" - Over-committing leading to Under-delivery

b) High Software Defects and Brittleness

c) Excessive Rework Cycles.

The  Fallacy of "Yes" is an over-committing problem that invariably leads to under-delivery and in the worst case scenario leads to unacceptable software quality. 

For example, the product owner in a scrum team may have incentives to include ever more features in a release beyond the capacity of the team to engineer, and test all of the promised features.  This is a clear case of an agile fat-cat who has lost touch with reality and has adopted the fallacy that you can get something for nothing with agile.  What's one more wafer-thin feature? 

In the real world, there is a balance of features that do not introduce unnecessary risk to a software product release. In the same vein, you may have a scrum master who doesn't know how to decline a request from the product owner.  It is vital for an agile team to grapple with the risk of adding additional features to a sprint or a release with at least equal weight to the reward of delivering said new features.  The Scrum Master must be able to have a dialogue with the Product Owner relative to any such requests and determine the tangible risks and trade-offs that apply.  This must be done objectively -- emphasizing pragmatism, resource capacity and bandwidth at all times.

6) Inadequate Regression Testing

Regression testing is a mandatory activity in agile development.  Every change -- regardless of how minute -- requires comprehensive regression testing.  Without the necessary and essential test cycles, software is sure to break and exhibit defects and unreliable behavior.  This fallacy is closely related to the Non-Agile Software Architecture Pitfall; however, this fallacy may occur even when the software is designed for testability.  This may occur in the following conditions:

1.     The QA or test team fails to invest adequately in understanding the software and it's failure modes.

2.     The engineering team fails to provide adequate hand-off of the software and its features and functionality to the test team.

Here, we are not simply talking about unit test suites.  Rather, we mean full cycle regression testing.  It is difficult to anticipate the myriad unintended consequences of a small change made to even a modestly complex piece of software.  A change to one component may result in cascading behaviors that bleed over into the function of other components.  Thorough unit testing that includes testing failure modes and edge conditions will ensure that a given module does not malfunction even when presented with edge-condition values that are otherwise unexpected and even unanticipated.  As the integration of highly modular components of software becomes the norm, the cascading effects of a failure in one module may have amplifying effects on other components of the software.  These amplifying effects seldom occur in normal operational scenarios, but software engineers in agile teams, including DevOps engineers need to plan for catastrophic cascading failures that can bring a software product to it's knees.

Conclusion

Assessing many companies that have undertaken a transition to Agile methods, it is clear that Agile is massively  to be a software engineering method, when in fact Agile is not a software engineering method at all.  Agile is a management methodology with application to software engineering and other disciplines.  Most important, because Agile is not well understood by management and also has been heavily promoted as a solution that introduces efficiencies within software engineering teams, non-technical managers see it as a master key that will magically transform a software engineering organization, with little or no cost.  This perception is a tremendous disservice to Agile methods and also to software engineering as well.  Software engineering is a highly disciplined endeavor involving the management of voluminous information and technical details.  

While some managers may perceive the exacting discipline  of software engineering as overkill, it is certainly not the case that the high level of discipiline of a software engineer is excessive by any means.  If the software that is being built is a trivial web page, then perhaps less discipline may be required, but if the software is for a complex satellite network used to defend one's country from attacks from a hostile nation, then clearly the software must work reliably at least 99.99% of the time and preferrably 100%.  If the software is for a self-driving automobile, then we must demand 100% reliability as the vehicle is transporting our most precious cargo - namely ourselves, our spouses, our children and our dear friends.

In fact the most powerful aspect of Agile methods is the concept of iterative development and iterative testing.  The software must pass unit tests under a variety of common operating scenarios as well as under uncommon conditions and edge or boundary conditions.  Software must also operate predictably, and recover gracefully, from failures when invalid and even corrupt inputs or data are passed to the software.

In short, Agile works best when employed with engineers employing repeatable processes.  It can require painstaking effort to create these repeatable processes, and most companies are not patient to allow these repeatable processes to be developed by highly compensated engineers.  Such perceptions doom Agile transformations to fail.  Since the Agile Alliance was formed to formulate solutions to the challenges experienced by software engineering teams and to emphasize the value of working software, it is a fallacy to think that the discipline that fundamentally underscores Agile methods may be tossed aside as useless.

But this does not imply that we simply replace one unsatisfactory management paradigm with another unsatisfactory software engineering management paradigm.  Rather we conclude that repeatable processes are the engine for increased velocity within software engineering teams.  By developing software with discipline and building out repeatable processes, software engineers can deliver high quality working software at velocity.  We've been doing it for years and we continue to do so.  The key to delivering high quality working software at velocity is to employ tremendous discipline and create the repeatable processes that are crucial to engineering high quality working software.  Furthermore, we've uncovered some important lessons for exactly how this should be done and we refer to this concept as the Simplicity Pattern.  The simplicity pattern builds on decades of research and refining the concept of Software Stability Method (SSM)  Software Stability Method is an approach that employs stable and reusable architectural and design patterns throughout the software engineering process.  The Simplicity Pattern is summarized as follows:

Complexity is an anti-pattern

Software engineers and managers within software organizations tend to become complacent with regard to software complexity.  This complacency is dangerous.  The more complex that a software product becomes, the more likely it is to fail.  Furthermore software failures are tremendously costly to fix after delivery. Thus we must simplify relentlessly within all aspects of software engineering encompassing: 

1) Software Architecture

2) Software Design

3) Software Development

4) Software Test

5) Software Deployment and Installation.

Thus the Simplicity Pattern dictates that software must adhere to the following simplicity principles by embodying the following concepts:

1) Architectural Simplicity Pattern 

Software Architecture must be as simple as possible while still satisfying the system requirements.

2) Design Simplicity Pattern

The Design Simplicity Pattern is best understood by two popular concepts touted within strong Agile Teams:

a) Single Responsibility Principle (SRP)  - a software module should have a single responsibility.  Software designs that adhere to SRP tend to remain simple and mitigate complexity

b) Test Driven Design (TDD)  - Software must be designed to be testable, and this is most readily achieved when employing TDD in which the tests that the software must satisfy are the key artifacts of the software design

3) Build Simplicity Pattern

In short Build Simplicity means that the process of building the software need not be a complex software engineering activity unto itself.  By simplifying the build process it makes it easier to continuously build the software and this in turn is the foundation of the continuous integration that is a cornerstone of Agile methods.  Moreover, software must be designed such that the software build is not prohibitively time consuming.  If builds take a long time, then engineers will be reluctant to build the software and defects inherent in the build process may not be identified until very late.

4) Test Simplicity Pattern

Software must be architected, designed, and developed to be fundamentally testable. When software is inherently testable, it is easy to test and testing itself is more easily realized as a repeatable and iterative process.

5) Deployment and installation Simplicity Pattern

Ultimately software is of no value unless it works reliably for its intended purpose and is installed for the end user or customer's use.  Whether the software is installed on a microprocessor within the engine compartment of a car, or within the CPU of an ATM machine. or on a satellite, the software must be easily and quickly deploy-able and install-able to the target operating system.  Software that is prohibitively complex, and difficult to deploy or install is prone to experience defects within the installation process.

In short, to succeed with Agile, or any other project management methodology, we recommend that teams adhere to the Simplicity Pattern.

Copyright © Thought Rising 2020 All rights reserved.