CO2 and Climate Change

According to The Earth Institute, CO2 is a greenhouse gas as it traps heat that would otherwise escape the earths atmosphere.  In contrast, Oxygen and Nitrogen gases do not behave the same way.

This prompts more questions than The Earth Institute would like to think about:

1) Don't humans as well as cattle emit CO2 when exhaling?

Of course we do?  So, it would seem, this means that the only way to save the planet is to wipe out a large number of humans!

Or is it?  

As we all learned in grade school. We live in a most wonderful ecosystem, in which Humans breath Oxygen and exhale Carbon Dioxide (CO2).  Our partners in the ecosystem are plants.  Plants are our equalizers.  They consume Carbon Dioxide and expel Oxygen.  

Unfortunately, the cycle has been damaged by enormous deforestation across the globe.  So, there is a simple fix to our CO2 problem:  Plant more trees!!!

Thoughts on the Art of Abstraction in Software Engineering

My esteemed colleague, Dr. Mohamed Fayad has stressed the importance of abstractions in Software Engineering.   I firmly agree with Dr. Fayaad that the Art of Abstraction is a key area of concern in the science and practice of Software Engineering.  This concern is not well addressed in any widely adopted software engineering methodology.

What I have learned over the course of my career is that for any given computing problem there is a right-sized abstraction, which substantially simplifies the design and the code required to realize the solution.  

For example, in one of my first software engineering jobs, the PhD. who owned the firm asked me to write a program using recursion to address the requirements of an interface protocol to automate a piece of factory floor equipment.  Initially I spent time to design a solution that employed backward chaining recursion, but unfortunately the interface protocol in question was far more elegantly implemented with forward-chaining iteration rather than recursion.  Could I have built a solution with Recursion?  Absolutely!  I'm a talented engineer, and I would have made it work, but at what additional cost of design, development, and maintenance over the life of the software?

The challenge was that the solution utilizing recursion was vastly more convoluted and brittle than the natural solution that utilized a simple forward looping structure.  My analysis convinced me that this was inherent to the nature of the interface specification. 

Imagine me as a young software engineer attempting to explain to my new boss that his approach was far more costly and risky than the alternative that I envisioned.  Would he be upset and perhaps fire me on the spot?  The risk was tangible and worrisome.  But at the end of the day, Courage is the Key to life itself.  as a software engineer, you must be confident in your skills and be willing to defend your work, even against very hostile opposition.  My boss listened intently to my findings and recommendations.  It was clear that there was some degree of disappointment and frustration on his face as I explained why recursion was a poor abstraction for this particular design; however, he couldn't argue with my reasoning and he concluded our conversation, by saying, "Dave, you have made a convincing case.  Please proceed as you have recommended.  I was a valuable member of that team for several years.

A software engineer must  be open-minded, and deeply understand the inherent nature of a given solution that you are trying to build with a computer program.  It is helpful to think about how the problem might be solved using a functional programming language versus a declarative programming language. or even using a rules engine or an AI platform.  Most envisioned solutions broadcast a natural design along with natural abstractions that are the most simple and sensible for the domain, as well as the underlying  requirements of a solution (both the functional and non-functional requirements).  This is the inherent nature of virtually any software engineering exercise.  

In the converse, we may contemplate why do software projects fail?  

Failures are often attributable to time and money pressures, which tend to short-shift design cycles and tend to restrict efforts to analyze the correct abstraction for the solution.  Thus engineers fail to conceptualize a proper abstraction.  because we lack the tools and methods to arrive at an exceptional abstraction to a given software engineering problem.

More often than not it's because the wrong abstraction is used - perhaps due to the biases of the engineers on the team which lead the solution away from the natural and implicit abstractions that yield a superior solution.  In many cases, it is a side-effect of the software engineering methodology that the team employs.  Fundamentally, these failures result because our profession lacks predictable and reusable tools for abstracting a solution and measuring the goodness of fit of the abstraction to the domain and the requirements.

As engineers, we dance around these topics (time, money, resources and scope) because the first three are always constrained whereas the 4th is always expected to be maximized.

Perhaps this critique of the software engineering domain is unfair, after all, software has permeated virtually every facet of our lives and we might conclude that the solutions that make our lives so much easier are proof that software projects do not fail.

I argue that this assessment is a complacent argument The plethora of software that permeates our daily lives is not proof that software projects are not failing at a very high rate.  Rather it is a testament of the size of the software engineering profession that we have such a critical mass of software engineering professionals in our world who are producing software at a phenomenal pace.  This doesnt mean that the vast majority of solutions are successful.  It just means that there is a critical mass of successful projects and we don't see the failures since they don't have commercial value, and therefore are not visible from an everyday perspective.

I have observed many hundreds of solutions in my career that simply needed to be re-engineered, because they were the artifacts of failed software engineering products.  Furthermore, I have studied dozens of solutions that were so artificially complex and cumbersome that they are simply not viable.  Yet every company in the same industry or domain uses these products because a prominent software vendor is marketing the product.  This does not mean that the project was successful, per se.  It mearly means that the vendor has a marketecture that meets a market demand; however, it is common that failures of the underlying technology are frequent and repeatable.  

Why, exactly, does this happen?  In the end analysis, it is most often due to needless complexity of the abstractions used in modeling the solution.  A complex abstraction will invariably lead to a complex design and a complex code-base.

Perhaps at no time was this more obvious to me than when building a commissioning engine.  Initially, our CTO suggested that we didn't need any fancy tools to build the solution.  He further suggested that a p-code solution (a poor-man's rules engine would do the trick) our team was skeptical about this as we would first need to construct the p-code interpreter and then apply it to our problem domain.  Therefore, we decided to invest in a rules engine from a respected product vendor.  At first we learned that many features of the rules engine technology did not work as advertised.  This caused us to throw out many assumptions at the start of the project.  

A second challenge was that the vendor's consultant to our team was not fully expert in the rules engine technology that the vendor sold us and he heavily struggled in identifying a proper abstraction to meet our requirements.  So, the original solution was rather cumbersome and difficult to maintain.  The company had invested $1.6 million to develop the solution, but regrettably, it didn't work especially well.  

Less than a year into the project I persuaded management to pursue an overhaul of the solution to make the software easier to maintain.  This iteration of our software was far more maintainable, but still required a team of 4 engineers to support the technology.  The cost of sustaining the software was still too high, as was the failure rate of the software.  

We realized that no one possesssed a deep understing of the  the features of the rules engine platform well enough to further simplify the solution.  Several production failures of the software prompted the team to invest in formal training for the engineers on the team, but no one returned from the training with any viable solutions to our woes.  

Finally, I went to the training with my mind completely focused on abstracting our solution to be even more elegant.  I quickly learned about features of our development platform that could further simplify our model and reduce the support costs by 50% and reduce the defects by more than 90%.  I shared my  recommendations and showed that the developmnent could be completed in just 3 weeks time.

Ultimately our solution became recognized as a gold-standard solution in the industry and we realized a positive cash return on investment for the solution within 2 years.

When the "Whole" is less than the sum of it's parts

Agile practitioners and advocates are prone to a fallacy that the "Whole" is equal to the sum of it's parts.  While this principle should be true in most cases, it is not always true.  

For example, lets consider two major categories of transactions in a system  Category A and Category B:

 

Category A: Domestic Purchase/Shipment		Category B: International Purchase/Shipment
Taxable?	Yes	Taxable?	Yes
Requirements	State Sales Taxes apply bsased on the location where the product is shipped. Some states have no sales tax (e.g.:  0%)	Requirements	Value Added Tax Import Tax and Tariffs as per current regulations.

 

A subsystem that orchestrates the shipment, and also the application of Taxes and Tariffs may not work correctly in  particularly in cases, when Tariffs may be assigned by a foreign government on short notice.  This is a rather simple example, but it demonstrates a form of complexity that may surface in a Scrum Team.  

If the work of the overall orchestration of the Taxation and Tariff assessment is split out among different Scrum teams, the results could be unpredictable.  Taxation rates vary from state to state and country to country.  The proper division and ownership of duties is important here.  Perhaps there is an expert on the team who understands US State sales taxes, and another expert who primarily works with international sales taxes and tariffs, but no one who understand both. In addition, there is a great need for coordination with the DevOps team who will be responsible for the build and deployment of the solution.  

Consider that certain product is no longer available for shipment from the United States to customers in the United States due to a structural change in off-shoring of the product's manufacturing.  As a result the products will ship from China to the USA; however, China has recently had new Tariffs and sanctions imposed on its imports into the USA, due to its failure to adhere to a trade agreement.  In this scenario, we have import shipments from a foreign nation to the US.  This breaks the division of labor into a less-than-ideal outcome.  As the shipments will be taxed based on the destination state, plus the shipment will also have a tarrif, which is not normally the case for imports into the USA. The Scrum Team may decide to break this piece out into a module of its own; because, over time, things may change and the product again may be sourced domestically.

Where the difficulty arises for the overall Agile team is when the teams are not clear about the division of responsibilities and overlap exists or the work is divided up in a hurry due to pressing enterprise obligations.  As a result team-members anxious to help with the effort may needlessly duplicate efforts or fail to provide the coverage for a feature that is required.  The end result is defect that are unanticipated.  Who is responsible for pruning the citrus tree of code artifacts and ensuring that this doesn't happen within agile teams?  Perhaps the scrum master owns this responsibility, or does she?  If agile teams are self-organizing, then they may not receive this coverage from the scrum master.  Moreover a question is prompted.  When was the last time that you observed a living breathing scrum-master in the wild?  They are an interesting, but rare species, and apparently nocturnal creatures, because they are rarely observed in the daytime.

There are vast examples of agile teams dropping the ball as in the scenario that I featured above.  The question is two-fold:

1) Are we so arrogant as to believe that agile is the end-all be-all of software engineering practice?

2) What is missing from Agile that seems to produce these unfortunate outcomes?

I have found specific software engineering discipline to be nearly absent in agile teams.  Many teams simply struggle to adopt the ceremonies required for agile with the level of discipline that is needed.  DevOps teams pushed to the hilt with a barrage of deployment tickets simply jury-rig the deployments just-in time to meet the insane schedules that they deal with on a daily basis.  This results in further concerns such as deployment of artifacts that were never intended to be deployed, simply because they have worked before, or adding unnecessary and volatile steps to a release as a stop-gap, but introducing production failures as a side effect, If you think that these are funny examples, I've got a million of them.

Perhaps instead of riffing on the unfortunate souls who have been hit and run by agile practice we need to shed the arrogance of agile and get back to some modicum of software engineering.

First principles of software engineering are a good place to start.  

1.1) There is a root cause for every effect or outcome.

1.2) Garbage in yields garbage out.

By analogy, we can assert the following Theorems as well:

2.1) There's no such thing as a free lunch.

2.2) Agile methods require software that is crafted to satisfy agile expectations

       2.2.1) Testable

                  2.2.1.1) Has a very high percentage of automated integration tests.

                   2.2.1.2) Code coverage through unit tests is also equally high.

                   2.2.1.3)  Every test is correlated to functionality specified in user stories.

       2.2.2) Well understood

                  2.2.2.1) There are no significant gaps in the user stories, unit tests or integration test suites.

                  2.2.2.2) The application or solution is well understood.

                  2.2.2.3) Knowledge of the solution is substantial and not limited to the minds of a small cadre of experts.

                  2.2.2.4) Experts are expected to share their knowledge with the team, and this knowledge is shared freely

Regrettably, these principles are not widely adopted in software engineering organizations, nor is knowledge shared broadly.  In fact, critical knowledge is often owned by a small fraction of the team and there are large portions of a system that are so complex that no one could ever explain how it works as it does or why it works in such a way.

In practice leading software engineers understand that there is an elegant solution for every software application and there are fully elegant techniques for reducing software complexity.  Agile teams rarely reach the maturity required to realize how to mitigate software complexity, because everyone is so focused on sprints and deadlines.

This prompts many questions.  The first is this notion of deadlines.  Weren't deadlines supposed to be a thing of the past in the agile world?  We were supposed to be killing the notion of artificial deadlines with the transition to agile, but it ultimately turns out that there are more deadlines than ever.  Daily deadlines, weekly deadlines, sprint deadlines, story deadlines and epic deadlines.  But in the agile ceremonies, no one is really managing to these deadlines, certainly not the scrum masters that eschew deadlines, and think in terms of nothing more than the current sprint and the next sprint.

 

KARMA!!!

 I worry that I might have inadvertently been responsible for the wholly self-righteous model of the left as a result of my time as a progressive activist at UCSD in the 1980s.  Imagine if you will that the totality of liberal strategies and methods was authored by a sex-and-drug-crazed twenty-something college student in the 1980s.

I was merely hoping to get laid by some loose liberal babes and get buzzed, and accidentally created Modern Democratic Socialism... Crap!

Words of Wisdom

If you died tonight, your employer will advertise to fill your job by the end of the month, but your loved ones, friends and family will miss you forever. 

Don't ever get too busy making a living, that you forget to work on making a life.

It's Only a Pop Song - A Pragmatists Analysis of "Imagine" by John Lennon.

So much of the present-day woke mentality is steeped in the brew of a Popular song by written by John Lennon, plus a rather uninformed view of the Summer of Love and the philosophy of the Hippies of the late 1960s and early 1970s.

But it is most important to understand that "Imagine" is a Pop Song, not a serious world-view.

Imagine there's no heaven
It's easy if you try
No hell below us
Above us, only sky

Commentary: Fluffy words... Heaven, Hell, sky... nothing particularly seditious here, but if you want to try communism because John Lennon said that it's easy if you try, don't believe it for a minute.  Communism and Socialism has failed miserably the world over.  It's history is marred by human rights violations that persist to this day in Russia, China, Cuba, Venezuela and everywhere that it's been attempted. Socialism is utter nonsense. My naive 16 year-old step-son asked me why we need to have money in the world?  He also wanted to know why a house isn't free.  He finds these concepts easy to imagine. Well, I explained to him that no one is going to build a nice house for you if there is no financial incentive why would they do that.  You would have to build the house yourself. You'd have to be expert in all of the trades that are necessary to build your house: structural architecture, procurement and supply chain management, planning, design, Heating, Ventilation and Air Conditioning, Plumbing, Framing, Drywall, Painting, Tile Work, and the like.  Most people in the United states live in very nice homes and apartments that would not be available if everything was free.  Furthermore, who would manufacture automobiles, build roads, highways street signs and street lights without the exchange of currency?  Finally, who is going to grow your food.  In short Capitalism enables specialization, so that one person doesn't have to do everything under the sun in order to achieve a good life.  If there is no currency, who will operate the government and what is the benefit of being a public servant?

Imagine all the people
Livin' for today
Ah

Commentary In the original recording, John Lennon sings, "Ahaaaa..."  This is an Aha moment all right! If you live only for today, then you are not striving to better yourself for tomorrow. You would likely develop into a morbidly obese person carrying 100+ pounds of fat if you don't take care of your body while you are living for today.

Imagine there's no countries
It isn't hard to do
Nothing to kill or die for
And no religion, too

Commentary: This is what Christians and Jews alike ought to fear!  "... and no religion, too" If there were no countries, then who is going to decide which side of the road that you should drive on?  who decides where the highway ends.  How would you even travel from one place to another if you wanted to visit:

1) Your cousin in the old country

2)  Your parents

3)  Your high school pals.

4)  Mount Everest.

5) Etc.

This is not to infer that killing or dying for a cause is a good thing.

Imagine all the people
Livin' life in peace
You

Commentary:  This is an inverted predicate.  You should live your life in peace!  All must strive to live your life in peace.

You may say I'm a dreamer
But I'm not the only one
I hope someday you'll join us
And the world will be as one

Commentary: "One world is enough for all of us" - with apologies to The Police (the band).  That is until the population exceeds the capacity of the planet....Furthermore, China is not interested in the "one world" mantra.  They fully intend to dominate this world and annex as much land as possible, whether it be Taiwan, the Philippine Islands, Indonesia, Japan and even Alaska (if they can get their hands on it!)

Imagine no possessions
I wonder if you can
No need for greed or hunger
A brotherhood of man

Commentary: Another plug for socialism.  I repeat.  who will build your house if there are no possessions?  Will you even own or have control over your house when you live in it? Enough said.  Imagine that there is no house.  No streets, highways, bridges, no clothes, no food.  We'd be living life just like our Cro-Magnon ancestors. In caves.  Everyone would be hungry.  As for "A brotherhood of man" - This is utterly chauvanistic!  Why not a sisterhood?

Imagine all the people
Sharing all the world
You...

Commentary: Well, if we have no houses, no possessions, no food, we would indeed be sharing all the world, but much like Cro-Magnon did!

You may say I'm a dreamer
But I'm not the only one
I hope someday you'll join us
And the world will live as one

Commentary:  ... And the world will live as one... what? One society... no.  One anarchy...Precisely!

The history of nations is that we engaged in discourse about the purpose of society and social change.  The problem that we have today is a minority of the people are dictating what changes will be implemented, and I don't care for this nonsense.  People less educated than I are dictating what is right and what is wrong.  That is a problem... a big problem, because I don't consider myself exceptionally well educated.

The insanity of Marxists in Americal

 

Here's an hysterical post from one of my Marxist, former high school classmates. I thought to break it down. 

The following explains why this is utterly pie-in the-sky:

1) This Marxist propaganda piece commences, "No one wants to work anymore" As if this is a wholly new idea?  Unreal! These are the musings of a 16 year old, not of a grown adult.

2) It Continues:

"Nobody ever wanted to work. We wanted to be productive, be creative, be part of a community, be supported, be validated, and have the time and space to truly rest. Nobody actually wants to trade in hours of their life to "earn" necessities."  Perhaps not, but we wanted to eat and make a comfortable life and this also requires communities and cooperation.  The free market with some assistance from the church aided villages in achieving improvements in the quality of life.

This quote reflects a naïve lack of understanding of sociology and of the formation of social groups from the early and ancient beginnings of human societies and through the present, not to mention a tremendous ignorance of the literature of social science and the notion of the Social Contract as written about by Jean-Jacques Rousseau.

Why did humans form social groups in the first place? Self-preservation had a lot to do with it! There was the importance of defending oneself and one's offspring from predators. Social groups formed essentially as small protection syndicates. Rules quickly followed.

It is a simple fact that I don't form a group with you to provide protection for you and get nothing in return. Each party to such partnership has a value that they receive for their contribution to the partnership. Perhaps it is a mutual protection-syndicate: You watch my back - I've got yours? 

From these rather individual social contracts, societies grew by sharing work whether it be joining a collective hunt for food, tending crops, gathering nuts and berries, teaching children the skills that the community had honed over many years, etc.

Further on down the road, people learned that there must be some contribution by all in the community to ensure that no one was slacking in their obligations to the community. The early notions of work were established. This provided an accounting of what each member of the tribe contributed to the collective in exchange for the benefits that the collective provided. In part this was to ensure that the collective might survive in both good times and bad times. Slackers were also bad for the morale of the collective, and they would often be turned away from the collective.

In fact, it turns out that the church bell came into being as European agrarian villages evolved and a single timekeeper was needed to alert the community when it was time to tend crops and cattle   The church found an opportunity to integrate into existing social structures by building churches with a church bell to be tolled hourly throughout the day a number of tolls to alert the villagers as to the hour of the day, this allowed the community to coordinate time to commence work in the fields, and the proper time when villagers were permitted to conclude their workday. In short, human history clearly contradicts this notion that people don't want to trade their time for necessities.  Further on down the line factories had a whistle blow which signified the time to start the work day.  Similarly, schools have a buzzer to indicate the start of the school day and the start of each class period.

Rousseau's social contract was an effort to maximize the notion of human freedom. In doing so, he arrived at the recognition that people do sacrifice some independence and freedom in exchange for the benefits they realize from the society, and furthermore, in this desired social contract, everyone will be free because they all forfeit some rights and impose the some duties in exchange for some privileges. Rousseau argues that it is absurd for a man to surrender his freedom for slavery, and this is quite true. The challenge with Marxism is that far too much is surrendered to the collective and only the elitists of the collective flourish.

The notion that we don't want to exchange our time for necessities is totally ignorant. Long before any of us were born, brilliant people grappled with this very question, and concluded that indeed we do enter into this social contract willingly, specifically for the benefits that we all realize from the social contract.

Another question that deserves some mindshare is what exactly are necessities? Clearly Marxists will pursue a strategy of maximizing the list of items that are necessities. Society provides for the necessities of protection, safety and security, without which there is no foundation for the other things that we may perceive as necessities. Eastern philosophies have a simple measure of necessities:

- Earth

- Sun

- Shelter 

- Water 

- Food

Tell me what is this Marxist definition of necessities, and what exactly are we content to trade our time for? How exactly do we distinguish necessities from luxuries? Ultimately, if no one is willing to work for necessities, how, pray tell do the rest of us obtain the necessities of life? Imagine, no Toilet paper, no bread, no salt, no sugar, no butter and no meat, not to mention, no vitamins, no kitchenware and no running water. If you think this is fiction, just go to one of several thousand cities and towns throughout the globe where running water is considered a luxury.  A few of these are as follows:

1) The Northeast ghetto of Bangkok, Thailand,

2) Tolyatti in the Russian Federation Province of Samara.

3) Many parts of the Republic of the Philippines.

4) Many parts of India and Malaysia

5) Many places in Africa, including: Ethiopia, Nigeria and Ghana, Somalia, Congo and Rwanda.

There are parts of the planet where prostitution remains the only reliable source of income for a woman.  I feel that this is an undesirable outcome, but a common outcome in socialist countries.

So, I repeat, where do we draw the line between what is a necessity and what is a little more. Clearly progressives in the USA have designated medical care as a necessity. While I do not reject that medical care is a necessity in the present day, I simply don't understand who determines what is medically necessary and what is not in a socialized medicine model.

That's just for starters. Compound the question by taking away the incentives that people have to pursue medicine as a profession and what are you left with? Fewer options for medical care and a lower quality of care. What is the incentive to work as a nurse, a janitor or an executive of a hospital if we've socialized all medicine and all necessities? Quick answer is that there is no incentive for doing this work!