Pattern seeking or pattern making

[This piece was part of an e-mail I sent to a colleague after interviewing a candidate.]

I used to think of us humans as pattern seekers. Now I'm inclined to think of us as pattern makers.  We imprint patterns upon unconnected events. We need to. Finding method, structure and causation in everything is not only intellectually satisfying; it's an evolutionary survival skill. If most sabertooths lurk in tall savanna grass, then it's advantageous to associate the pattern of swaying blades of grass with danger. That's why abstract art works, and traffic lights ... and our intuitions about people. Most of the time.

Smiling people are mostly friendly. Frowning people are mostly unfriendly. Talkative people who monopolize conversations and say things like "those PhD candidates knew more than they think" are mostly dominant, polarizing types.

However, these pattern-forming traits of ours aren't fail-proof. We find that smiling politicians act like barbarians. We find that genuine-looking works of art are fakes. And we may find that garrulous, opinionated people are in fact quite genuine, warm, insecure types under the bluster.

As I grow old(er), I trust my intuitions less and less, and the world is an ever-more brilliant place because of it!

Retrospective Prime Directive in Urdu

I haven't found an Urdu translation of the Retrospective Prime Directive. I'm furnishing one here in the hope someone finds it useful.

Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand.

قطع نظر ان تمام باتوں سے جو ہم ابھی دریافت کریں، ہم یہ نہ صرف سمجھتے ہیں بلکہ یقین محکم سے جانتے ہیں کہ ہم میں سے ہر شخص نے اس وقت پر دستیاب معلومات، اپنی مہارات اور صلاحیات، اور موجودہ وسائل کے مطابق بہترین کام انجام دیا، جو اُس صورت حال میں کیا جا سکتا تھا۔

Privilege of different kinds

Privilege can come packaged in multiple forms. Here are a few I've experienced.

As a male software developer, I am given privilege that is undeserved. When I speak at conferences, or in front of a client, or while facilitating a workshop; I cannot help but be aware that if I were a woman, it wouldn't be as easy. For one, I would probably not have as many opportunities to speak in the first place. For another, I'd probably be taken less seriously when I did speak.

Being a man in my industry gives me undeserved privilege. And I despise that.

As a dark-skinned man with a security-alert-causing-name who was born in what, according to one news outlet, is the most dangerous country in the world, I lose privilege when I walk into an airport. It is depressing to know that in the US, even the law isn't necessarily on the side of justice and equality. It requires a much higher burden of proof to show discrimination in the civil, as opposed to statutory context. I have to accept the reality that I am subject to a different set of laws and regulations when I'm in an airport.

Laws that suspect me because of my ethnicity, name and place-of-birth take away my privilege. And I despise that.

Then there's the privilege of working with people who I call my friends. The privilege of learning from them, being able to share with them what I've learned before. The privilege of their company, the insight into their thoughts and feelings, the human connection that is the underpinning of all creativity and an indispensable prerequisite to crafting beautiful software.

Working with creative, passionate, motivated, empathic friends is a privilege that I acknowledge consciously and for which I am very grateful. And I love that.

Beware of the translated Arabic Agile Manifesto

(And not just because the world gets its hackles raised when they see "Arab" and "Manifesto" in close proximity.)

The Agile Manifesto has been translated into many languages, including a few that are written from right to left like Arabic and Urdu. If you translate one of these using Google Translate [1], you'll be misled. Because Google will translate the last sentence starting with و یعنی (AR) or یہ اسلئے (UR) literally. Which comes out to "... we give greater value to items on the right side". This is a literal translation, but it's false. Because in the original Arabic, the "items on the right side" are الأفراد وتعاملهم فيما بينهم (individuals and their mutual interactions), البرمجيات الصالحة للاستعمال (correctly working software), etc. You know, the things we all value highly.

Context is everything. If you don't pay attention, your left hand won't know what the right hand values more.

[1] Very easy to do if you view the page using Google Chrome.

Organizational Dysmorphia

In my consulting work, I've all too often observed organizations that have chosen to structure themselves in a way that is at odds with their stated goals and objectives. In extreme cases, the structure of such an organization actively subverts its raison d'être. I term such self-defeating structures as Organizational Dysmorphia.

A recurring example is when the development and quality-assurance teams are isolated from each other and evaluated on mutually inconsistent criteria. The development team is measured on the amount of functionality (feature-points, story-points, etc.) that they deliver to the QA team. The latter, in turn, are measured on the number of defects they can find in the software they receive from the development team. This arrangement actively dissolves any collective focus on quality and instead promotes a highly localized -- almost selfish -- way of thinking about one's job.

I observed another, more subtle example recently at a company. Some developers were responsible for creating interfaces. Others were responsible for implementing those interfaces. Apart from not promoting collective code ownership, this doesn't sound so bad at first. However, when you reprimand and reward (financially and otherwise) the different individuals based on exactly how the resulting software got broken, things quickly spiral down into a finger-pointing, responsibility-avoidance culture rather than one that values collective effort towards relentless quality improvement. This is what I found at that company.

Organizational Dysmorphia is consistent with and is a specialization of Conway's Law. Conway's Law is neutral: organizations are constrained to produce systems that mirror the communication structure of these organizations. Organizational Dysmorphia is the subcategory that deals with severely abnormal and dysfunctional instances of Conway's Law.

Magnanimous Writer

Martin Fowler's essay on TolerantReader was a timely read for me, since I've been dealing with both coupling with downstream services and providing services for upstream consumers at my most recent project.

It is nice to find on the other side of a TolerantReader a MagnanimousWriter. A writer that can anticipate and forgive some of the mistakes that readers that are less tolerant will inevitably commit.

One way we tried to make our writer tolerant on our project was to abide by the following contract to the upstream consumers of our RESTful services:

1. We will version our resources thus: /v/1/widgets etc.

2. We will not make any incompatible changes to any of our existing resources.

2.a. On a GET of an existing resource, we will never provide fewer entities than we do today, although we may optionally provide extra entities over time.
2.b. On a POST to an existing resource, we will never demand more required entities than we do today, although we may allow additional optional entities over time.

3. If we ever need to modify our resources in a way that we cannot live by rule #2 above, we will always create a new version of our resources: /v/2/widgets, etc.

One way we enforced this contract upon ourselves was through a test suite that validated our schemas. Here is a sample rspec test, (over)simplified:

describe "widgets view" do
  it "should match the schema" do
    orders = create_array_of_widgets_from_factory
    assigns[:widgets] = widgets  # sets "widgets" variable in the view
    render "widgets/index.xml"
    rng = File.join("/dir/to/relaxng/schemas/widgets.rng")
    response.body.should be_valid_against_rng(rng)
  end
end

The /dir/to/relaxng/schemas directory had its SVN commit access control set up in a way that only a few people (tech lead, etc.) could commit to it. This reduced the risk where a pair of devs (erroneously) changed the element "price" to "cost" in both the code and tests, committed everything, verified all builds were green ... and cause massive heartache with our upstream consumers when the app was put into production.

Of course, it is probably impossible for a writer to be magnanimous for ever. No team can support a very large number of versions of (even slightly) different versions of the same resource. On our team, we only had one version by the time I left but we seemed to be on the cusp of "v/2/something". However, it was our collective understanding that before we ever went to "v/3/anything", we would retire "v/1/everything".

Congratulations − I won the lottery and you're all screwed!

I have worked at several companies − both as a consultant and as a full time employee − where one key person had unshared knowledge about some critical aspect of the business. Meeting such people fills me with a feeling of foreboding. I wish I could tell the manager of such a person what I know to be true.

"You're one resignation away from temporary operational seizure."

It is surprising to me − although less so each time I find such a case − that more managers do not find this situation as worrisome as I do. It is not as if people haven't seen such operational seizure in reality before. It appears that the same kind of nonchalance that sanctions other reckless behavior propels managers to think "it will never happen to me; Joe will never quit on me like that".

And yet, it happens every day in organizations small and large across the land.

It could be that we deny the possibility of operational seizure because it is not a pleasant thought. We have to force ourselves to think about unsavory outcomes before we can think about solutions. Crossing that valley of doom before we may reach any safety is perhaps too depressing for many people. However, simple refusal to consider unsavory outcomes doesn't reduce the probability of those outcomes by one iota. Positive, can-do thinking is good until we have to think about risks.

Lister and DeMarco talk about risks at length in their book "Waltzing with Bears". The one example they give that has stuck with me is the kind of thinking most parents subconsciously do with regard to their children. We, as parents, fashion all sorts of negative scenarios in our heads − some of them quite fanciful and even whimsical − and then undertake plans to protect our children from the effects of such accidents. No one begrudges us such negative thinking; indeed watchful observers would think of us as neglectful parents if we were to ignore the dangers that could possibly befall our children. Yet we fail to take the same lessons when dealing with responsibilities at our workplace. We willfully neglect possible risks, including the risk of unexpected staff attrition, when planning projects or setting goals.

The core values of Extreme Programming; especially feedback and communication; are geared towards mitigating this risk. One of my professional goals is to help people realize the effectiveness of these values (and courage, simplicity, respect and others). I want to help create a culture where people are treasured because of the value they add by their presence, not the feared loss of value they would cause by their departure.