Q6 - Israel Gat

Q6: What should question 6 be?  And what's it's answer?

I would suggest the following question: Can one apply Agile beyond software?

While I have not really done any systemic primary research on the subject, my hunch is that the answer is a resounding “Yes!” I offer three data points to support my hypothesis, as follows:

1. Jim Highsmith has been doing Agile with an architecture firm in Italy.
2. Paul Beavers and Becky Strauss are currently working on introducing Agile in BMC’s Customer Support organization.
3. My own experience with my staff.

Of these three data points, I will elaborate on the deep transformation we went through as the staff of BMC’s Distributed System Management business unit for the simple reason that it offers special insight on the nature of Agile principles.

As mentioned in previous postings, we started doing Scrum in 2004. By 2006 we had mastered many of the aspects of Scrum. Furthermore, the business unit was “Scrummed” in entirety at this point in time.

During this year (2006) a product line was moved into my business unit. While the leaders of this product line were generally competent, their assimilation in the business unit was not working. It was clear they were struggling. We, as staff, were struggling to figure out why they were struggling…

The answer to the question why they were struggling eluded us for quite sometime. Examination of the “usual suspects” – strategy, market, execution, structure and time (for integration/assimilation) yielded no insight. There existed, of course, all kind of very legitimate differences in thinking and approach between the old hands and the new product line folks, but nothing to write home about, nothing that could shed light why the assimilation was so problematic.

Sean Duclaux, who was the director of product management for the business unit, came up with the insight we needed. Sean grasped that doing Agile transformed me and my staff with respect to our mode of operation. Unbeknown to us we became so-very-agile in the way we functioned. Actions were taken quickly; decisions were postponed to the latest responsible time; something we were working on in a certain staff meeting would be dropped in favor of a more important item in the next meeting; etc. None of these was conscious – we did not set ourselves to run staff using Agile principles. These things were just happening.

Checking with the struggling product line team, Sean’s hypothesis was validated. From their perspective we were an unbelievably chaotic bunch. We were not even managing an “organized chaos” – in their eyes we were increasing chaos. They found it next to impossible to operate within the fluidly dynamic environment that de facto prevailed in my staff.

The problem actually proved very hard to solve. While we had any number of staff members and staff members of staff members who could immediately help with just about any implementation aspect of Scrum, we really had no expertise on the transformation in our own functioning as decision makers. I am out of my depth here, but I tend to think we had gone through a prolonged evolution period in which Agile principles, in the most general sense, sank into us and became part of our standard operating procedures in any domain we focused on. This evolution period was missing altogether for the new folks on my staff. Expecting them to adapt quickly, we were sort of implicitly asking them to skip a developmental phase that one has to go through whether one is doing Agile in the R&D lab or performing in an agile manner in staff.

I find it fascinating that we had not realized Agile was transforming us as staff while we were transforming software engineering practices at BMC through Agile. There is an intriguing symmetry here that speaks straight to the heart of the software craftsman that I am.

[Clarke: My thanks to Israel.  If you have a question for Israel then leave a comment and I'll pass it on.]

Q5 - Israel Gat

q5) Has the conversion stuck and what are you doing to keep it sticky?

 

The conversion has more than stuck. In terms of scope of the practice, it flourished to the level of close to 1,000 Scrum users/seats at BMC nowadays. In terms of quantifiable accomplishments, the joint SQMA/Cutter Consortium/ BMC study has concluded:

"Nearly three times faster time to market than industry average; 20-50% improvement in individual team productivity; and, one quarter the expected number of defects based on team sizes and schedules...”

Furthermore:

“In their 20 years of benchmarking software development projects, the QSM Associates team has not seen such a high performing team [BMC’s] distributed across such a broad geography."

Suffice it to say I believe transforming BMC to achieve this level of excellence in Agile/Scum is probably the greatest accomplishment of my whole career.

To reach even higher levels of effectiveness and efficiency in software development, I believe we need to focus on four elements: methodical change, system development,operational innovation and organizational adaptation.

The initiatives in these four areas I am promoting/pushing these days are as follows:

1. Methodical change: We are more and more moving these towards Lean Agile.
2. System development: Agile, to me, is not “just” a R&D “thing”. Rather, one reaps much greater benefits by end-to-end Agile, instilling it all the way from the R&D lab to the customer shop. For example, Innovation Game® from Enthiosys is a tool I am considering using to refine our Agile/Scrum system and process.
3. Operational Innovation: My recent work on Agile-Based-Market-Of-One (ABMOO) is a good example (even if I have to say so…) of innovative exploitation of the methodology. The power of ABMOO is in transforming the business design to fully capitalize on the methodical capabilities.
4. Organizational adaptation: With so much programming work carried out through outsourcing, the boundary between where the corporation ends and the outsourcer begins becomes very fuzzy. My contention is that the next big frontier for Agile is in developing joint Agile infrastructure which will be used by both the corporation and its strategic outsourcers.

It is a little premature to assess which of the initiatives listed above will really take off big time. Until we know the answer, I am so very happy to be part of the Agile drama. This is not a drama we passively watch in the theatre. Rather, you, me, the developer in the cubicle adjacent to yours, the tester next to him, etc. are active players in this drama. What else could one ask for?!

Q4 - Israel Gat

q4) When did you start seeing results?

I will have to give a complex answer. Some results were discernible very quickly. Others took quite a long time to materialize.

Three discernible results became very clear within three months:

1. Our testers became much more knowledgeable on the product(s) almost overnight. Working “real time” with product management and development from the very early stages of product conception, design and implementation immensely enriched their understanding, experience and knowledge.

2. It soon felt “As if a new sun had arisen”¹. Various things were not working well for the business unit before we started Scrum, and numerous team members were quite worried. The introduction of Scrum shifted the mindset from “The world hates us” to “Gee, this stuff [Scrum] is cool!”

3. In three months we shipped the first Scrum release. It was not a great release, but it was working code at the hands of our customers. Walter Bodwell, the director in charge of the BMC Performance Manager (BPM) summarized the effect very astutely:

“If we used Waterfall on BPM, we would still be in development. We would likely be cutting features right and left to try to bring the date back in. Changes requested along the way by the solutions teams would have been pushed back on rather than embraced.”

In contrast, it really took us one to two years to master various important aspects of Agile. Examples are uniformity in story cards, accurate assessment of velocity and effective Scrum of Scrums. All in all, I would characterize the phases of our methodical progress as follows:

1. First year – we were learning how to do Scrum on a fairly large scale basis.

2. Second year – we were starting to be proficient in Scrum.

3. Third year – we rocked.

Various practitioners with whom I discussed our Scrum evolution felt that the three phase learning curve described above is too slow for executive management to buy into Agile. As this observation indeed is quite true sometimes, whenever I discuss Agile with executives who seem to have a very short time focus, I add a stern warning:

“Don’t agile the Agile!”

1 This phrase had originally been coined to describe the reign of Edward III (1327-1377)  in England.

Q3 - Israel Gat

q3) What were the (say) 3 biggest obstacles to conversion and how did you overcome them?

1. Is Agile another good-for-nothing management consulting fad? We did not start from square zero trying to implement Agile. Rather, having been exposed in the past to various methodical and management consulting experiments which were carried out half-heartedly, people in the trenches wondered at the beginning whether Agile was another one of “this too shall pass” initiatives. Will Israel and his staff walk the talk was an open question when we started rolling.

The defining moment for our sincerity and commitment happened at the conclusion of one of the release planning days. Paul Beaver, the senior director in charge of PATROL and the BMC Performance Manager, asked for a thumb-of-five vote¹. One of the teams came back with an average of 2 as their confidence level. Upon enquiring, the team members sort of said “Well, we do not really believe we can deliver, but we thought this is the plan management wanted…”

Paul, to his great credit, sent the team back to the drawing board, held another release planning day at a later date, and ultimately accepted a revised plan for which the team confidence level was 3.5. Since this episode, nobody ever doubted our total commitment to Agile.

2. Always-releasable: I can’t convey in words how difficult this intuitively simple principle was to implement. There is nothing extraordinary that I can point out to as the reason for always-releasable being so difficult to attain. The grand total of “infinite” number of trivial deltas/incompatibilities between components of the product suite, their containers (i.e. the SCM’s), and the engineering practices across five development sites proved to be almost insurmountable. We overcame the challenge by simply putting in an incredible amount of hard and tedious work, eliminating the obstacles one by one over a period of time that felt like ages.

Once we reached the always- releasable state, the improvement in productivity, velocity, quality and predictability was dramatic. The code became reasonably stable on an on-going basis rather than rarely stable till the very end of the release. The effects of various bugs in our code were not accumulated nor compounded. Rather, most of the bugs got knocked out “real time.”

While in general I believe Agile practices should be reasonably flexible – i.e. tailor the practices to the situation at hand – always-releasable is for me a non-negotiable.

I feel so strongly about it I would go as far as saying one does not really practice Agile until the always-releasable state has been achieved.

3. When the magic fails to work: Towards the delivery of a major release I had a layover in NYC on the way back from Tel Aviv, Israel to Austin. TX. I called my good friend Roy Ritthaler who was in charge of product management for the business unit. To my friendly question “How are you doing, Roy?” I got the authentic, maybe too authentic, answer “Better than you, Israel”…. Turned out the release was in fairly deep sneakers. The number one reason for the difficult situation we were facing was me being overly aggressive on the release, betting that our velocity would rise to the level required to produce a very complicated suite of products. The fact of the matter was our velocity at this point in time was not yet high enough (thought it became incredibly fast later). I lost in the critical race between Scrum methodical progress and the need to introduce a “big bang” product suite in our market.

The crisis around the release being in trouble revealed I had not done a good enough job educating Marketing, Sales, Software Consultants and Customer Support on the nature of Agile. Things that are standard operating procedure for Agile in R&D - e.g. ship the features you can and catch up on the features you missed in the next release – became highly controversial. In particular, the spirit of Agile, as distinct from the mechanics, was extremely difficult to explain amidst the crisis.

Bill Miller, who was my boss at the time, saved the day. He grasped that the fundamental problem had nothing to do with Agile – we would have faced the very same crisis had we been playing Tin-ker-toy instead of Scrumming. The furor we were facing was about our go-to-market machine being thrown out of balance as a result of the release problems. The challenge was to align R&D with the business, not about Agile.

Painful that the crisis was at the time, it was the trigger for our transition from Waterfall optimized organization to Agile organization. This transition eventually led to our pioneering work in the area of Agile-Based-Market-Of-One. While it certainly did not feel so at the time, the release crisis actually was a blessing in disguise – it forced us to thoroughly revisi our approach to Agile. We ultimately gained much deeper mastery of the subtle issues of end-to-end Agile within the corporate fabric.

 

1 A 1-5 confidence voting scale based on the number of fingers one displays: 1=this project is heading towards total disaster and I am updating my resume as soon as the meeting ends; 5=we will deliver even if a hydrogen bomb explodes in 10431 Morado Circle, Austin, TX (one of the BMC site where we were Scrumming).

 

Q2 - Israel Gat

q2) Can you describe the scale of your Agile conversion, the time frame and how you went about it?

By the end of the first year (2005) of Agile operations we had 350 product managers, testers and developers furiously “Scrumming”. Today (summer 2008) BMC has close to 1000 Scrum users/seats. Details of growth as function of time during the first year are given in the chart at the bottom of this answer.  [clarke: click the image below to get the full size view]

Three “ingredients” were at the heart of our Agile/Scrum implementation:

1. Intentionality: We were crystal clear about converting to Agile no matter what. Becky Strauss, who managed the Scrum assimilation program, summarized it so very eloquently, as follows: 

“There has never been a thought towards returning to Waterfall – we only think about how to be more agile – how to do this better.  No one wants to go back!”

2. Trust: We fully trusted the teams to do the right thing(s) within the boundaries of the methodical rigor prescribed by the Rally Software consultants. The fundamental tenet we held was very simple: we hired the team members for their creativity and motivation; both go down the drain without trust. 

Our trust was amply rewarded in multiple ways: A) it was reciprocated - we, as the management team - were trusted by employees in the trenches; B) the trust instilled confidence in the teams; and, C) the trust, compounded by the first few visible accomplishments, built enormous pride.

3. Risk taking: Doing Agile is like learning to ride a bike – you had better be ready to fall, fall and fall again until you master the art. The risks of falling are nicely compensated by the velocity – on the bike and in doing Agile - one ultimately attains. The important thing is to be able to learn the lessons from calculated risks that were not well calculated.

My philosophy of risk-taking in the Agile context is deeply rooted in my view of the art of programming being craftsmanship that one best learns through apprenticeship. An apprentice does not become a good craftsman, let alone a great craftsman, by minimizing risks. Rather. It is the willingness to accept failure as the sometimes inevitable twin brother of aspiring to always do better that makes a craftsman.

To my way of thinking, the three elements listed above form a cohesive philosophy of Agile transformation:

• Trust is a derivative of intentionality. One cannot transform an organization single handedly. Trust “recruits” partners for carrying out the transformation.
• Unless one is willing to takes risks, many of the benefits of trust are lost. Trust leads to empowerment; and, empowerment and risk-taking are inextricably linked. One needs to manage empowerment and risk-taking in tandem as the foundation on which successful Agile projects evolve.

I will discuss various details of BMC’s Agile transformation in my forthcoming answers. However, if you are starting an Agile program today, you do not need to burden yourself with the myriads of details that need to be figured out along the way. If you genuinely adhere to them, the three principles discussed here – intentionality, trust and risk-taking – will get your Agile program going.

Q1 - Israel Gat - BMC

Q1. Hi Israel, we first chatted about 3 years ago (I think) when I helped shepherd your agile2006 experience report. I remember being very impressed at the time with the approach you took to converting to agile. Could you tell us a little about your team, about your team's pre-agile life, and what lead up to your decision to embrace agile?

When I joined BMC Software in summer 2004, I was not too certain about the “Other duties as assigned” line item in my job description. Within a few short weeks it became very clear: “other duties” meant doing quick psychotherapy for the endless stream of strangers who came to my office to complain about the software I was responsible for. I did not need to ask the classic question “How did you feel about the software bug?!” – I was proactively advised how the person calling upon me – every person! - felt about it... Some actually reverted to Hebrew (my native tongue) in order to make doubly certain I did not miss any nuance of their disappointment, dismay, despair, anger and anguish. A distinguished industry analyst actually described my challenge as “Israel, you need to cross a chasm of Biblical proportions”…

The view of my role as a resident psychoanalyst was soon dwarfed by an acute feeling of cognitive dissonance. On the one hand, everyone and his grandmother were complaining about software problems. On the other hand, my staff members, and their staffs, and many other individuals whom I met in the business unit seemed quite competent and dedicated. I could not reconcile the gap between good developers, testers and product management professionals on the one hand and unsatisfactory results on the other hand. More and more I felt like one foot in cold water, the other foot in hot water.

After scratching my head for a period of time, I reached a very simple conclusion: good people who do not achieve satisfactory results must be using the wrong methodology and/or lacking adequate tools. A quick investigation revealed the business unit was choking in rigid Waterfall practices. I found the diagnostic clue I needed – severe case of chronic Waterfall addiction punctuated by acute outbursts around release dates. My true mission, I concluded, was to turn the guys into recovered Waterfallics. My initial impression of my role as a resident psychoanalyst turned out after all to be quite accurate…

Vetting the radical idea of Agile/Scrum turned out - much to my surprise - to be a piece of cake. I think there were four main reasons for the ease with which Scrum was wholeheartedly adopted by the business unit, as follows:

   

1.

      I was reporting to an exceptional executive and human being – Mary Smars. While Mary was not an expert in Scrum, she possessed this rare combination of pragmatic common sense, wisdom of life, patience and ability to trust people in a deep manner. To this very day I believe these are the four requisite virtues to look for in an Agile executive.

2.

      I was blessed to have numerous competent colleagues who rallied around Agile. Walter Bodwell, Becky Strauss, Paul Beavers, Michael Cote, Roy Ritthaler, Glenn Jones, Chet Henry, Igor Bergman, Mike Lunt, Melody Locke and many others enabled effective diffusion of Scrum through the organization. They ensured the business unit was entirety galvanized around Scrum, had the required know-how at each and every level and operated cohesively across multiple sites.    

3.

      We invested a lot in top notch consulting. The Rally Software A team - Dean Leffingwell, Ryan Martens, Jean Tabaka, Hubert Smith and Michele Sliger - spent a ton of time with our teams, ensuring we do Scrum right from the very beginning.

4.

      We were willing and able to learn and improve every step of the way. I am still not 100% certain how we so naturally evolved to be a learning organization. My hunch is that unbeknown to us we all shared the philosophy so eloquently expressed by Alexander Pope:

 

“Be sure yourself and your own reach to know,

How far your genius, taste and learning go;
Launch not beyond your depth; but be discreet,
And mark that point where sense and dullness meet.”

A couple of months after we started Scrumming, I met with one of the teams to brainstorm about their Scrum experience. I was most pleasantly surprised how well things were working for them and expressed my deep appreciation to the team. One of the developers said “Israel, you do not understand – we have been dreaming about doing Agile!” To my naïve question “What stood in your way of so doing?” I got the real life answer “Your predecessor would not let us…”

A Social Contract for Agile

Here is an added bonus from Israel:

Perhaps the most difficult aspect of my leading Agile projects over the past five years has been dealing with the broken social contract. With globalization, off-shoring, outsourcing and reduction in the workforce being so pervasive, the social contract between employees and corporations in the software industry has for most practical purposes broken. The implied agreements by which employees form productive teams and maintain cohesive organizational order have at the very least been shaken, and very possibly voided altogether.

The voiding of the social contract typically manifests itself in the refusal – overt or covert - by employees to collaborate in their own demise. Knowledge transfer to an “assassin” in a remote country – i.e., to a person who will displace me after I train him – is the way employees often describe their reaction. Collaboration and empowerment go down the drain in the absence of an effective antidote to the “assassin” syndrome. Needless to say, trying to do Agile without collaboration and empowerment is like trying to drive your car without gas and wheels.

Conversely, an employee could and should expect an employer to pay close attention to the economical realities of business, including the cost of labor. This consideration is particularly true for the software industry where labor cost is often the #1 component in the cost structure. With software becoming more and more pervasive – in car safety bags, avionics, cell phones, etc. – software labor cost is fast becoming an issue for any industry that embeds software in its products.

The pity of it all is that nobody benefits from a broken social contract. Employers often stare at disappointing results from off-shoring and outsourcing as well as unsatisfactory outcomes from software initiatives that span multiple countries. Employees are frequently uprooted, either in terms of their sense of security/belonging/commitment; or, in terms of the actual dislocation that often accompanies a new job; or both.  The simple fact that both employers and employees are driven by very similar Darwinian rules of economics is of little consolation to either party in the face of mutual failure(s).

While Agile most certainly is not a panacea, it actually provides the opportunity to resurrect the social contact in the corporate setting in the following two interrelated levels:

• Enhance productivity big time. Results reported by Sutherland, Mah, and me show that productivity can be increased to the point that it compensates for the differential in labor cost from one country/continent to another.
• Provide safety net for employees through the development of marketable Agile skills.

If I were to start a major new Agile project today, I would deliver something similar to the following message:

“Team, my overarching organizational objective is to preserve our team and its institutional knowledge for our corporation and its customers for years to come. We will achieve this goal by enhancing our productivity to the level that it offsets disparities in the cost of labor across geographies. I wholeheartedly believe the current state of the Agile art would enable us to reach this level of excellence. In the event that we fail to reach such a high bar and our assignments fade away, you will find yourselves in demand in the market, as in the course of doing the project you will become competent Agile practitioners. I am committed to your professional development, regardless of the outcome. A significant amount of your time will be spent in Agile training, coaching, and retrospectives.”

Saying so and walking the talk might not constitute a full-fledge social contract in the classical sense of the term. However, I believe it could be quite effective as “contracting in the small,” i.e., compensating for mega issues in the software industry by way of redefining the contract at the Agile project/business unit level. I tend to think about it as “mini social contract” – i.e., a contract between the team(s) and me.

This “mini social contract” provides the critical third ingredient in the “secret sauce” that I plan to henceforth use to drive Agile projects, as follows:

• Empower them – everyone loves to have the opportunity to make an impact.
• Excite them – genuinely executed Agile is a great antidote to the weariness, bitterness, and cynicism that a broken social contract generates.
• Invest in them – hone their Agile skills in a way which (should the need arise) will transcend organizational/corporate boundaries.

While the approach I advocate might fly in the teeth of some prevailing wisdom, practices and policies, I believe it is mutually beneficial for both employers and employees. Neither employer nor employee can single-handedly change mega trends in the industry; both can, however, learn to live and evolve with these trends in a way that leaves both as whole as is realistically feasible. The simple sentence “Whether you will or will not be with the company in the future,  I acknowledge your need to develop professionally as an Agile practitioner” is all it takes to start reconstituting the social contract.

Israel Gat - One of Us

Years ago, during Bill Clinton’s campaign for the presidency, I watched a town hall meeting that Bill and Hilary held somewhere in the Midwest. Question from the participants indicated that they were concerned about the state of the economy and its impact on the standard of living in rural America. After some references to various real estate investments and stock transactions from which the Clintons had benefited, a participant in the meeting asked, “But are you really one of us[as you claim in your campaign messages]?!”

While I don’t remember the Clintons’ response, I remember the question as the defining moment of this town hall meeting. Instead of arguing whether GDP growth could be expected to be 2.1% or 2.2%, the person who asked the question went straight to the fundamental question of who the Clintons were in their heart of hearts. The piercing power of the question reminded me of Richard Nixon’s vicious quip about his opponent, Helen Graham Douglas, during his bid for the Senate in the early 1950s: “Pink right down to her underwear.” Competent and accomplished though Congresswoman Douglas might have been, during the Cold War with Russia, most US citizens were reluctant to cast a vote in favor of a candidate who might harbor pro-Socialist or pro-Communist leanings. During the weeks following Nixon’s accusation, California voters did not come to terms with the real Congresswoman Douglas. The rest, as they say, is history: Nixon won the campaign and became California’s Senator-elect.

I mention these two political anecdotes because of my strongly-held conviction that Agile leadership must be absolutely genuine. As an Agile leader, you must be of the same DNA (so to speak) as the developers, testers, writers, and product managers in your Agile/Scrum team, and identify with them completely in terms of both the talk and the walk. While they might not explicitly ask, “Are you really one of us?!” the question can linger like the proverbial elephant in the family room. For example, I sincerely doubt that you can achieve great success as an Agile leader if your passion is revenue recognition. You might be able to make various hard decisions in a dispassionate manner, but can you “touch” your Agile buffs (and those who are still sitting on the fence about adopting Agile) in a deep manner? Can you galvanize them around Agile as a very cool methodology?

You might be asking, “What on earth is in the DNA of software developers?” I believe the simple answer has been given in the Cluetrain Manifesto: the passion and pride of the craftsman in his/her work. If you consider your software to be an extension of yourself--worrying about each bug as if it were a potentially malignant wart on your nose--you probably possess the right DNA for Agile leadership. On the other hand, if you do software en passant, and do not take insane pride in the software that you produce, you are unlikely--IMHO--to make a great Agile leader. Because of your skills, experience, professionalism, and dedication, you might do a credible job, but I doubt that you will be able to electrify the Agile team to the level of hyper-productivity.

I do not pretend that I can measure your software passion and pride – these are for you to assess. Rather, my simple suggestion is as follows: before you start leading an Agile project, take a good introspective look in the mirror and come to terms with the real you – craftsman or no craftsman?! Agile can easily put you in a situation in which you will curse yourself for ever embarking on an Agile “adventure.” Do your self-test before you start, not during the course of a disastrous iteration when you start suspecting that Agile had been conceived by people who were “inhaling”

I am not sure how one administers a self-test for passion and pride. However, I will share a self test that I experienced after the time-to-market, productivity, and quality results for the BMC Performance Manager releases were published by QSM Associates. In preparing for an all-employees meeting of my business unit I wondered whether I should conclude the meeting with the following words:

Between the quarterly results and the QSM Associates study reside the 386 employees of this business unit. To each of you, I say, “this great quarter is yours alone!”

After wrestling with this question for quite some time, I ultimately used these words in the meeting (and never regretted it) for a very simple reason: they precisely expressed what I felt in my heart. I had no doubt that I really was one of usand would be viewed in this light by the business unit employees.

Other values, skills, and virtues are vital to great Agile leadership – see for example Jim Highsmith’s excellent presentation on the subject in the Agile 2006 conference. However, I would contend that the “One of Us” requirement in the sense defined above is absolutely necessary. Please remember – you can’t fake authenticity.

Israel Gat - Over the Top

Like every young man in Israel, when I reached the age of 18, I was drafted into the army, and started adjusting to the army way of everyday life. While army life posed many challenges, it soon became very clear and that I faced a single all-consuming question: how do you go over the top at the face of enemy fire? The question actually was two-tiered:

• How do I get myself to go over the top?
• How do I get other soldiers to go over the top?

Of course, physical fitness, small arms handling, and tactical skills were critical. But, they were of little value unless you could break out of the shock and paralysis caused by incoming enemy fire.

While the answer to the question of how one gets over the top might seem complex, the key ingredient turned out to be camaraderie. We were comrades-in-arms. Our bonds enabled us to rise above the instinctive self-centric and self-preservation view of a battle situation and elevated our concerns to the team (squadron, platoon, company) level, enabling us to concentrate on the tactical task at hand. The looming presence of death, particularly during the 1967 and 1973 wars, was omnipotent, but camaraderie was our antidote.

I recently received a lovely email from a former employee of mine, which connected the dots for me from the 1973 war in the Middle East to 2008 Agile in the US, as follows:

 

“…I want to thank you for pushing DSM into the Agile waters "feet first". Working with the agile teams at BMC those last two years were some of the most rewarding in my career because of the sense of purpose, direction and camaraderie that the process instilled in everyone.”

 

The most fascinating point to me in this email was the process -> camaraderie directionality. I had tried hard, perhaps too hard at times, to instill camaraderie as the means to improve our Agile process. As the email attests, the process actually fostered camaraderie among the team members.

I can imagine that camaraderie was instilling process that was instilling camaraderie that was instilling process, and so on, in a virtuous cycle that over time became extraordinarily effective for us. If you accept this premise, you must then ask yourself, “Where does one start in order to trigger a virtuous cycle in Agile?” My hunch is people over process is a good place to start.

Israel Gat - Cover to Cover

I have been reading James Anderson Winn’s masterpiece, “The Poetry of War,” during my recent vacation. Regardless of whether or not you like poetry or war, this book is a must-read. Why the strong recommendation? I have never read anything that can be applied to the roots of programmer behavior in Agile context as well as this book does. Winn might not know what Agile or Scrum or XP means, but standing on the shoulders of giants like Siegfried Sassoon, Wilfred Owen and Robert Graves, he penetrates the psychology of motivation in an exceptionally deep manner. If you are asking yourself “What makes them tick?” – a question that Paul Beavers of BMC addressed in his blog entry – you owe it to yourself to pick up Winn’s book and read it.

In analyzing the poem, “To Lucasta, On Going to the Wars – For the Fourth Time” by Robert Graves, Winn makes the following observation:

“Graves dismisses courage, fear, love, anger, and hate as motives, claiming that the soldier fights only because of his pride”.

 

Readers of this blog who read my post, One of Us, might remember that I emphasized how essential the pride of the craftsman is to great Agile leadership. Borrowing the eloquence of Winn, I would dare say something like the following:

“IMHO economical rewards and fear of failure are secondary factors in Agile leadership. A great Agilist leads primarily through taking great pride in his/her work”.

 

Fascinatingly enough, Winn illuminates numerous motives (in addition to those cited above) that are quite applicable to debates currently taking place in the Agile community. For example, his insightful discussion of Honour as an inexhaustible treasury versus honour being finite, applies nicely to a major theme presented and discussed in the Agile 2008 conference - individual rewards versus team rewards.

I don’t doubt that various methodical aspects of Agile – e.g. Planning Poker and/or Continuous Integration – are critical for success. There exists, however, a deeper, human truth to Agile that is both simple and elusive. It is the simultaneous application of rigorous methodical practices together with the touch of the pride that makes the Agile leader One of Us.

If you are interested in Agile, I encourage you to read “The Poetry of War,” cover to cover.

Israel Gat - The Equipoise of Agile

August 31, 2008

Over the past couple of years I have tried very hard to distill the BMC success in Agile to a set of best practices that I could share with anyone who is thinking of starting an Agile project. Unfortunately, after I examined the various “secret sauces” that I concocted, presented, and discussed since 2006, I had to conclude that all of them – including my recent Post, A Social Contract for Agile, in Agile Thinkers – somewhat missed the mark. Having said that, everything stated to date in my secret sauces is true, relevant, and hopefully useful to other Agile executives. What has been missing, however, is the deeper truth that binds together the many ingredients– the equipoise of Agile.

Equipoise is the equilibrium formed by offsetting conflicting forces. The equipoise of Agile is the skill of an Agile leader to orchestrate conflicting forces and pressures in and around Agile in a manner that utilizes Agile principles, without denying the realities of the non-Agile world. It is the ability to function and get teams to function amid contrast and ambivalence that are systemic.

The systemic conflicts to which I refer are the inevitable outcome of Agile being a disruptive methodology. When you apply Agile, you are trying to align two worlds that exist in parallel. In one world is the set of core competencies, practices, procedures, policies and cultural norms that made your company successful. In the other world are the origins, life, and momentum of the Agile movement. Both worlds evolve, but at a different pace and at a distance.

A good way to think about how the two worlds affect each other is to study the chapter in Japanese history that started with Commodore Perry steaming into Tokyo Bay in 1853. Bringing gunship technology in touch with the Samurai sword technology (and culture) started the difficult process of the modernization of Japan. The modernization process has been traumatic to the degree that made (Commodore Perry’s) Black Ships a term of art in the Japanese culture--expressing the surprise and confusion induced by such two dissimilar worlds, as the Japanese world and the US world touching each other.

In the way that the people of Japan maneuvered through the duality and contrasts that followed Commodore Perry’s visits to Japan, Agilists must often strike a balance between two parallel worlds. The measurement of Agile maturity at BMC is a splendid example of striking this kind of balance. After about a year of practicing Agile, I became increasingly anxious to know how the teams were performing. At my request, Dean Leffingwell and Becky Strauss devised a self-assessment methodology for the BMC Agile teams. Effective that it was, there was nothing revolutionary about the methodology itself. What was revolutionary was our decision to treat the self-assessment as privileged data that was owned by the respective teams -- separate from the overall set of measurements that we used at BMC to evaluate projects, teams and individuals.

From a traditional management perspective, the decision to exclude these self-assessments from the official measurement system was quite problematic. We were investing a fair amount of money to convert to Agile; many forthcoming products were critically dependent on the progress we would make assimilating Agile; certain commitments to customers could be met only through good progress on the Agile learning curve. Clearly, Agile maturity was an important metric by traditional Balanced Score Card standards.

Our decision to exclude Agile self-assessments from our official measurement system stemmed from our belief the data was too important to be used as part of a system that associates accomplishments with monetary rewards. Because we were betting the business unit on Agile, we could not afford team maturity to be gamed in the usual manner in which CPM metrics tend to get gamed by everyone and his grandmother. The Agile assessment data had to be pristine in the sense that there was nothing “good” or “bad” about the attained level of maturity -- it was what it was. The teams needed to be able to use their assessments of maturity as baselines for self-induced improvement. We empowered them, in collaboration with their Rally Software consultants, to look in the “mirror” of their self-assessments and determine what to do about this Agile “wrinkle” or that Scrum “wart.” We trusted the Scrum teams to use the data to drive their Agile maturity and to own their Agility.

Managing the friction between the official corporate hierarchy and Agile structural constructs is another good example for the equipoise that an Agile leader must orchestrate. For Agile to scale at the enterprise level, the Scrum team needs to cascade--like fractal building blocks spanning multiple levels. Consequently, responsibility and accountability get vested in the network of Scrum Masters and Scrum of Scrum Masters, obsolescing in many ways some traditional job definitions, like QA Director or Senior Engineering Manager. Due to these dynamics, the Agile world at BMC created its own de-facto Scrum hierarchy that existed parallel to the formal corporate hierarchy. Corporate inertia being what it is we did not believe we could successfully consolidate the two hierarchies into one in a short period. Hence, we depended on the equipoise that we struck between the two worlds, enabling the Scrum hierarchy to be focused on the delivery of value to customers, and the traditional corporate hierarchy to be focused on the administrative aspects of “life.”

You too will need to strike the kind of balance that we at BMC struck with our measurement systems and with the corporate hierarchies. To succeed, you must teach your teams, as well as yourself, to live with duality and ambivalence without feeling torn apart inside. Do keep in mind that you are dealing with two levels of conflict that accentuate each other – the visible conflict between the two worlds, and the invisible one that is likely to evolve in your heart as to where your allegiance lies.

The Equipoise of Agile as a critical skill, which enables the Agile methodology to be effective in corporate setting in spite of conflicting belief systems, might seem a little intangible. Yet, whatever you do, maintain the equipoise. It will keep your projects going even if the multi-faceted answers that you provide to various tough questions are considered lacking by some purists.

Acknowledgement: I am indebted to Dean Leffingwell and Becky Straus who created the Agile self-assessment methodology at BMC and provided very useful comments and insights to this post. I am also most thankful to Melody Locke who helped me immensely clarifying and articulating my musings.

Israel Gat - Cutter article

If you enjoyed Israel's articles then pop over to the Cutter website where you can download:

To Release No More or To "Release" Always: Part I -- The Myth

In this Executive Update, the author explains why he considers the whole release concept a myth. Like a set of nested Russian dolls, the duality of the term "release" often masks a deeper duality. You can view the end user as the primary target of a release or you can view the release as a vehicle for market development. Philosophical as this differentiation of the release might seem, it actually brings to the forefront essential questions about the nature of software and the nature of the market.

http://www.cutter.com/offers/releasemyth.html - use promocode RELEASEMYTH

Israel Gat - The Social Contract Revisited

Macro-economics were very much on the mind of everyone I spoke with during last week’s Agile Development Practices conference and APLN Summit in Orlando, FL. While the specific financial concerns pertaining to Agile varied from one conference participant to another, the overarching theme was the inevitability of change. For some, it was the way in which Agile would be impacted. For others, the way Agile could impact the new economics, micro- and macro-, we all face.

In response to these concerns, I tweaked my “Leading the Agile Disruption” presentation. Specifically, I proposed the following variant of the Social Contract I had originally posted in Agile Thinkers:

• “Team, my overarching organizational objective is to preserve our team and its institutional knowledge for our corporation and its customers for years to come
• We will achieve this goal by enhancing our software engineering prowess to the level that the resultant benefits will outweigh the repercussions of the current financial crisis
• The state of the Agile art should enable us to attain hyper-productivity
• In the event that we fail to accomplish hyper-productivity and our assignments fade away, you will find the Agile skills you developed much in demand in the market
• Whether you will or will not be with the company in the future, I acknowledge your need to develop professionally as an Agile practitioner and commit to invest in your education/training”

Later in the day I discussed this subject with an Agilist working for a Fortune 500 company. She was kind enough to say the proposed Social Contract went straight into the bottom of her heart. Yet she was emphatic that HR and Legal would never ever allow her (or anyone else) to propose such a “contract” to the teams with which she worked.

We discussed in some length and depth why the proposed “contract” - which is basically a gentlemen’s agreement between an Agile executive and his/her teams - would be a non-starter in her company. Both of us agreed that the five points listed in the contract need to be stated. To ignore the subject would be like denying the existence of the flu.

While I can’t say I really know this Agilist well, she undoubtedly was a smart and knowledgeable lady. I was surprised that an Agilist like she would feel hesitant about addressing the subject in her company. It was obvious she considered doing so a career limiting step.

As I felt a fairly strong cognitive dissonance during and after speaking with her,
I e-discussed the subject with a few knowledgeable colleagues and asked for a second opinion on the Social Contract. Here is a summary of the reflections I got:

• Jim Highsmith, Information Architects, thinks the Social Contract actually goes beyond the current economic situation. Tenure, in Jim’s opinion, is variable. The Social Contract is for the period of time in which an employee and employer are together.
• Debra Germaine, CTPartners, agrees with Jim’s observations. She views mutuality of interests as the “cement” in the employer/employee relationship. To quote Debra: “contribution to company in parallel and in sync with increasing the dollar value of the employee.”
• Tim Miller, Rally Software, indicates Rally already took the proposed path. The key, according to Tim, is to create the culture within which employees are pursuing the goal of “creating your own reality.”  By helping folks create their own reality Tom creates loyalty to Rally. The temporal nature of the Social Contract is very clear to Tom: “Sometimes the only progress is by transferring or moving on.”
• Evangelos Simoudis, Trident Capital, suggests reinforcing the principles articulated in the Social Contract by highlighting the power of Agile as a “differentiating weapon to augment a software solution in ways that will satisfy a customer or prospect and facilitate the closing of a sale.”
• Steve Greene, Salesforce.com, believes the Agile Social Contract should be framed in optimism with a nod to the reality of the current macro-economic environment. He suggests the message “now is the time for us to invest in the company's future and your future (increased skills, Agile in particular)”.

Having received the second opinions cited above I am even more perplexed by what I heard from this Fortune 500 Agilist. Call me naïve if you want, but for the life of me I do not know how to instill trust, collaboration and empowerment in Agile teams without having authentic conversations on the subject of layoffs and doing something meaningful to counter the corrosive effects of layoffs. The Social Contract proposed above is an informal antidote anyone of us could easily apply at the team/project/business unit level. I trust many of us will do so.

Cutter Articles

Yet More from our good friend Israel:

Download your complimentary copies of these Cutter ConsortiumExecutive Updates by Israel Gat, below!

In Part I, the author explains why he considers the whole release concept a myth. Like a set of nested Russian dolls, the duality of the term "release" often masks a deeper duality. You can view the end user as the primary target of a release or you can view the release as a vehicle for market development. Philosophical as this differentiation of the release might seem, it actually brings to the forefront essential questions about the nature of software and the nature of the market.

Part II of this series suggests that software that is alive and always evolving poses unique opportunities for custom-tailoring solutions directly from Research and Development. It also describes an effective way to distribute custom-tailored solutions, and it highlights partial disintermediation that transforms the traditional software value chain.

New Blog!

Michael Cote and I joined forces to start a new blog - The Agile Execxutive: http://theagileexecutive.com/. Michael is an analyst with RedMonk - the first analyst firm built on open source. He was a key member of the team that started Scrum at BMC software in January 2005. His passion for Agile is nicely captured in the name he had chosen for his own blog - People Over Process.

Both Michael and I are of the opinion Agile methods can make a big difference during the current macro-economic crisis. We cater in this blog to the needs of the executive who realizes you can't sustainably cut costs by cutting costs. We focus on the business benefits of Agile. For example, mitigating business risk through Agile is a thread we plan to emphasize.

You don't need to be an expert in software engineering in order to enjoy this blog. Our intent is to overtime help refine your pragmatic knowledge of the benefits to be had by applying and using Agile methods. The blog will be relevant to you whether you are in Sales, Revenue Recognition, Finance, HR, Marketing, R&D, General Management or any other discipline.

We hope you will be a regular reader of The Agile Executive. And, we trust you will give us feedback that will enable us to tune the blog contents to your needs.

The best,

Israel

Israel Gat - podcast

I've just spent a fascinating hour talking with Agile Executive Israel Gat.  Click here to download the mp3 podcast where you'll hear about what Israel is up to now, the hugely impressive agile transformation at BMC (we're talking >1000 engineers), what's going on in the agile market place, amongst other equally interesting things.  Israel knows his stuff and - most importantly - he knows how to translate Agile into the language of the executive.  Great stuff, if you ignore the occassional splutter where I was holding the microphone too close to my mouth ...

• Israel's blog: http://theagileexecutive.com/

• Israel's Agile2006 BMC presentation: http://www.slideshare.net/IsraelGat/how-bmc-is-scaling-agile-development

• Here's the paper (but unless you've got membership you'll probably have to pay for it): http://www2.computer.org/portal/web/csdl/doi/10.1109/AGILE.2006.33

• Israel's Cutter page  http://www.cutter.com/meet-our-experts/gati.html