The Essence of Agile

One thing that has changed for me over the last few years is that I find myself working with people for whom agile has become the norm. Indeed, I have even begun to work with people who have never done waterfall. My frames of reference are changing; I am starting to find that comparisons with waterfall don’t carry very much weight these days.

Ten years ago, I was working as a test manager on multiple projects, some waterfall in delivery approach, some with agile aspirations that were experimenting with elements of Scrum. I wrote a piece entitled “The Essence of Agile” to see if I could get to the heart of this new way of doing things.

Here it is:

"Essence of Agile" sounds potent, doesn't it?

Since the Agile Manifesto is already a succinct summary, then its essence will surely pack a powerful punch.

It sounds like something that I might be able to sell. So, before I go any further, I'd better think up some tag lines for my marketing campaign.

"We build software. We build it fast. We build it right. That's agile."

I like that one because: It's snappy, it's bold and it delivers the message that we know what we're doing.

"We build software. We deliver it incrementally. We adjust when we need to. That's agile!"

I like that one because: It's honest, it empowers the customer and it delivers the message that we know what we're doing and are willing to discuss it.

"We build software. You may not get what you want! You may not pay what you want! But you'll get something! THAT'S agile!!"

I like that one because: honesty, humour and exclamations! Knockout!

However, building software is an exercise in engineering, not marketing, so maybe I should put some effort into the product itself. Or at least consult Wikipedia on the definition of the word “essence:”

"...essence is the attribute or set of attributes that make an object or substance what it fundamentally is, and which it has by necessity, and without which it loses its identity. Essence is contrasted with accident: a property that the object or substance has contingently, without which the substance can still retain its identity."

As a way of building and delivering software, “agile” looks to me like the application of common sense in a sequence of coordinated actions designed to deliver a pleasing result.

In the whole short history of software development, we may well consider that all methodologies have been attempts to apply common sense in a sequence of coordinated actions designed to deliver a pleasing result. So, it may be argued that agile methods are simply the natural outcome of smart people working together to build useful software at a reasonable cost.

If we were to pay someone to estimate the dollar cost to businesses and governments of failed software projects, their fee alone would make us yelp.

So how loud would we be yelping if and when we knew the dollar cost of failure?

Maybe we could estimate it using the “software project under-estimation” method: Think of a large number. Now double it.

This is to say nothing of the impact to human lives, societies and communities of failed software projects.

The attributes that make agile fundamentally what it is are: "common sense" and "coordinated action."

This is in contrast to pure common sense which can be suggested, mooted, opined, considered and even written down without ever being translated into a verifiably actioned result (i.e. saleable code).

This is also in contrast to pure action whereby 10 developers or two architects or maybe one developer with a smartphone start to "bash" out what we all hope will one day become saleable code, whilst a project manager and a handful of business analysts engage in long and detailed discussions with the people who want the software built.

Somehow, not everyone is quite exactly available at the same time, so they fly around a lot and talk about how the software will look and review their decisions. Those decisions are then re-reviewed and re-decided with the provision that any decision made can always be unmade, and then made again later, because that is what agile is all about.

Fortunately, the developers never actually meet the business analysts, so they are able to get some real work done without having to refer to outdated and changing requirements.

This is great news for the testers who are able to catch up on their blogging commitments, speaking tours and internet-based self-teaching programmes.

The essence of agile is direct and frequent communication based on a combination of data about how we are measured against what we’re aiming for. The goal in this communication is to evaluate what we did, what we need to do next and what is preventing progress.

It sounds simple, but communication is the key, and it is the hardest component to perfect.

What is useful about the agile approach in this regard is the emphasis on regular, focused conversations in a facilitated format.

We constantly have to check our assumptions and agreements with each other. Have you noticed how easy it is to go off track and how quickly it can happen, even when everybody “knows what they’re doing,” the team is “experienced” and the client “knows what they want?”

In fact, in those cases it is especially easy to get side-tracked. I may know what I’m doing, but is that helpful to our goals right now? The team may be experienced, but what are the lessons of that experience that are going to be useful this time around? Do we have the skills, willingness and courage to learn what is needed as we go? The client may know what they want, but have they asked for what they want? And, how much control are they expecting to exert over what can change over the life of the project?

Taking daily doses of the essence of agile minimises the key killer of software project value:

Doing the wrong thing at the wrong time even if we do end up getting paid for it.

Happy 20th Birthday to “Managing the Design Factory”

“Managing the Design Factory” by Donald G. Reinertsen was first published 20 years ago, and its tools, techniques and message are still relevant today.

In software delivery, Scrum and agile are not mandatory. I find it useful to remember this, since I often find myself in conversations with people who are interested in the successes and risk mitigations that Scrum enables. As a Scrum Master, understanding when and how Scrum can be successful helps me make responsible and informed recommendations. Yes, it’s true, I don’t always recommend Scrum, but I do always recommend that people learn more about it, try it and then decide for themselves.

So, I’d like to recommend a book to you, which was itself recommended to me by Mike Cohn when I attended his product owner training in London last year. If you have not read it, then I hope the following brief review will encourage you to do so.

First published in 1997, in the early days of Scrum and before the Manifesto for Agile Software Development was created, “Managing the Design Factory” takes a ground up, economic value-driven view of software development. It will be of interest to senior IT and business managers, product owners, Scrum Masters and anyone else with an interest in the history of software.

It contains no explicit references to agile or Scrum, but it does contain clear expositions of process design that can be seen in the Scrum Guide. I have found the beautifully concise Scrum Guide to be enriched by the detailed and soundly reasoned explanations in “Managing the Design Factory.”

Each chapter is clearly and methodically written, and simple but effective planning and estimation tools are provided. This allows the reader to systematically acquire the knowledge and techniques that will enable them to make the best decisions possible when designing the process best suited to their own particular business requirement for software.

Some of the key messages I have taken, and which we hold in our minds when we enact Scrum, are:

The purpose of the design process is to make money. In addition, we need to understand where in the process value is being added or taken away. This is manifested in Scrum’s product backlog, where there is constant review of value and priorities. Manufacturing process design and software development process design require different approaches. In fact, applying manufacturing quality standards to software not only removes the potential value of the software, it reduces software quality. It is worth remembering that there are still national and global enterprises whose software delivery processes are based on manufacturing process rules. In the software delivery process, where there is risk, there is also potential value. Development of small increments of software product release business value more quickly. As I am sure we all know, the delivery of regular, valuable software is a core agile principle.

“Managing the Design Factory” addresses manufacturing process design, firmware process design and software process design. This breadth provides excellent context for the reader to understand what is different about the design of the processes as well as the management of the people involved in software development. This knowledge will help managers decide if Scrum and agile could work for them.

For those of us who are already convinced of the benefits and value of Scrum, this book may help you understand why Scrum works and thus may also help you explain it to others – something I find myself doing almost daily.

Reading this book has made me a more informed, more confident and more effective Scrum Master. So, thank you, “Managing the Design Factory,” and Happy 20th Birthday!

Have you read “Managing the Design Factory,” or any of Donald G. Reinertsen’s other books? What did you think? Let me know in the comments section below.

The Retrospective Meeting: A Golden Opportunity to Learn

In 2007, just before my first encounter with Scrum, I went to a lessons learned workshop as the test manager for a multinational company implementing a global technology and corporate change program.

By this time, I had attended many lessons learned workshops, which often took place at the end of projects, as a way of learning from the past project and improving the next. I enjoyed these events as they were an opportunity to work with the whole team to look at and solve all those day-to-day problems that prevent large complex implementations from succeeding.

I was often excited by lessons learned opportunities, but I was particularly looking forward to this one, since it was taking place midway through the project, and there was a chance that much-needed change would be implemented.

We workshopped away for a couple of hours and ended up with a stack of issues that needed solving, all clearly visible on a very large wall. The session had been productive. At this point, the program manager stepped up and announced rhetorically: “Well, I don’t think there’s anything there that we don’t already know!” And then he closed the meeting. We just shrugged reluctantly and went back to our jobs.

That meeting cost several thousand dollars. What did I learn? That leadership was not listening to its teams. What did the teams learn? Nothing that they didn’t already know. That program did not fare well.

Then I Found Scrum …

I was excited when I was introduced to Scrum and the Agile Manifesto shortly afterwards, since continuous learning is not just encouraged, it is essential.

Sure, learning is essential in a waterfall program, too, but the beauty of Scrum is that it makes it achievable by:

Working with small teams so that shared communication is optimal Embedding frequent inspection into the framework through the daily scrum, review and retrospective

When you’re working with a team to get stuff done, being able to adjust and adapt to not only changing circumstances, but also your own internal team dynamics is pure gold. It is one of the behaviours that make a team great.

How Scrum Facilitates Learning Lessons

The Scrum framework gives the team a sprint goal and empowers them to deliver it. The team is therefore invested in learning about itself.

Because of the daily scrum and the close-knit working relationship of the team members, external impediments are quickly identified and raised, so don’t become “lessons learned” flotsam.

The desire to deliver something of value and put the goal of the team first enables individuals to negotiate “personal preference” impediments. These are the impediments that we all bring to any team we join, and are simply our personal preferences for ways of doing things.

Having a clear team goal helps us to put these to one side when we need to and/or negotiate a shared way of working. In this way, the team culture develops.

The Scrum Master, by practicing servant leadership can support teams and invite them to act as servant leaders to one other. The retrospective is a good event for developing this skill.

The Retrospective is a Gold Mine … But Not Everyone Sees It That Way

The retrospective acts as a container in which a team can influence the next sprint by looking at the last one. It is an opportunity for team members to support and acknowledge one other by reflecting on the good work they have done, and their ability to adapt and find creative solutions.

In a positive working environment such as this, issues and difficulties are more easily resolved and can make the retrospective enjoyable, interesting and productive. Conflicts will naturally occur, but in a culture of mutual respect, they can be resolved openly and for the good of all.

However, I have known Scrum teams to discard the retrospective on the grounds that they already know what the problem is, and believe they would make better use of the time in working to deliver more features, especially since they have not delivered much in the last sprint (or two). This is a team that has lost its way.

The following scene is based on a version of real events …

Enter the Scrum Master.

Scrum Master: O Team!

Delivery Team: Yes, O Servant Leader?

Scrum Master: Have you forgotten that if you do not do Scrum, you are not doing Scrum?

Delivery Team: Go away, for we are busy!

Scrum Master: But what is the sprint goal towards which you strive?

Delivery Team: Um…

Scrum Master: Ah, let us retrospect and entreat the product owner to share their vision so we may be enthused and therefore desire to make it so.

Delivery Team: OK, but the product owner is busy and wants us to just get it done.

Scrum Master: Fie!* Let us retrospect all together as one Scrum team and reflect on the importance of the energizing goal.

*interjection, used to express annoyance

In the past 12 months, I have worked with a number of teams that have been introduced to Scrum and retrospectives for the first time.

And one question I have posed is: “How do we know when we have learned a lesson?” After all, saying that you have learned something is one thing; changing a working practice or behavior is quite another. Change often occurs incrementally. So how can a Scrum Master facilitate this process?

How to Get the Most From Retrospectives

A Scrum Master can help by supporting the team to create its own team charter—becoming a great team is not an accident or coincidence; it needs to be a conscious decision.

The team charter is team created and co-owned by every member. It defines the behaviours and ways of working that will help the team grow and work together. By setting this vision for itself, the team creates something it can reflect on in a retrospective, not as a way of punishing itself for failure, but as a way to honestly look at how it has lived its own principles during the last sprint.

With a clear sprint goal and team charter in place, the retrospective will be a rich source of ongoing learning.

In time, retrospective behavior and thinking can become a part of the daily work: Not everything needs to be saved for the retrospective; equally, not everything can be resolved on the spot.

What I like to do is create an area on the Scrum board called “retrospective,” where team members can stick a note when something occurs to them that may not be an immediate problem, but a creative way to improve the team’s delivery capability. We then review the notes in the retrospective and work with the ones that are still resonant.

This can remove the pressure on team members at retrospective time, as they won’t spend so much time furrowing their brows trying to remember that thing that happened.

Another light touch technique is to get a large white board, draw a horizontal line across the middle, put a vertical start point far left and a vertical finish point far right.

The distance between the two points represents the sprint. The y-axis represents “level of activity”; the top is “frenzied activity,” the bottom is “nothing, zero, zilch.”

Now, ask each team member the draw a line from left to right over the life of the sprint that shows their level of activity as the sprint progressed (different colours work well).

It’s fun, and when done, there is a clear pattern that the team can see. Often the team will annotate sudden plunges or spikes. This technique can be a good place to start if you have a team that often inflicts three days of frenzied activity on itself at the end of every sprint.

In sum, I’ve been through many variations of “lessons learned” meetings throughout my career, and this is what I have learned so far:

Some lessons have to be learned many times. Some lessons are learned quickly, some slowly, bit by bit. Some are never learned. But, the most important lesson of Scrum, for me, is to challenge the team to take responsibility for its own development by creating a charter, trust it to live those principles and remind it to reflect on them at the retrospective meeting.