How does Agile help with long term planning?

I’m often involved in discussions about Agile that question its efficacy in one way or another. This is, in my view, a very good thing and I highly encourage this line of inquiry. It’s important to challenge the assumptions behind Agile so as to counteract any complacency or expectation that it is a panacea to project management ills. Even so, with apologies to Winston Churchill, Agile is the worst form of project management…except for all the others.

Challenges like this also serve to instill a strong understanding of what an Agile mindset is, how it’s distinct from Agile frameworks, tools, and practices, and where it can best be applied. I would be the first to admit that there are projects for which a traditional waterfall approach is best. (For example, maintenance projects for nuclear power reactors. From experience, I can say traditional waterfall project management is clearly the superior approach in this context.)

A frequent challenge the idea that with Agile it is difficult to do any long-term planning.

Consider the notion of vanity vs actionable metrics. In many respects, large or long-term plans represent a vanity leading metric. The more detail added to a plan, the more people tend to believe and behave as if such plans are an accurate reflection of what will actually happen. “Surprised” doesn’t adequately describe the reaction when reality informs managers and leaders of the hard truth. I worked a multi-million dollar project many years ago for a Fortune 500 company that ended up being canceled. Years of very hard work by hundreds of people down the drain because projected revenues based on a software product design over seven years old were never going to materialize. Customers no longer wanted or needed what the product was offering. Our “solution” no longer had a problem to solve.

Agile – particularly more recent thinking around the values and principles in the Manifesto – acknowledges the cognitive biases in play with long-term plans and attempts to put practices in place that compensate for the risks they introduce into project management. One such bias is reflected in the planning fallacy – the further out the planning window extends into the future, the less accurate the plan. An iterative approach to solving problems (some of which just happen to use software) challenges development teams on up through managers and company leaders to reassess their direction and make much smaller course corrections to accommodate what’s being learned. As you can well imagine, we may have worked out how to do this in the highly controlled and somewhat predictable domain of software development, however, the critical areas for growth and Agile applicability are at the management and leadership levels of the business.

Another important aspect the Agile mindset is reflected in the Cone of Uncertainty. It is a deliberate, intentional recognition of the role of uncertainty in project management. Yes, the goal is to squeeze out as much uncertainty (and therefore risk) as possible, but there are limits. With a traditional project management plan, it may look like everything has been accounted for, but the rest of the world isn’t obligated to follow the plan laid out by a team or a company. In essence, an Agile mindset says, “Lift your gaze up off of the plan (the map) and look around for better, newer, more accurate information (the territory.) Then, update the plan and adjust course accordingly.” In Agile-speak, this is what is behind phrases like “delivery dates emerge.”

Final thought: You’ll probably hear me say many times that nothing in the Agile Manifesto can be taken in isolation. It’s a working system and some parts if it are more relevant than others depending on the project and the timing. So consider what I’ve presented here in concert with the Agile practices of developing good product visions and sprint goals. Product vision and sprint goals keep the project moving in the desired direction without holding it on an iron-rails-track that cannot be changed without a great deal of effort, if at all.

So, to answer the question in the post title, Agile helps with long term planning by first recognizing the the risks inherent in such plans and implementing process changes that mitigate or eliminate those risks. Unpacking that sentences would consist of listing all the risks inherent with long-term planning and the mechanics behind and reasons why scrum, XP, SAFe, LeSS, etc., etc., etc. have been developed.


Image by Lorri Lang from Pixabay

The Changeability Decision Matrix

Responding to change over following a planThe Agile Manifesto

That’s one of the four values to the Agile Manifesto. It’s also one of the values that is commonly plucked from the context of three other values and twelve principles. Once isolated, it’s exaggerated and inflated to some form of “We can’t define scope before we start work! There’s too much discovery work to be done first! We don’t know what we don’t know! Scope (and requirements) are emergent!” That bends the intent of the Manifesto and disregards the context from which a single value has been extracted.

I don’t believe Agile practices ever meant for software development to be a free-for-all, a never ending saga of finding and implementing better and better ways to code something before a product can be released. Projects run like this never see the light of day, let alone a shelf to languish on waiting for a long since departed market opportunity.

What isn’t in the Agile Manifesto, but is implicit in the Agile methodologies I’ve worked with is the notion of decision points. These are the points around which change, to a small or large degree, is not allowed. At least not for a while. Decision points bring stability to the development process from which Agile teams can move forward with a stable set of assumptions. If subsequent discoveries inform the team that they need to revisit a decision, than they must do so. The key element is that the work subsequent to the decision is what generates the need to revisit the decision. It isn’t done arbitrary, on a hunch, or with minimal information.

There are numerous decision points that exist within Scrum and SAFe, for example. Stories are decisions. “We need to create this thing.” Acceptance criteria, definitions of ready and done, sprint duration, feature and epic definitions, milestones, minimum viable/valuable products are also examples of decisions. Some of these can be quite changeable. Stories, for example, can be refined many times prior to and during sprint planning. The description, acceptance criteria, definition of done, and effort estimation can change many times before a story is committed to a sprint. And there’s the decision point. When the team agrees that a story can be brought into a sprint and they commit to completing it before the sprint is over, they have made a decision and the story shouldn’t change on its way to being completed by the team. (As noted previously, the work on the story may reveal a need to change something about the story – maybe even indicate that work on the story should stop – but that should be an edge case and not part of common practice.)

To help teams understand these distinctions, I’ve developed a 2X2 matrix called the Changeability Decision Matrix. Its purpose is to help teams evaluate the effects of changing work in the queue. The horizontal axis goes from “Small Impact” to “Big Impact.” The vertical axis goes from “Few Changes” to “Many Changes.”

The two questions the team needs to ask when thinking about changing a decision they’ve made (acceptance criteria, story description, MVP, etc.) are:

  • Will this change have a small or big impact? They may consider any number of variables: cost, time, productivity, effort, etc.
  • Will this change require a few or many changes (lines of code, documentation updates, other components that consume the code, budgets, release dates, etc.)

Where the proposed change resides on the grid may be dependent on where the team is on the project timeline. Consider the Epic, feature, and story hierarchy: Early in the project – during the design phase, for example – there may be little more than features in the backlog. As placeholders for ideas, they may be quite volatile as new marketing information enters the conversation or obvious technical issues become apparent. So changing an epic or a feature may have a relatively small impact on the project and involve few changes. Most probably there won’t be any code involved at this point.

As the project progress and backlog refinement continues, epics and features will be broken up into large stories. More detail is added to the backlog and more time and money has been invested in the design so the epics and features are less changeable. If any changes are needed, it is probably that the impact of those changes and the number of things that need to change will be greater than it would have been during the design phase.

Eventually, as the project moves into high gear, the backlog will become populated with more and more smaller stories that can be easily estimated and planned into sprints and increments.

For the duration of the project, it’s likely most of the stories in the backlog can and should be responsive to multiple changes…right up to the point the decision is made to drop the story into a sprint.

The Changeability Decision Matrix is an easy way to evaluate whether or not an Agile team is pondering undoing a small or large decision by forcing the conversation around the consequences of making the change. If either of these two axis are not a good fit for your organization or what you consider important to consider, then change them to something that makes more sense to your project.

Here is a representation of these phases on a hypothetical project timeline:See also:


Photo by Linus Nylund on Unsplash

The Practice of Sizing Spikes with Story Points

Every once and a while it’s good to take a tool out of it’s box and find out if it’s still fit for purpose. Maybe even find if it can be used in a new way. I recently did this with the practice of sizing spikes with story points. I’ve experienced a lot of different projects since last revisiting my thinking on this topic. So after doing a little research on current thinking, I updated an old set of slides and presented my position to a group of scrum masters to set the stage for a conversation. My position: Estimating spikes with story points is a vanity metric and teams are better served with spikes that are time-boxed but unsized.

While several colleagues came with an abundance of material to support their particular position, no one addressed the points I raised. So it was a wash. My position hasn’t changed appreciably. But I did gain from hearing several arguments for how spikes could be used more effectively if they were to be sized with story points. And perhaps the feedback from this article will further evolve my thinking on the subject.

To begin, I’ll answer the question of “What is a spike?” by accepting the definition from agiledictionary.com:

Spike

A task aimed at answering a question or gathering information, rather than at producing shippable product. Sometimes a user story is generated that cannot be well estimated until the development team does some actual work to resolve a technical question or a design problem. The solution is to create a “spike,” which is some work whose purpose is to provide the answer or solution.

The phrase “cannot be well estimated” is suggestive. If the work cannot be well estimated than what is the value of estimating it in the first place? Any number placed on the spike is likely to be for the most part arbitrary. Any number greater than zero will therefore arbitrarily inflate the sprint velocity and make it less representative of the business value being delivered. It may make the team feel better about their performance, but it tells the stakeholders less about the work remaining. No where can I find a stated purpose of Agile or scrum to be making the team “feel better.” In practice, by masking the amount of business value being delivered, the opposite is probably true. The scrum framework ruthlessly exposes all the unhelpful and counterproductive practices and behaviors an unproductive team may be unconsciously perpetuating.

Forty points of genuine business value delivered at the end of a sprint is 100% of rubber on the road. Forty points delivered of which 10 are points assigned to one or more spikes is 75% of rubber on the road. The spike points are slippage. If they are left unpointed then it is clear what is happening. A spike here and there isn’t likely to have a significant impact on the velocity trend over 8 or 10 sprints, for example. One or more spikes per sprint is another matter. In this case, numerous spikes in successive sprints will likely cause the velocity to sink. If this happens, a number of corrective actions can be taken – actions that may be missed if the velocity is falsely kept at a certain desired or expected value. In other words, pointing spikes hides important information that could very well impact the success of the project. Bad news can inspire better decisions and corrective action. Falsely positive news most often leads to failures of the epic variety.

Consider the following two scenarios.

Team A has decided to add story points to their spikes. Immediately they run into several significant challenges related to the design and the technology choices made. So they create a number of spikes to find the answers and make some informed decision. The design and technology struggles continue for the next 10 sprints. Even with the challenges they faced, the team appears to have quickly established a stable velocity.

The burndown, however, looks like this:

If the scrum master were to use just the velocity numbers it would appear Team A is going to finish their work in about 14 sprints. This might be true if Team A were to have no more spikes in the remaining sprints. The trend, however, strongly suggests that’s not likely to happen. If a team has been struggling with design and technical issues for 10 sprints, it is unlikely those struggles will suddenly stop at sprint 11 or beyond unless there have been deliberate efforts to mitigate that potential. By pointing spikes and generating a nice-looking velocity chart it is more probable that Team A is unaware of the extent to which they may be underestimating the amount of time to complete items in the backlog.

 

 

Team B finds themselves in exactly the same situation as Team A. They immediately run into several significant challenges related to the design and the technology choices made and create a number of spikes to find the answers and make some informed decision. However, they decided not to add story points to their spikes. The design and technology struggles continue for the next 10 sprints. The data show that Team B is clearly struggling to establish a stable velocity.

 

And the burndown looks like this, same as Team A after 10 sprints:

However, it looks like it’s going to take Team B 21 more sprints to complete the work. That they’re struggling isn’t good. That it’s clear they struggling is very good. This isn’t apparent with Team A’s velocity chart. Since it’s clear they are struggling it is much easier to start asking questions, find the source of the agony, and make changes that will have a positive impact. It is also much more probably that the changes will be effective because they will have been based on solid information as to what the issues are. Less guess work involved with Team B than with Team A.

However, any scrum master worth their salt is going to notice that the product backlog burndown doesn’t align with the velocity chart. It isn’t burning down as fast as the velocity chart suggests it should be. So the savvy Team A scrum master starts tracking the burndown of value-add points vs spike points. Doing so might look like the following burndown:

Using the average from the parsed burndown, it is much more likely that Team A will need 21 additional sprints to complete the work. And for Team B?

The picture of the future based on the backlog burndown is a close match to the picture from the velocity data, about 22 sprints to complete the work.

If you were a product owner, responsible for keeping the customer informed of progress, which set of numbers would you want to base your report on? Would you rather surprise the customer with a “sudden” and extended delay or would you rather communicate openly and accurately?

Summary

Leaving spikes unpointed…

  • Increases the probability that performance metrics will reveal problems sooner and thus allow for corrective actions to be taken earlier in a project.
  • The team’s velocity and backlog burndown is a more accurate reflection of value actually being created for the customer and therefore allows for greater confidence of any predictions based on the metrics.

I’m interested in hearing your position on whether or not spikes should be estimated with story points (or some other measure.) I’m particularly interested in hearing where my thinking described in this article is in need of updating.

[An earlier version of this article originally appeared on the Agile Alliance blog.]

Teams, Tribes, and Community – 0.1.0

Several months ago, I made bold decision: Take command of the helm for a brilliant tribe of diverse creative thinkers dedicated to helping each other succeed. This is the first of an on-going series of posts – maybe once or twice a month – describing this evolving effort.

For an extrovert, this might not have been a bold decision. But in my case, you should know I designed the card that card-carrying introverts carry. So this decision involved a more thorough application of my already robust decision-making process. On a professional level, this may be the most significant challenge I’ve taken on to date. Will my years of experience with forming and guiding teams help this tribe further their success? Will I be able to find the gravitational force that holds us together and the spark that keeps us inspired? These are open questions. They are also questions that occupy much of my thinking.

We are not dedicated to achieving a single goal or moving in a unified direction. We each have our areas of expertise and independent business goals. We are much more a tribe than a team. As such, I believe we will be guided more by tribal dynamics and models than team rules and policies. The path is not clear, but this much I know…

  • There is no leader of this tribe. Not in the sense of a single person who’s responsible for setting the direction and making all the decisions that impact the organization. There is no “Chief” or “Czar” of anything. I’ll fill the role of Launch Commander and Flight Commander in order to get us organized and moving forward. However, I have been clear from the start about my intention to structure our tribe on principles of self-organization.
  • The emphasis is on simple and accessible technology and easy ways to organize meetings based on Agile principles and practices – lean coffee, for example, has served us well for our initial meetings. What has emerged since then are more involved and interactive meeting formats, such as client role-plays and accountability exercises. Keeping things simple and remaining mindful of barriers to participation is vital. Too many tools with too many logins risks the creation of a Tower of Babel. For now, the weekly video call is the center-point around which we all meet. This in itself is enough of a challenge given the global participation. Other than this, email is the acknowledged primary channel for asynchronous communication.
  • We are not accepting new members. Whether or for how long this remains the case is undecided. We have discussed various ways of introducing new members, but have decided to decide on this issue later. The circumstances that brought each of us together created a unique bond of trust and familiarity with each other’s business interests that makes the introduction of new members a risk to maintaining these relationships. At the moment, we are tipped slightly toward being on the large size and everyone acknowledges if we grow much bigger the meetings may become unmanageable and the interactions less valuable. Since trust is foundational, none of the details related to who we are and what we discuss will be revealed in this space. My writing will be limited to the general case of what I discover from having participated in and helped guide our tribe. It is my hope this may help others with forming and guiding their own teams and tribes.

Whatever the outcome, it’s been more fun thus far than I’ve had in a loooooong time.


Image by Youssef Jheir from Pixabay

Agile and Changing Requirements or Design

I hear this (or some version) more frequently in recent years than in past:

Agile is all about changing requirements at anytime during a project, even at the very end.

I attribute the increased frequency to the increased popularity of Agile methods and practices.

That the “Responding to change over following a plan” Agile Manifesto value is cherry picked so frequently is probably due to a couple of factors:

  • It’s human nature for a person to resist being cornered into doing something they don’t want to do. So this value gets them out of performing a task.
  • The person doesn’t understand the problem or doesn’t have a solution. So this value buys them time to figure out how to solve the problem. Once they do have a solution, well, it’s time to change the design or the requirements to fit the solution. This reason isn’t necessary bad unless it’s the de facto solution strategy.

The intent behind the “Responding to change” value, and the way successful Agile is practiced, does not allow for constant and unending change. If this were otherwise nothing would ever be completed and certainly nothing would ever be released to the market.

I’m not going to rehash the importance of the preposition in the value statement. Any need to explain the relativity implied by it’s use has become a useful signal for me to spend my energies elsewhere. But for those who are not challenged by the grammar, I’d like to say a few thing about how to know when change is appropriate and when it’s important to follow a plan.

The key is recognizing and tracking decision points. With traditional project management, decisions are built-in to the project plan. Every possible bit of work is defined and laid out on a Gantt chart, like the steel rails of a train track. Deviation from this path would be actively discouraged, if it were considered at all.

Using an Agile process, decision points that consider possible changes in direction are built into the process – daily scrums, sprint planning, backlog refinement, reviews and demonstrations at the end of sprints and releases, retrospectives, acceptance criteria, definitions of done, continuous integration – these all reflect deliberate opportunities in the process to evaluate progress and determine whether any changes need to be made. These are all activities that represent decisions or agreements to lock in work definitions for short periods of time.

For example, at sprint planning, a decision is made to complete a block of work in a specified period of time – often two weeks. After that, the work is reviewed and decisions are made as to whether or not that work satisfies the sprint goal and, by extension, the product vision. At this point, the product definition is specifically opened up for feedback from the stakeholders and any proposed changes are discussed. Except under unique circumstances, changes are not introduced mid-sprint and the teams stick to the plan.

Undoing decisions or agreements only happens if there is supporting information, such as technical infeasibility or a significant market shift. Undoing decisions and agreements doesn’t happen just because “Agile is all about changing requirements.” Agile supports changing requirements when there is good reason to do so, irrespective of the original plan. With traditional project management, it’s all about following the plan and change at any point is resisted.

This is the difference. With traditional project management, decisions are built-in to the project plan. With Agile they are adapted in.


Photo by Chris Lawton on Unsplash

Behind the Curtain: The Delivery Team Role

Even with the formalization of Agile practices into numerous frameworks and methodologies, I have to say not much has changed for the software developer or engineer with respect to how they get work done. I’m not referring to technology. The changes in what software developers and engineers use to get work done has been seismic. The biggest shift in the “how,” in my experience, is that what were once underground practices are now openly accepted and encouraged.

When I was coding full time, in the pre-Agile days and under the burden of CMM, we followed all the practices for documenting use cases, hammering out technical and functional specs, and laboriously talking through requirements. (I smile when I hear developers today complain about the burden of meetings under Scrum.) And when it came time to actually code, I and my fellow developers set the multiple binders of documentation aside and engaged in many then unnamed Agile practices. We mixed and matched use cases in a way that allowed for more efficient coding of larger functional components. We “huddled” each morning in the passage way to cube pods to discuss dependencies and brainstorm solutions to technical challenges. Each of these became more efficiently organized in Agile as backlog refinement and daily stand-ups. We had numerous other loose practices that were not described in tomes such as CMM.

But Agile delivery teams today are frequently composed of more than just technical functional domains. There may be non-technical expertise included as integral members of the team. Learning strategists, content editors, creative illustrators, and marketing experts may be part of the team, depending on the objectives of the project. Consequently, this represents a significant challenge to technical members of the team (i.e. software development and  tech QA) who are unused to working with non-technical team members. Twenty years ago a developer who might say “Leave me alone so I can code.” would have been viewed as a dedicated worker. Today, it’s a sign that the developer risks working in isolation and consequently delivering something that is mis-matched with the work being done by the rest of the team.

On an Agile delivery team, whether composed of a diverse set of functional domains or exclusively technical experts, individual team members need to be thinking of the larger picture and the impact of their work on that of their team mates and the overall work flow. They need to be much more attentive to market influences than in the past. The half-life of major versions, let along entire products, is such that most software products outside a special niche can’t survive without leveraging Agile principles and practices. Their knowledge must expand beyond just their functional domain. The extent to which they possess this knowledge is reflected in the day-to-day behaviors displayed by the team and it’s individuals.

  • Is everyone on the team sensitive and respectful of everyone else’s time? This means following through on commitments and promises, including agreed upon meeting start times. One person showing up five minutes late to a 15 minute stand-up has just missed out on a third of the meeting at least. If the team waits for everyone to show up before starting, the late individual has just squandered 5 minutes multiplied by the number of team mates. For a 6 member team, that’s a half hour. And if it happens every day, that’s 2.5 hours a week. It adds up quickly. Habitual late-comers are also signaling a lack of respect to other team members. They are implicitly saying “Me and my time is more important than anyone else on the team.” Unchecked, this quickly spills over into other areas of the team’s interactions. Enforcing an on-time rule like this is key to encouraging the personal discipline necessary to work effectively as a team. When a scrum master keeps the team in line with a few basic items like this, the larger discipline issues never seem to arise. As U.S. Army General Ann Dunwoody (ret.) succinctly points out in her book, never walk by a mistake. Doing so gives implicit acceptance for the transgression. Problems blossom from there, and it isn’t a pretty flower. (As a scrum master, I cannot “make” someone show up on time. But I can address the respect aspect of this issue by always starting on time. That way, late-comers stand out as late, not more important. Over time, this tends to correct the lateness issue as well.)
  • Everyone on the team must be capable of tracking a constantly evolving set of dependencies and knowing where their work fits within the flow. To whom will they be delivering work? From whom are they expecting completed work? The answer to these questions may not be a name on the immediate team. Scrum masters must periodically explicitly ask these questions if the connections aren’t coming out naturally during stand-ups. Developing this behavior is about coaching the team to look beyond the work on their desk and understand how they are connected to the larger effort. Software programmers seem to have a natural tendency to build walls around their work. Software engineers less so. And on teams with diverse functional groups it is important for both the scrum master and product owner to be watchful for when barriers appear for reasons that have more to due to lack of familiarity across functional domains than anything else.
  • Is the entire team actively and consistently engaged with identifying and writing stories?
  • Is the team capable and willing to cross domain boundaries and help? Are they interested in learning about other parts of the product and business?

Product owners and scrum masters need to be constantly scanning for these and other signs of disengagement as well as opportunities to connect cross functional needs.


Photo by Yu Hosoi on Unsplash

Behind the Curtain: The Product Owner Role

When someone owns something they tend to keep a closer watch on where that something is and whether or not it’s in good working order. Owners are more sensitive to actions that may adversely affect the value of their investment. It’s the car you own vs the car you rent. This holds true for products, projects, and teams. For this reason the title of “product owner” is well suited to the responsibilities assigned to the role. The explicit call-out to ownership carries a lot of goodness related to responsibility, leadership, and action.

Having been a product owner and having coached product owners, I have a deep respect for anyone who takes on the challenges associated with this role. In my view, it’s the most difficult position to fill on an Agile team. For starters, there are all the things a product owner is responsible for as described in any decent book on scrum: setting the product vision and road map, ordering the product backlog, creating epics and story cards, defining acceptance criteria, etc. What’s often missing from the standard set of bullet items is the “how” for doing them well. Newly minted product owners are usually left to their own devices for figuring this out. And unfortunately, in this short post I won’t be offering any how-to guidance for developing any of the skills generally recognized to be part of the product owner role.

What I’d most like to achieve in this post is calling out several of the key skills associated with quality product ownership that are usually omitted from the books and trainings – the beyond-the-basics items that any product owner will want to include in their continuous learning journey with Agile. In no particular order…

  • Product owners must be superb negotiators when working with stakeholders and team members. The techniques used for each group are different so it’s important to understand the motivations that drive them. I’ve found Jim Camp’s “Start with No” and Chris Voss’ “Never Split the Difference” to be particularly helpful in this regard.
  • One of the four values stated in the Agile Manifesto is “Responding to change over following a plan.” Unfortunately, this is often construed to mean any change at any time is valuable. This Agile value isn’t a directive for maximum entropy and chaos. Product owners must remain vigilant to scope changes. And the boundaries for scope are defined by the decisions product owners make. So product owners must be decisive and committed to the decisions, agreements, and promises that have been made with stakeholders and the team.
  • Product owners need to have a good sense for when experimentation may be needed to sort out any complex or risky features in the project – creating spikes or proof-of-concept work early enough in the project so as to avoid any costly pivots later in the project.
  • Even with experimentation, making the call to pivot will require a product owner’s clear understanding of past events and any path forward that offers the greatest chance for success. As the sage says, predictions are difficult to make, particularly about the future. The expectation isn’t for clairvoyance, rather for the ability to pay close attention to what the (non-vanity) data are telling them.
  • Understanding how to work through failures and dealing with “The Dip,” as Seth Godin calls it, are also important skills for a product owner. The team and the stakeholders are going to look to the product owner’s leadership to demonstrate confidence that they are on the right track.

More than other roles on an Agile team, the product owner must be a truly well-rounded and experienced individual. Paradoxically, it is a role that is both constrained by the highly visible nature of the position and dynamic due to the skill set required to maximize the chances for project success.


Image by Free-Photos from Pixabay

Agile Metrics – Time (Part 3 of 3)

In Part 1 of this series, we set the frame for how to use time as a metric for assessing Agile team and project health. In Part 2, we looked at shifts in the cross-over point between burn-down and burn-up charts. In Part 3, we’ll look at other asymmetries and anomalies that can appear in time burn-down/burn-up charts and explore the issues the teams may be struggling with under these circumstances.

Figure 1 shows a burn-up that by the end of the sprint significantly exceeded the starting value for the original estimate.

Figure 1

There isn’t much mystery around a chart like this. The time needed to complete the work was significantly underestimated. The mystery is in the why and what that led to this situation.

  • Where there unexpected technical challenges?
  • Were the stories poorly defined?
  • Were the acceptance criteria unclear?
  • Were the sprint goals, objectives, or minimum viable product definition unclear?

Depending on the tools used to capture team metrics, it can be helpful to look at individual performances. What’s the differential between story points and estimated time vs actual time for each team member? Hardly every useful as a disciplinary tool, this type of analysis can be invaluable for knowing who needs professional development and in what areas.

In this case, there were several technical challenges related to new elements of the underlying architecture and the team put in extra hours to resolve them. Even so, they were unable to complete all the work they committed to in the sprint. The the scrum master and product owner need to monitor this so it isn’t a recurrent event or they risk team burnout and morale erosion if left unchecked. There are likely some unstated dependencies or skill deficiencies that need to be put on the table for discussion during the retrospective.

Figure 2 shows, among other things, unexpected jumps in the burn-down chart. There is clearly a significant amount of thrashing evident in the burn-down (which stubbornly refuses to actually burn down.)

Figure 2

Questions to explore:

  • Are cards being brought into the sprint after the sprint has started and why?
  • Are original time estimates being changed on cards after the sprint has started?
  • Is there a stakeholder in the grass, meddling with the team’s commitment?
  • Was a team member added to the team and cards brought into the sprint to accommodate the increased bandwidth?
  • Whatever is causing the thrashing, is the team (delivery team members, scrum master, and product owner) aware of the changes?

Scope change during a sprint is a very undesirable practice. Not just because it goes against the scrum framework, but more so because it almost always has an adverse effect on team morale and focus. If there is an addition to the team, better to set that person to work helping teammates complete the work already defined in the sprint and assign them cards in the next sprint.

If team members are adjusting original time estimates for “accuracy” or whatever reason they may provide, this is little more than gaming the system. It does more harm than good, assuming management is Agile savvy and not intent on using Agile metrics for punitive purposes. On occasion I’ve had to hide the original time estimate entry field from the view of delivery team members and place it under the control of the product owner – out of sight, out of mind. It’s less a concern to me that time estimates are “wrong,” particularly if time estimate accuracy is showing improvement over time or the delta is a somewhat consistent value. I can work with an delivery team member’s time estimates that are 30% off if they are consistently 30% off.

In the case of Figure 2 it was the team’s second sprint and at the retrospective the elephant was called out from hiding: The design was far from stable. The decision was made to set aside scrum in favor of using Kanban until the numerous design issues could be resolved.

Figure 3 shows a burn-down chart that doesn’t go to zero by the end of the sprint.

Figure 3

The team missed their commit and quite a few cards rolled to the next sprint. Since the issue emerged late in the sprint there was little corrective action that could be taken. The answers were left to discovery during the retrospective. In this case, one of the factors was the failure to move development efforts into QA until late in the sprint. This is an all too common issue in cases where the sprint commitments were not fully satisfied. For this team the QA issue was exacerbated by the team simply taking on more than they thought they could commit to completing. The solution was to reduce the amount of work the team committed to in subsequent sprints until a stable sprint velocity emerged.

Conclusion

For a two week sprint on a project that is 5-6 sprints in, I usually don’t bother looking at time burn-down/burn-up charts for the first 3-4 days. Early trends can be misleading, but by the time a third of the sprint has been completed this metric will usually start to show trends that suggest any emergent problems. For new projects or for newly formed teams I typically don’t look at intra-sprint time metrics until much later in the project life cycle as there are usually plenty of other obvious and more pressing issues to work through.

I’ll conclude by reiterating my caution that these metrics are yard sticks, not micrometers. It is tempting to read too much into pretty graphs that have precise scales. Rather, the expert Agilest will let the metrics, whatever they are, speak for themselves and work to limit the impact of any personal cognitive biases.

In this series we’ve explored several ways to interpret the signals available to us in estimated time burn-down and actual time burn-up charts. There are numerous others scenarios that can reveal important information from such burn-down/burn-up charts and I would be very interested in hearing about your experiences with using this particular metric in Agile environments.

Agile Metrics – Time (Part 2 of 3)

In Part 1 of this series, we set the frame for how to use time as a metric for assessing Agile team and project health. In Part 2, we’ll look at shifts in the cross-over point between burn-down and burn-up charts and explore what issues may be in play for the teams under these circumstances.

Figure 1 shows a cross-over point occurring early in the sprint.

Figure 1

 

This suggests the following questions:

  • Is the team working longer hours than needed? If so, what is driving this effort? Are any of the team members struggling with personal problems that have them working longer hours? Are they worried they may have committed to more work than they can complete in the sprint and are therefore trying to stay ahead of the work load? Has someone from outside the team requested additional work outside the awareness of the product owner or scrum master?
  • Has the team over estimated the level of effort needed to complete the cards committed to the sprint? If so, this suggests an opportunity to coach the team on ways to improve their estimating or the quality of the story cards.
  • Has the team focused on the easy story cards early in the sprint and work on the more difficult story cards is pending? This isn’t necessarily a bad thing, just something to know and be aware of after confirming this with the team. If accurate, it also points out the importance of using this type of metric for intra-sprint monitoring only and not extrapolate what it shows to a project-level metric.

The answer to these questions may not become apparent until later in the sprint and the point isn’t to try and “correct” the work flow based on relatively little information. In the case of Figure 1, the “easy” cards had been sized as being more difficult than they actually were. The more difficult cards were sized too small and a number of key dependencies were not identified prior to the sprint planning session. This is reflected in the burn-up line that significantly exceeds the initial estimate for the sprint, the jumps in the burn-down line, and subsequent failure to complete a significant portion of the cards in the sprint backlog. All good fodder for the retrospective.

Figure 2 shows a cross-over point occurring late in the sprint.

Figure 2

On the face of it there are two significant stretches of inactivity. Unless you’re dealing with a blatantly apathetic team, there is undoubtedly some sort of activity going on. It’s just not being reflected in the work records. The task is to find out what that activity is and how to mitigate it.

The following questions will help expose the cause for the extended periods of apparent inactivity:

  • Are one or more members not feeling well or are there other personal issues impacting an individual’s ability to focus?
  • Have they been poached by another project to work on some pressing issue?
  • Are they waiting for feedback from stakeholders,  clients, or other team members?
  • Are the story cards unclear? As the saying goes, story cards are an invitation to a conversation. If a story card is confusing, contradictory, or unclear than the team needs to talk about that. What’s unclear? Where’s the contradiction? As my college calculus professor used to ask when teaching us how to solve math problems, “Where’s the source of the agony?”

The actual reasons behind Figure 2 were two fold. There was a significant technical challenge the developers had to resolve that wasn’t sufficiently described by any of the cards in the sprint and later in the sprint several key resources were pulled off the project to deal with issues on a separate project.

Figure 3 shows a similar case of a late sprint cross-over in the burn-down/burn-up chart. The reasons for this occurrence were quite different than those shown in Figure 2.

Figure 3

 

This was an early sprint and a combination of design and technical challenges were not as well understood as originally thought at the sprint planning session. As these issues emerged, additional cards were created in the product backlog to be address in future sprints. Nonetheless, the current sprint commitment was missed by a significant margin.

In Part 3, we’ll look at other asymmetries and anomalies that can appear in time burn-down/burn-up charts and explore the issues may be in play for the teams under these circumstances.

Agile Metrics – Time (Part 1 of 3)

Some teams choose to use card level estimated and actual time as one of the level of effort or performance markers for project progress and health. For others it’s a requirement of the work environment due to management or business constraints. If your situation resembles one of these cases then you will need to know how to use time metrics responsibly and effectively. This series of articles will establish several common practices you can use to develop your skills for evaluating and leveraging time-based metrics in an Agile environment.

It’s important to keep in mind that time estimates are just one of the level of effort or performance markers that can be used to track team and project health. There can, and probably should be other markers in the overall mix of how team and project performance is evaluated. Story points, business value, quality of information and conversation from stand-up meetings, various product backlog characteristics, cycle time, and cumulative flow are all examples of additional views into the health and progress of a project.

In addition to using multiple views, it’s important to be deeply aware of the strengths and limits presented by each of them. The limits are many while the strengths are few.  Their value comes in evaluating them in concert with one another, not in isolation.  One view may suggest something that can be confirmed or negated by another view into team performance. We’ll visit and review each of these and other metrics after this series of posts on time.

The examples presented in this series are never as cut and dried as presented. Just as I previously described multiple views based on different metrics, each metric can offer multiple views. My caution is that these views shouldn’t be read like an electrocardiogram, with the expectation of a rigidly repeatable pattern from which a slight deviation could signal a catastrophic event. The examples are extracted from hundreds of sprints and dozens of projects over the course of many years and are more like seismology graphs – they reveal patterns over time that are very much context dependent.

Estimated and actual time metrics allow teams to monitor sprint progress by comparing time remaining to time spent. Respectively, this will be a burn-down and a burn-up chart in reference to the direction of the data plotted on the chart. In Figure 1, the red line represents the estimated time remaining (burn-down) while the green line represents the amount of time logged against the story cards (burn-up) over the course of a two week sprint. (The gray line is a hypothetical ideal for burn-down.)

Figure 1

The principle value of a burn-down/burn-up chart for time is the view it gives to intra-sprint performance. I usually look at this chart just prior to a teams’ daily stand-up to get a sense if there are any questions I need to be asking about emerging trends. In this series of posts we’ll explore several of the things to look for when preparing for a stand-up. At the end of the sprint, the burn-down/burn-up chart can be a good reference to use during the retrospective when looking for ways to improve.

The sprint shown in Figure 1 is about as ideal a picture as one can expect. It shows all the points I look for that tell me, insofar as time is concerned, the sprint performance is in good health.

  • There is a cross-over point roughly in the middle of the sprint.
  • At the cross-over point about half of the estimated time has been burned down.
  • The burn-down time is a close match to the burn-up at both the cross-over point and the end of the sprint.
  • The burn-down and burn-up lines show daily movement in their respective directions.

In Part 2, we’ll look at several cases where the cross-over point shifts and explore the issues the teams under these circumstances might be struggling with.