Yesterday was good - I learnt something about Cumulative Flow that I didn’t know before and it isn’t in any of the books that I’ve read so far.
Unfortunately my discovery means I’m not as far ahead as I’d hoped in my project and my predictability is not as good as I thought. But good to know for next time.
Hopefully this blog post will be helpful for someone using Kanban in the same way that we have been, hopefully you can avoid the same mistake with velocity projection.
How we use Kanban
A bit of background information. I lead a software development team. We are running a fixed duration project and building two different products. We also have a lot of interrupt driven work that we need to handle in an effective way.
Our motivation for using Kanban was to understand the methodology really well (we used Scrum previously) and to get a level of predictability that Scrum didn’t give us, especially with this ad-hoc, interrupt driven class of work.
You can see our board here..
It has been given a festive makeover at the moment.
You can see our workflow - Requirements, Construction, Acceptance and Completed.
Classes of service
You can also see that we’ve implemented classes of service using horizontal swim-lanes on the board. We allocated 40% effort to each of our products and 20% effort to ad-hoc activity.
Kanban is interesting to us because we want to deal with both day-to-day support activities and to be predictable in building the product that we’re going to ship.
We track our performance using 3 metrics captured on a 2 week iteration basis
- Due date SLA: We deliver 80% of work items in XX days or less (based on the cycle time of each work item)
- Throughput: How many work items we deliver per iteration
- Quality: How many process breakdowns or rework items we need to do per iteration
And we’ve been fairly stable in our performance since using Kanban
So you can see over the last five 2-week iterations we’ve had a 80% cycle time ranging from 14-18 days.
And our throughput was slow in the beginning as the team size was increased - we had 2 iterations delivering 7 and 5 work items, and 2 iterations delivering 13 and 11 work items.
We track the cycle time for each item - The science of Kanban tells us that we should be able to expect a nice bell shaped curve with the distribution of cycle time across all of the work items we’ve completed.
Spoiler: Our distribution is not like this :(
Our Cumulative Flow chart
I blogged last week about our cumulative flow chart.
With the two products that we are working on we have a backlog defined for each and we eventually learnt that defining these separately on the chart (light grey and dark grey) would be a good idea.
That way we can see the work remaining in each product. Although the backlog is being refined constantly in reaction to feedback from our design partners you can see that we’ve made steady progress against each product plus we’ve also been handling ad-hoc work as it comes in.
Projecting our velocity to the end of our release
We’re now about 60% of the way through our release so we have enough data to project ahead and see where we think we’ll land.
By projecting lines against our best and worst progress we can see the “cone of uncertainty” and make a good prediction on where we’ll be in February.
But we made a mistake in our projections
Now that we understand the mistake we made it sounds really obvious - but we made a significant error in our projection here.
Our worst case line told us that we’d be able to handle around 20 more work items before we hit the end of the release. In the best case between 30 and 45 work items.
Our mistake was that we simply projected lines forward to the end of the project without understanding our distribution of throughput and cycle time for each class of work.
What we found is that although we allocated 40%, 40% and 20% to two products and ad-hoc work our distribution was actually closer to 33/33/33.
This isn’t necessarily a big deal - but the fact that ad-hoc work is nearly always done a lot quicker than features means that our projection on how many features we have time to build was incorrect.
Remember the nice distribution of cycle time above - the nice bell shaped curve. We don’t have one of those.
Most of our work is done in a short cycle time and most of that quick work is ad-hoc. Because all of the work to the right of the chart is features we won’t get as many done in the remainder of the time as our cumulative flow chart seems to suggest.
OK - so what did I learn
Having different classes of service in Kanban is a really powerful tool and is something that differentiates it from Scrum.
We used classes of service to define how much effort we spend on each of our two products and to handle ad-hoc work effectively.
BUT - make sure that you separate feature work from ad-hoc work when you project forward to make predictions. Because we only had “Done Total” in green in our chart we weren’t being realistic about the number of features we’d complete in the remaining time. My prediction was actually 33% out which is a massive margin of error.
We’ve been back through our statistics to make a distinction between “Feature work done” and “Ad-hoc work done”.
The prediction is now more accurate but less exciting.
Oh well - This is still good news, I can now start to prioritise the features that are most valuable based on (now) accurate data way before the end of the project.
Gotta love predictability!