Last night I went to the London Kanban Coaching Exchange to listen to a talk by Nader Talai titled “Does size matter”. In his talk Nader told us a story of a team that gained trust with its stakeholders through improved estimation.
Our journey with Agile estimation…
Personally I have been through an Agile estimation journey with my team over the last year. In our work before last August my team ran a fairly vanilla Scrum routine, meeting three times a sprint for an hour to discuss and size the top of the backlog.
For us, at that time, estimation was a useful practice and we felt we were accurate enough. Taking work that we’d recently completed and comparing upcoming work against it; asking “is upcoming feature X bigger or smaller than completed feature Y" gave us a good sense of what we could take on.
At that time, during sprint planning we’d take our targeted stories and plan in detail assigning an estimate of hours against each task. To be fair - we were often wildly inaccurate at estimating the hourly tasks but it all held together and we enjoyed good trust with our stakeholders.
Moving from Scrum to Kanban…
In August we made a few changes. Firstly we started working on new projects for new stakeholders. Secondly I had read a lot of debate about #NoEstimates and wanted to remove a lot of the overhead associated with estimating - especially estimating tasks in hours where we were failing to improve. Lastly - for business reasons - it made sense for us to become very familiar with the mechanics of Kanban rather than Scrum.
So between August and January we shifted our routine to Kanban. I blogged a fair amount about our experiences if you missed it and we felt fairly successful in that transition. Certainly we felt predictable in our delivery even though we’d stopped estimating work in either story points or hours.
Additionally, by tracking the cycle time of each user story we had a good basis for Kaizen - identifying the elapsed time from accepting the work to it being completed and where work had stopped in the process.
We were able to point to our cumulative flow chart and say with confidence "By the end of the release we’ll be here" or "Based on our previous data the next feature will be delivered in 12 days (with 80% certainty)". Those probabilistic predictions felt a lot more stable than using abstract points to project into the future.
But we lost something by not estimating work…
At the moment we are planning our next release which will run over a number of months. The business drivers for using a strict Kanban methodology have changed so we have some free reign to adopt our process.
In our release retrospective recently we agreed that striving for continuous flow and moving away from iteration goals meant we lost some valuable interactions in the team.
Some members missed the focus that a sprint goal and commitment and the two week timebox gave them.
Others missed the planning and estimating meetings where they had the opportunity to discuss and get a shared understanding of the work. For them the practice of estimating was more valuable than the outcome of the estimating session.
The evils of estimation
Last nights talk by Nader crystallised some thoughts for me. When we moved away from Agile estimates in our last release we realised the value of cycle time and the Lean method of identifying waste in our process.
Holding up a “13 point estimate” as the only measurement of size abstracts a lot of evils that might lurk in the teams work.
Firstly if teams feel under pressure to meet a certain velocity (lets say 26 points a sprint) it’s too easy to buckle and start to play with the numbers.
"OK, lets take this feature and call it 13 points even though last time we called it an 8. We need to keep our velocity up"
"We’re 2 days away from the end of the sprint, lets make sure we get this feature done quickly to keep our velocity up"
Neither of these patterns are helpful. By making the work bigger you mask a lot of inefficiencies that will follow. If the developer can actually complete the work faster because he over-estimated to make the velocity report look good will he gold-plate the feature to use up the remaining time? If the team finish early and bring in more work does that make the velocity chart more accurate? Of course not.
By rushing to complete work before the end of the sprint, normally an arbitrary date anyway, only quality can suffer. Why rush to complete a feature in order to satisfy the velocity chart if you have to then rework it later or ship it with low quality.
How about this one from the Product Owner
"You can do 26 points in a sprint right? So I can trade these two 13 point features for twenty-six 1 point features??"
Of course not.
How we are going to use estimates
Despite the above I believe estimates to be helpful. They only go bad when the output is used as a management tool and especially when that management exists outside of the team.
Here is how we are going to use estimates in the next release.
- Use story points as a planning guide but use cycle time for predictability, reporting and kaizen
Estimating in T-Shirt size or story points is a useful activity because it encourages conversations about features and promotes shared understanding of what is complex (big).
It also identifies features which are candidates for decomposition so they can flow easily through the process.
To track predictability cycle time (how long a feature takes to complete) is a much more powerful metric. It provides a guarantee to stakeholders with an accompanying range (“We can provide a feature in 12 days (with 80% predictability)” and that is backed up with concrete data.
- Always remember that the value in estimating is the activity, not the output
Taking time to estimate throughout the release promotes discussion and highlights where team members have different interpretations of the requirements.
If one member estimates a feature at 1 point and another member at 13 points… you’ve just realised the value of estimates. That misunderstanding needs to be straightened out before development starts.
- Don’t drive the team to get better at estimating
Mike Cohn has the following model in his book “Agile Estimating and Planning”
I like it - it shows that the accuracy of estimates increases dramatically with some low amount of effort but will never reach 100% accuracy.
And actually trying to improve the accuracy of estimates requires significantly more effort and actually ends up reducing accuracy as you gather more and more data.
Teams will continue to be inaccurate in their estimates - thats fine and expected. Your predictability and trust with stakeholders should be built using probabilistic data (cycle time and cumulative flow) leaving estimates to be a team activity and metric.
Thanks Nader for a thought provoking talk