How To Do More Reliable Estimations

The success of your tech business depends on it

Estimations are part of our everyday life and most of the decisions that we make. Estimations are especially important when it comes to running a viable business, in particular when you’re a tech company developing one or more software products.

How reliable are the estimates for your software project? How often do you deliver on time and within budget?

From “how much will it cost?” to “how long will it take to deliver?”, tech companies constantly require estimates to make decisions with any software project. But it is rare for tech companies to quantify and measure the reliability of their estimates. The reason for this is due to the many variables and unknowns, in addition to the information available or lack of at different points in time throughout the software development process to make one or more decisions. As a result, uncertainty is the norm due to the sparsity of information (eg. all we have is an idea and we want to know if it is worth spending a certain amount of dollars to pursue the opportunity?). So for those developing software, this begs the question:

How many hundreds of thousands, millions or billions are your current estimations methods costing your organisation?

This question is highly relevant for a few key reasons. Firstly, if your estimations are being misinterpreted and/or miscommunicated, that’s money down the drain. Secondly, if your team cannot produce consistent estimations, how can your organisation reliably compare budget and time estimates across projects? Again, more money down the drain. Thirdly, if you’re producing estimates that aren’t really anything more than an “estimation theatre”, that’s both time and money wasted because it could have been spent contributing to obtaining a bigger advantage for your business if your estimations were more reliable.

Whilst it may seem easy to produce an estimate, research studies show that producing reliable estimates are hard. In fact, we are naturally good at lying about estimates. But yet none of our estimation techniques used today help us in preventing the lies. Instead, our estimation techniques are inaccurate, inconsistent and easily misinterpreted by people of all levels within an organisation. This is a significant weakness in the way we develop software and its costing companies tens of thousands of dollars, if not more, as we speak.

The Problem

We live in a world where technology is woven into pretty much every aspect of our lives. Thus, the way we develop software is becoming more and more competitive. In some ways, technology is also becoming more and more challenging with the way people are incorporating it into their daily lives and business. And yet, we don’t take software estimations serious enough.

“There wasn’t a single distinct method that most people used. Instead there were lots of different methods.” — Product Habits

The fact that there are so many different methods is a problem in itself. Could it be because software experts did not know that there was a risk element to estimations that they needed to be educated about? Nonetheless, the complexity of the estimations problem has a few different components to it:

Certainty is expensive

The ultimate estimation would be to just do the software project itself. We would then know would know exactly how much it cost and how long it would take. However, this is neither feasible nor affordable as shown by Barry Boehm’s cone of uncertainty — a project is highly uncertain at the start and the uncertainty reduces only as the project progresses.

Estimate, not guesstimate

Guesstimating is the riskiest thing we can do when it comes making decisions in software development. Surprisingly, we do this on a daily basis, often without realising, which is the more worrisome part. I’ve had the opportunity to observe this in almost every company that I’ve consulted for and worked at. From my observations, this happens because we cannot instantly see or feel the implications of poor estimations. Similarly, we have no way of easily measuring the cost of producing poor quality estimations.

The “estimation theatre”

We do not have a way to easily detect, imagine and articulate the problems and shortcomings associated with the current quality of our estimations produced today until its too late (eg. the software project fails). There is often a significant amount of time that passes between when the estimate was first provided and when we expected to realise some outcome, delivery of value or failure of it.

Dangers of denial

We’re human and we love being right. What we love even more, is telling ourselves stories that we want or need to believe. This is usually because we believe that we are highly confident in what we know or have a strong desire for a particular outcome because something important, like a job promotion or bonus, depends on it. However, not recognising the implications of our cognitive bias and irrational thinking will have negative impacts on the reliability of the software estimations produced.

In summary, the root cause of this problem is that our expectations of software estimations do not match reality. We do not recognise the uncertainty involved in software development as a form of risk analysis. Yet, the two most popular questions we ask with every software project (eg. “how much will it cost?” and “how long will it take?”) plays an influential role in whether an investment of both time and money is will be made or not. So what if we took a more disciplined and scientific approach to software estimations? Tech companies could save hundreds of thousands and potentially millions, if their decisions were based on more honest, accurate and credible estimates. Let’s take a look at how to turn those useless estimations into something more useful.

4 Simple Steps To More Reliable Estimations

There is no shortage of evidence that software estimations require a little more effort than brute force, a quick tutorial or reading a few articles. The facts show that estimations are hard. Walk into any tech company and I’m pretty sure that there has been at least one project that was delivered late, went over budget or failed in the past year. That is because software development is innately problem solving.

To start producing smarter estimations in a tech business, we need to first acknowledge that every estimation is in fact a form of risk analysis. Without this, you will not be able to successfully determine the appropriate technique for the estimations. If you accept (or are open to accepting) that software estimations are a form of risk analysis, please continue reading on how to do smarter estimations.

Step 1: Produce estimates as a range

If you do this step and nothing else, you’re already making significant progress because providing estimates as a range instead of a single value or abstract point (eg. S, M, or L) is more accurate. More specifically, a single value is mathematically associated to a much smaller confidence level than a range range of values and abstract points are easily misinterpreted because “Medium” likely translates to different numerical values for different people. Whether its estimating time or money, all software estimations should be made as a range.

• For example, instead of labelling some piece of work “M” for “Medium” or saying that it would take approximately 3 days to do, provide your estimate as a range, eg. “2 to 4 days”.

Step 2: Estimate with 90% confidence

When providing an estimate range, you should be 90% confident about it. An easy way to think about it is if you produced multiple estimations, you should be right 90% of the time. If you end up being right more or less than 90% of the time, this is because you are either overestimating and underestimating and need to adjust the estimation range that you’ve provided so that the range reflects you being right 90% of the time. Note that estimating with 90% confidence does take some practice and calibration.

• For example, with your next estimate keep track of the actual result compared to your estimate. From there, utilise that learning to provide a wider range in the case your previous estimate was under confident or narrower estimate range in the case your previous estimate was overconfident.

Step 3: Determine probabilities for the estimation

In the tech business, guarantees are hard to come by. Single value estimates are also more susceptible to being turned into a promise or commitment and have a higher risk associated to them. Hence, producing software estimations as a range with corresponding probabilities is more honest, and thus reliable, due to the uncertain nature of developing software.

• For example, using Monte Carlo Simulations enables you to determine probabilities for software estimations, specifically the probability that a project will be delivered within a particular time range. Simply collect time range estimates for each story/task in the upcoming sprint, from each team member, in either a Google Sheet or Excel spreadsheet. Then aggregate each person’s estimates and apply the Monte Carlo technique to create a probability distribution for an estimate of the project’s duration. From this distribution, you can assign probabilities to corresponding estimate ranges.

Step 4: Bring your estimates to life

Estimations should continuously be updated and always visible, otherwise they are useless. We are continuously learning new information each day (eg. how much a project has or has not progressed, how much we spent on a project today, etc) so for optimal transparency of a project’s progress, the original time and cost estimates should be frequently updated as we learn this new information to get the most accurate possible view of where the project is headed.

• Example Action: At daily stand ups, share any learnings and how they have influenced your original time/budget estimations, share an updated estimate and explain how this could affect your deliverables. As a manager, be sure to also aggregate the updated estimates from the relevant team members in order to get an updated holistic view of how the project is performing in terms of money and time spent to date.

With these steps, you and your team are now ready to start producing smarter and more reliable software project estimations.

Make Estimations A Breeze Every Time

The key to making every estimation a breeze from this point forward to is produce reliable estimations. Zat Rana recently wrote an article about The Two Types of Knowledge (Or How To Be Smart) which stated:

“Fortunately, evolution has also programmed us with the ability to learn. With a mind that experiments, predicts, and corrects, we can build empirical knowledge to adapt us to other relevant environments.”

This statement can also be applied to the way that we think about software project estimations. Today, we are open to learning as demonstrated by the widespread adoption of Agile methodologies, such as Planning Poker and T-shirt Sizes, and “test and learn” mindset. We also make an attempt to predict, correct and “build empirical knowledge” but, in reality, we don’t get far on this part. The Agile methodologies hardly produce empirical knowledge and on top of that, we neither use or evolve this knowledge in practice. We fall short on taking the existing knowledge that we do have and keeping it up-to-date so that it is useful. This is often because it is too cumbersome and many of us lack the knowledge of how to do step 3 and 4 above well. The good news is that I’m working currently with a few people to make the 4 steps from above quick and easy to use. Here are a few concepts if what is coming soon:

Seamlessly collect unbiased estimates

Trello Power Up — Seamlessly enter estimate ranges to any card

Each team member can only see the estimates that they entered so that they do not bias their peer’s estimations. However, everyone can see who has provided an estimate on a task/issue/story and who still needs to provide an estimate to encourage transparency across the team.

Reliable estimation reports within seconds

Criticide’s project estimation report

Criticide produces honest and more accurate estimation reports that make communication easier. These reports are effortless to keep up-to-date as well when any team member updates their estimates from new learnings and/or discoveries throughout the project execution.

We make it effortless to keep estimates up-to-date

Trello Power Up — Easily update any existing estimate

We learn and discover new information during the execution a project. Criticide makes it effortless for individuals to update their estimates. More importantly, it only takes one-click to update any changes to a project’s estimations and obtain an updated holistic view.

What are you doing to improve the predictability of producing successful technology products?

Want early access?

Sign up to get notified when our JIRA app and Trello Power Up is ready. Free yourself from the pain of estimations today;)

Curious to find out more? Email us at [email protected] or follow us to receive notifications on upcoming articles.

Learned something? Hold that 👏 to say “thanks!” and help others find this article.

Do you want to read more? Follow me or read more here.

You may also like:

One-Off Human Estimate (blue) vs. Monte Carlo Estimations (red)

How To Improve Estimations With Monte Carlo Simulations

A comprehensive way to start producing more accurate estimates. Coming soon! Follow Critically Deciding to get notified.