A Comprehensive Guide to Effective Project Estimation

Project estimation is a critical aspect of successful project management. Accurate estimation ensures that projects are completed on time, within budget, and with the desired quality. However, project estimation can be a complex and challenging process, often leading to deviations from the initial estimates that entail budget creep. In this comprehensive guide, I'll explore effective strategies for mastering project estimation. We'll delve into why projects frequently take longer than estimated and how to address this issue for both project teams and business owners.

A staggering 91.5% of projects go over budget, over schedule, or both

Project overruns are a common phenomenon across various industries.

Let's explore some illuminating examples of how large-scale projects went off the tracks:

The book titled "How Big Things Get Done: The Surprising Factors That Determine the Fate of Every Project, from Home Renovations to Space Exploration and Everything In Between" by Bent Flyvbjerg and Dan Gardner highlights a common trend in project management. It emphasizes that the majority of projects tend to deviate from their original plans.

In particular, nuclear power construction and IT projects are notorious for their propensity to encounter disastrous and highly expensive setbacks. These examples represent just a few instances within the broader category of megaprojects, which the authors, Bent Flyvbjerg and Dan Gardner, define as projects with costs exceeding US$1 billion. Megaprojects, regardless of their nature, often experience significant budget overruns and delays, making them a cause for concern for business leaders, politicians, and organizations.

Bent Flyvbjerg has compiled an extensive database on this topic, encompassing more than 16,000 projects spanning over 20 fields across 136 countries, with records dating back to 1910. These projects range from Olympic games to nuclear power plants, including complex endeavors like NASA's James Webb Space Telescope, which exceeded its budget by 450%. Shockingly, the research reveals that a staggering 91.5% of projects face either budget overruns, delays, or both. Moreover, less than 1% of projects manage to adhere to their original timelines and budgets while delivering the promised benefits.

On average, projects experience a cost overrun of 62%. The distribution of these megaproject overruns exhibits a heavy tail, signifying the presence of catastrophic risks. To illustrate further, the authors point out that 18% of IT projects encounter cost overruns exceeding 50% in real terms, with an average overrun of 447%.

These examples underscore the complexity of project estimation and the frequent occurrence of projects exceeding their initial estimates. However, it's crucial to understand that accurate estimation is attainable with the right approach, and by accurate, I mean an estimate with minimum possible deviation.

The Two Sides of Project Estimation

Project estimation is a delicate balancing act, requiring collaboration and understanding between the development team and the client or business owner. While the previous sections discussed the challenges faced by development teams and strategies for accurate estimation, it's equally crucial to consider the client's perspective. From the client/business owner's point of view, there are compelling reasons to add a buffer to the project estimate. Let's explore why this is essential and highlight scenarios where the initial project scope can be exceeded.

The Client's Dilemma: Why Buffer Matters

Clients and business owners have their own set of concerns when it comes to project estimation. They seek to achieve their objectives within a specified budget and timeframe, but they also understand that changes and uncertainties are inherent in any project. Here's why adding a buffer to the project estimate is vital from the client's viewpoint:

1. Evolving Design Requirements:

Studies in project management have shown that design iterations are a common occurrence in project development. A client may have a vision for the project, but it often takes several iterations before the design aligns perfectly with their expectations. These design changes, while necessary for a successful project, can impact the project's timeline and budget.

2. Comprehensive Product Testing:

Thorough testing is a fundamental part of project development to ensure the final product meets quality standards. However, extensive testing can reveal issues that were not initially anticipated. Addressing these issues may require additional time and resources, potentially extending the project's duration.

3. User Testing and Feedback:

Incorporating real user feedback is essential for creating a user-friendly and effective product. Pilot groups and user testing often reveal usability challenges, logic gaps, or areas for improvement that were not evident during the initial planning stages. Adjustments based on user feedback can result in scope changes and project extensions.

4. Regulatory Compliance:

For projects in regulated industries such as healthcare or finance, adhering to compliance standards is non-negotiable. Navigating regulatory requirements can introduce complexities and additional tasks that were not initially factored into the project plan.

Strategies for Effective Project Estimation

In this section, we delve into strategies for effective project estimation. We won't be covering the well-known practices that I am sure are already applied in your approach, such as proper preparation before committing to a project, working in areas of expertise or acquiring that expertise, breaking down work into smaller chunks (the Lego principle), using sprint methodologies and etc... Instead, I'll provide you with insights that go beyond the surface, offering practical approaches drawn from real-life experiences to help you tackle project estimation from a more detailed and low-level perspective, ultimately securing better results.

At Uinno, we employ a dual-point estimation approach for our estimations. This practice is founded on two key pillars: optimistic and realistic estimates. These serve as powerful tools for effectively navigating the uncertainties and risks inherent in the estimation process.

Below are detailed descriptions and examples of how we interpret and apply both optimistic and realistic scenarios:

Optimistic Estimate (O)

This is the "sunshine and rainbows" scenario. It's when all the stars align, the coffee tastes just right, and the Individual Contributor (IC) — that's you — is bouncing off the walls with an energy that puts the Energizer Bunny to shame.

Here's a real-life scenario. You're tasked with implementing authentication functionality with the Awesome Auth Provider, and you've never worked with it before.

Under our rosy optimistic scenario, the Awesome Auth Provider's manual looks like it's been crafted by angels — updated, user-friendly, and reads like your favorite novel. You might even get an easy-peasy development playground and a bunch of examples to play with. On this day, the birds are singing, you're fresh from a restful sleep, and you're so upbeat you feel like you can finish it without breaking a sweat.

So you estimate this task to be finished within a single business day (8 hours).

Realistic Estimate (R)

This is the “Murphy’s Law” scenario where everything that can go wrong, does go wrong. Coffee's bitter, traffic's a nightmare, and you feel like you’ve been run over by a bus.

Take the same task — implementing authentication functionality with the Awesome Auth Provider.

In our not-so-sunny Realistic scenario, the Awesome Auth Provider's manual looks like it was written by a toddler — confusing, outdated, and features a dreaded XML API that could make grown developers weep. You've had a mere four hours of sleep, and there's a strong suspicion you're coming down with something. The thought of coding for more than two hours feels like a Herculean task, and you need a 15-20-minute siesta after each session. Well, life’s not all sunshine and unicorns, right?

In this case, you feel like two business days of work (16 hours) is probably a better and more realistic guess.

Balancing out the estimate. What it really takes to deliver work

A common pitfall in estimations is the tendency to consider only the pure development time, without taking into account that additional factors will inevitably influence the actual feature delivery timeframe.

In essence, to generate a more accurate and realistic estimation, it's necessary to expand the initial timeline to accommodate for these influencing variables. The act of coding is merely one piece of a larger puzzle that needs to be addressed when formulating an estimation.

Let's see what are the other important parts you need to consider:

Communication (С)

Communication forms an integral part of every project, whether it be with the Project Manager, teammates, or other stakeholders. The number of individuals involved in the project or feature development typically correlates with the amount of communication required. It's a complex task to accurately anticipate the time required for communication, as it can vary considerably based on factors such as the project's pace, the meeting schedule, and the overall complexity of the project. Thus, it's crucial to take this into account when formulating an estimation.

But as a starting point, keep in mind that at least an hour of your day is spent on some sort of daily meeting, retro or communication with your teammates.

Testing (T)

Regardless of its apparent simplicity, each piece of work needs to be rigorously tested before it's deployed in either a production or development environment. Even in the absence of a dedicated Quality Assurance Engineer, Individual Contributors are still required to ensure compliance with the acceptance criteria.

This additional process significantly contributes to the overall estimation. It's essential to remember this critical step while formulating time estimations, as it ultimately guarantees the delivery of a high-quality, functional product.

Deployment (D)

A task isn't considered successfully completed in isolation, nor is it merely a piece of code on a local machine or git repository. Just as a well-crafted piece of machinery is of little use if left unassembled, so too does a task reach its full potential only when it's been deployed to the appropriate environment or at the very least, integrated with the main body of the application code.

Remember, in the world of software development, a task should be considered done only when it becomes a functioning part of the greater system.

Amendments (A)

It's important to remember that the phases of testing and deployment can often be akin to opening Pandora's box of bugs. Therefore, the time allocation for bug fixing is an essential factor that should be incorporated into the general estimation.

Environment (E)

Despite our best efforts, none of us are machines capable of working an uninterrupted eight-hour shift without catering to life's necessities. Be it a much-needed coffee break, a trip to the restroom, or a quick chat with a colleague, these moments are a reality of the workday. Especially when a task is projected to take around eight hours or more, these "real-life" interruptions need to be factored into the overall estimation.

Key Takeaways for Mastering Project Estimation

Project estimation is a multifaceted endeavor that plays a pivotal role in the success of any project. Given that a staggering 91.5% of projects go over budget or schedule, mastering this skill is non-negotiable. Here's a concise step-by-step guide to enhance your project estimation skills:

1. Acknowledge the Complexity

Understand that project estimation is not just about development time. It involves various other factors like design iterations, testing, user feedback, and regulatory compliance.

2. Employ Dual-Point Estimation

Use both optimistic (O) and realistic (R) scenarios to navigate uncertainties. The optimistic estimate is your best-case scenario, while the realistic one prepares you for potential setbacks.

3. Factor in Additional Variables

Don't just consider pure development time. Account for:

[CADDET]

• Communication ( C)
• Amendments (ex Fixes) (A)
• Deployment (D)
• Development (D)
• Environment (ex Life) (E)
• Testing (T)

4. Add a Buffer

Especially from the client's perspective, adding a buffer is crucial to accommodate evolving design requirements, comprehensive product testing, user feedback, and regulatory compliance.

5. Continual Learning and Adaptation

Use past projects as a learning experience to refine future estimates. Keep track of where your estimates were accurate and where they fell short to continually improve your estimation skills.

By incorporating these steps into your project estimation process, you'll be better equipped to deliver projects on time and within budget, thereby increasing your success rate significantly.

Conclusion

Project overruns are a reality, but they are not insurmountable. Accurate project estimation is attainable by adopting a comprehensive approach that considers both optimistic and realistic scenarios and accounts for influencing factors. Understanding the challenges of project estimation and learning from past experiences can lead to improved project planning and efficiency.

It is also a collaborative approach. A collaborative approach is essential to address these challenges and maintain a positive working relationship between the development team and the client/business owner. Clients should be encouraged to include a reasonable buffer in their project budget and timeline to accommodate potential scope changes and unforeseen challenges. At the same time, development teams must strive for accurate estimation and transparent communication to minimize surprises.