No More Deadlines: Tips for Healthy Project Estimation

I’ll never forget when project estimation was used against my team.

At the start of every project, my team and I went through the normal routine. The project lead scoped the project. We reviewed the plan. The team estimated when we would finish, and our product manager marked the date. Then the project’s scope shifted.

We discovered more work, but the original date never changed.

We worked hard to hit the deadline. We wanted to be trusted. When the team finished, the reward was more work, and the process started over.

Our project estimates were used to take advantage of us.

We never understood why our product managers kept the same date. Teammates questioned the point of estimates. We felt like the deadlines were arbitrary.

After being an engineering manager that’s helped companies reach IPO, I’ve seen the impact that estimates have outside my team.

So why is project estimation important and how can you learn to give better estimates?

Why is Project Estimation Important for Software Engineering?

Estimating software projects is an arbitrary task, but these dates are not for you.

Estimates are for the people that depend on you.

Let’s look at a product marketing team. This team produces resources for every feature release – blog posts, tutorials, etc. Every development team launches features at different times. Every team wants their marketing announcement immediately after release.

But the marketing team needs a schedule.

This team relies on your estimates to know when to prioritize your team’s new feature. Delays affect the marketing roadmap for the entire business.

So how can we create accurate project estimates that don’t overwhelm your team?

The trick is knowing when to share your estimates.

Embrace Changing Estimates at Different Project Phases

Estimates work best when we embrace their subjective nature. Most estimates are not set in stone. They can change as your project evolves.

So how do I embrace the adaptability of estimates?

How do Confidence Intervals affect Project Estimation?

First, I use a technique called confidence intervals. Confidence Intervals explain the accuracy of my estimates.

I learned about this technique from the wonderful book How to Measure Anything, by David Hubbard. Hubbard explains the confidence interval in the following way:

“…quantifying our current level of uncertainty is a key part of the statistical methods we are using…One method to express our uncertainty about a number is to think of it as a range of probable values. In statistics, a range that has a particular chance of containing the correct answer is called a confidence interval (CI). A 90% CI is a range that has a 90% chance of containing the correct answer.”
Excerpt From: Hubbard, Douglas W. “How to Measure Anything: Finding the Value of Intangibles in Business.”

I recommend How to Measure Anything if you want to improve your project estimation skills. It explains how to measure real-world tasks, the psychology of measurement, and many statistical methods you can use.

So how do I use Confidence Intervals in software project estimations?

Project Estimation at The Planning Phase

I see people go awry with project estimation at the start of projects. Many think their bosses are looking for the exact date of completion.

The assumption of accuracy is wrong.

Stakeholders are looking for a rough window of when they will see results. The planning stage’s estimates give insight on how to sequence the team’s roadmap.

At the start of a project, my estimates have a minimum confidence interval of 75%. My goal is to be somewhat accurate while acknowledging that unknown scope may exist.

Context matters. For most projects, a 75% confidence interval is enough. For projects that need more accuracy, I use two techniques:

Invest more time in upfront planning
Start the project and re-evaluate the estimate at a specific time.

The goal is to discover the information you need for a more accurate estimate. Talk with your team at this phase to understand how much precision is required. Figure out if this is a project to move fast or over-engineer.

Update Estimates Throughout the Software Project

We’ve figured out the project’s plan, accounted for contingencies, and given the perfect estimate. What happens next?

Our assumptions were wrong. A teammate gets sick. Stakeholders add a new feature.

The situation has evolved, which means the estimate must change.

To get ahead of this, I regularly update the project’s estimates. It’s a piece of data I can control throughout development. I incorporate the information of the current situation. I use the same confidence interval or better. As the project develops, I look to increase the confidence interval of my estimates.

When Should I Estimate a Release Date?

As the end of the project looms, about a month before release, estimates need to become more accurate. I shoot for dates that have a 95+% confidence interval. These are the dates that impact the teams that depend on you.

A trick I use is to not worry about the specific day. When a project nears completion, it’s ok if a last-minute issue props up. If marketing is delayed by a day or two, it’s not a big deal (most of the time).

Instead, I focus on getting the specific week of release right.

If my team is uncertain about how accurate the estimate needs to be, we talk about it. We evaluate the risk of a delay and how it will impact the product.

Project Estimation Techniques That Improve Accuracy

Experience is the best tool I’ve found to improve my estimation skills. Over time, I’ve created a library of projects I can use to gauge the effort of new work. But, I developed those past projects in a different context and team. Estimates from my past are wrong sometimes.

To mitigate past biases, I pay attention to the present. Here are a few techniques I use to improve accuracy.

Factor in Your Project’s Constraints

I need to understand the constraints to account for in estimates. These restrictions will affect the scope, speed, and stress of the team. Specific limitations I look for are:

Project complexity: how difficult will this project be for the team’s current skill set?
Effort: How much work is involved with each feature? How much automated testing should we perform?
Team Size: How many people are available to work on this project? Will the team grow over the project?
Deadlines: How much time do we have to develop?

Understanding these constraints leads to more accurate estimates. If you don’t have a full grasp of any constraint, learn from your team.

Learn Your Team’s Velocity

Knowing how much work your team can complete over a week is valuable information. The steps I use to learn my team’s velocity are:

Decide how to measure “effort.” Software development offers many ways to measure effort – points, tickets, etc. You just need a consistent unit.
Choose your agile sprint length. I’ve found one or two weeks works best.
Monitor your team’s velocity. At the end of every sprint, record how many units of effort were completed. Use a spreadsheet.

After a month or two, I have enough data to know how much work my team completes in an average sprint.

I’ve found that an average engineer completes about four project tickets a work week. This throughput assumes that all tickets are estimated to take two to three days to complete. For a project with 12 tickets, I estimate it will take one engineer about three weeks to complete.

Note: To clarify, the “two to three days of effort” is an estimate of effort. In reality, some tickets will take an hour to complete and others till take a full week. The actual cycle time is how I discovered engineers complete about four tickets per work week.

Don’t Assume Stability – Account for Chaos.

In all projects, I assume something surprising will happen. We need to account for this chaos in our estimates. While we could add buffers to our appraisals, it can be easy to call bullshit if the time is too long.

To be more data-driven, I run Monte Carlo simulations. These simulations use random choices from a data set to calculate how long a project will take. This randomness incorporates the chaotic events of a project. I’ve found the dates from Monte Carlo simulations accurate as long as the data you provide is sound.

I base my implementation of Monte Carlo simulations on Conal Scanlon‘s talk More Reliable Delivery with Monte Carlo & Mapping. I love this method because all he needed was a spreadsheet and a set of sprint velocities.

In practice, Monte Carlo simulations does well in estimating the week of completion.

Wrapping up

Estimating software projects should not be foreboding. They are a tool that helps an organization function.

To improve your team’s experience with estimating projects, remember the following:

Estimates are arbitrary! Embrace this and update them regularly.
Use Confidence Intervals to measure the accuracy of your estimates.
Understand all the constraints your team faces.
Account for chaos with Monte Carlo simulations.

These techniques have led to my teams having a healthy relationship with estimates. Sure, we don’t get them right all the time, but estimates have stopped being used against us.

How do you feel about project estimation? What are some techniques you use to create accurate project estimates? Let me know on Twitter or LinkedIn.

Before you go

Build the Stage is about teaching YOU how to be a successful leader.
If you learned something new from this essay, I’d appreciate it if you’d complete one of the following actions:

Share this essay: By sharing, you help others grow their skills. Save a click with the buttons below.

Tip a Coffee: Your support shows I’m helping you grow as a leader.
Give Feedback: Candid feedback is the most effective way to develop one’s skills. Send me an email to let me know how to improve Build the Stage.
Subscribe! If you’re not a part of the Build the Stage community, it’s never too late.