Will We Make It — Cars, Maps and Software Projects (Continuous Reforecasting)

How often do you forecast whether your project is going to finish on time or not? Once, at the beginning of the project? Every month? Every two weeks? Every day? How about every 15 minutes? For the teams I work with, we are forecasting our success probabilities every 15 minutes. Now, that might seem a bit of overkill, but we will get back to that a little later in this post. First, a real life anecdote about re-forecasting.

Through some planning and a lot of luck, work and home, for me, are in the same city. This means two things. First, on most days, my entire day is spent in a 2-mile radius. Second, almost all my driving during the week is city driving. Those two things are true on weekdays but not on Sundays. Playing in a regional sports league means traveling beyond my 2-mile comfort zone. It also means a good amount of driving on the highways.

My farthest drive is to a city on Florida’s east coast called Port St Lucie. It is almost exactly 100 miles from my home in the idyllic suburb of Weston, Florida. Recently on a Sunday morning, as I was preparing to leave, the dashboard display on the car told me that I had enough gas to travel 120 miles. The option was to either fill up now or on my way back from the ground so that I had enough gas to make the round-trip. I decided on the latter. Somehow, upon reaching my destination a hundred miles away, the projected miles the car could travel had changed from 120 to 98. I had traveled 100 miles, but only used gas required to travel 22 miles. From the outside, it might seem that either this is a miracle of physics or my car was lying to me when I started my journey.

It seems that the engineers at BMW (apart from creating some beautiful sports cars) have figured out what many project managers, scrum masters and team leads have had a hard time figuring out. It does not matter how good your initial estimate or expectation is, as you gain more information you have to adjust your projection and re-forecast. The computer in the car, as I was on my trip, discovered that I was burning fuel at a much different rate than I had during my recent trips. There were good reasons for this -

• City driving was replaced by highway driving, a lot less stop and go.
• I take full advantage of sports mode driving on a daily basis but decided not to do so as I was cutting the miles remaining really close to the trip distance.
• I usually drive in the city with the convertible top down, which increases wind resistance as opposed to driving on the highway when I usually keep the top up.

As the trip progressed and all these factors affected the consumption rate of the fuel, the car’s computer re-projected how many miles the fuel in the tank would last. Regardless of the initial estimate of the computer, it took the new information available into account and gradually adjusted its projections. Most likely, the projection algorithm was not altered, but the underlying data model used by the computer was. The projections were reactive to every mile traveled. Every bit of new information was consumed by the car to re-forecast the range the car would be able to cover.

The NOAA (National Oceanic and Atmospheric Association) does that same thing when forecasting hurricanes. They adjust the path that they project the hurricane would travel every time they receive new information. They don’t just issue a single line forecast when a hurricane is formed and leave it unaltered. In what is a matter of life and death, the NOAA, reacts to every new piece of information and re-adjusts their forecasts for the direction of the hurricane.

Mapping applications realize that their accuracy is greatly dependent on having the latest information and using it to update their predictions. The same is true when you are forecasting end dates for projects or releases.

The same is true for any “Maps” application that provides directions and estimated time of arrival. Google Maps, Waze, Uber, Lyft etc all reforecast continuously. Not only do they reforecast when they have new information they actively seek out information to make those forecasts. Mapping applications realize that their accuracy is greatly dependent on having the latest information and using it to update their predictions. The same is true when you are forecasting end dates for projects or releases.

Now, back to why we run forecasts for teams every 15 minutes. In an organization of 30 teams, the assumption is that one of those 30 teams has some new information that has caused them to change the scope or move a date. We want the result of those decisions to be available to the teams as soon as they make these changes. In fact, we want the changes to be visible to anyone who is interested, not just the team. That is why, just like BMW, with every mile that goes by, we want to be able to re-project and re-forecast our range of possibilities. New information shows up every day, all day and we should reforecast as soon as it shows up.

The feedback loop from the new information to actionable forecasts should be as small as possible…Forecast, early and forecast often

We want to know as early as possible when our initial assumptions have been invalidated. That is the crux of the problem. Our first projections have a lot of assumptions built into them. The more unstable our system and its rules(Sports mode vs comfort mode, Convertible top up vs down), the greater the chances that our assumptions are going to be invalidated as the project (or trip) makes progress. The variability in our systems that cause this are unavoidable, what is avoidable, though, is us turning a blind eye to the variability.

A large part of being Agile is building feedback loops. It is about seeking out new information and then reacting to it. Our systems (organizations, teams, cars, hurricanes etc.) are continuously providing us with new information. The feedback loop from the new information to actionable forecasts should be as small as possible. That is the major reason why we don’t wait two weeks or a month to re-forecast and instead do Continuous Reforecasting.

Our initial estimate, unless we are already running a very stable system (more on that another time), is going to be inaccurate. We, whether “we” means the team or the business, as long as we want to live in the real world, need to accept that. As we make progress, get new information and the cone of uncertainty begins to narrow, we will be able to make better forecasts. Re-Forecasting, when new information becomes available, is not just a good thing to do, but also the responsible thing to do. The same rules that apply to code check-ins — do them early and often, apply to forecasting as well. Forecast, early and forecast often.