Doing a project is like making an investment. A company does a project by investing engineering resources, hardware resources, operational resources, etc., in hope of (eventually) making more money than they initially invested.
As a project owner, managing your projects is just like managing your stocks. Before buying a stock (initializing a project), you need to decide if the stock is worth buying (if the project is worth doing). You only buy that stock if you believe that stock will have a positive return in the future, and the return should be large enough to worth the effort. You may plan to hold the stock for a period of time (plan the project duration), but if you estimate that you will lose money if you keep holding it, you will immediately sell the stock (abandon project) even though it hasn’t reached your financial goal. If you estimate that stock will give you more return in the future, you might want to hold the stock longer than you planned or to increase the investment (add more features to that project). Every decision stock investment is about the financial return.
Software project decisions can be made in a similar way. Essentially, you want to maximize the return, which is the difference between the gain (how much you have made) and the cost (how much you have invested):
return = gain — cost
Unlike the stock market, where you can always measure your return in dollars, measuring return in a software project is more complicated, because the gain and the cost are measured in different value metrics. For example, a project whose gain is to reduce p99 latency by 50% does not have same value metric as its engineering cost (e.g. one SDE working one month to release this project).
Here are some typical value metrics:
The challenge is that to be able to compare different value metrics, you need to project non-dollar value metrics to dollar value metric. For instance, figure out much 1ms latency is worth in dollars in your service. Some conversions are easy: you always know the SDE’s salary and the hardware cost; Some conversions are very difficult: how much does 1-mouse-click worth in dollars? Many companies use A/B tests to measure this.
Even though you can’t exactly project all non-dollar value metrics to dollar value metric, you at least should have a reasonable value range.
Because return is the difference between gain and cost, to maximize the return:
There are two common pitfalls when they make decisions in software projects:
In a project life-cycle, project return should be constantly re-evaluated, just like investors periodically evaluate the stock and estimate the return based on the new information.
While many projects conduct thorough initial scoping and planning, not many projects (especially smaller projects that span less than 6 months) actually do a regular re-estimating on return, while they regularly adjust the release dates. For those projects, my suggestion is that when adjusting the release dates, also re-estimate the project’s return.
Applying return-driven approach in project initialization phase, you need to answer this question: among all other candidate projects, can this project bring the highest return when it’s delivered?
Project scoping tries to answer this question by gathering information to estimate the gain and the cost. Talking to customers to understand their use case is a typical way to estimate the gain. Estimate the engineering cost and hardware cost is a typical way to estimate the cost.
When making a decision on whether to pick up the project, be aware of the pitfalls mentioned earlier: either gain and cost along can help you make the right decision, only return can.
If I want to see if a team can make rational decisions, one metric I would look at is how many projects have been canceled/abandoned in the design and implementation phases because the team does not believe this project could bring positive return (assuming the team already estimated that this project could have positive return in project initialization phase).
I find that developers rarely re-estimate the return after a project has been officially kicked off, while the leadership does that, but less frequently. It’s very common that, in the agile development, developers re-estimate the points they need to spend on different tasks in design and implementation phases. Often, they also need to adjust the project’s release date in the implementation phase. Nevertheless, it’s uncommon that developers frequently re-estimate the project’s return based on the new information they have learned in the design and implementation phases.
One of the famous fallacies in economics is sunk cost fallacy. While it happens to many inexperienced stock traders, I wonder if it happens more often to software projects, because developers can easily build personal bonds to projects when they invest a large amount of time and energy. I seriously doubt that a team working on a project for 3 years would have the guts to abandon this project if they found out that the future return will not be positive.
Even initially a project’s estimated return is positive, it could be negative later, when:
More situations can make the project’s return negative. Unfortunately, those situations are not or could not be anticipated in the project initialization phase. At this point, the team should evaluate the cost of abandoning this project and use return-driven approach to decide if it would be better to abandon this project.
After a project is completed, the team needs to maintain features delivered by this project. Depends on the design, implementation, architecture, or upstream/downstream services, a feature’s gain and cost will change constantly. Thus, a feature will generate either positive or negative return as the time goes by.
A regular return analysis should also be done after a project is completed. A return could be negative in either of two situations:
When the project’s return is negative and the analysis shows that it’s unlikely to be positive in the future, the team can:
It’s worth noting that deprecating a feature is not free. As any regular features, return-driven decision-making approach can also be applied to feature deprecation project. There are two types of cost in a feature deprecation project:
Sometimes the cost of deprecating the feature is so high that results a huge negative return. In this case, stopping on-boarding new customers and continuing supporting existing customers to confine the negative return is a better decision than deprecating this feature. This is especially true if you consider that hardware cost is always decreasing and the engineering cost is always increasing.
Return-driven decision-making approach can also be applied to multiple related projects (e.g. projects for one service) together. It’s not uncommon that a single project could have a negative return, but it will generate positive return when being combined with other projects later. When being applied to multiple related projects, return-driven approach can:
When you zoom out to estimate the return of multiple projects together, then zoom into a specific project, you will be able to estimate the return that this project brings to its related projects, in addition to the return generated by this project alone. It is a risk if you look at one project, but it could be a win for the service as a whole.
Return-driven approach is not useful when the gain and/or the cost cannot be reasonably estimated. This could happen in situations such as proposing long-term visions for a team or exploring new business domain, or new market.
The gain in those situations cannot be reasonably estimated because:
The cost in those situations cannot be reasonably estimated because cost, in most cases, depends on how to implement those visions and how to explore new business domain and new market.