How do you evaluate returns on a technology project?

If your job has anything to do with technology, you know that to detect value of a technology project is not an easy feat. Compare it with traditional investments. Say, you run a high-end used underwear store on madison avenue. You take orders on phone and to deliver the used underwear, you ride your bicycle. On average, you make 20
/ April 9, 2015

If your job has anything to do with technology, you know that to detect value of a technology project is not an easy feat.

Compare it with traditional investments. Say, you run a high-end used underwear store on madison avenue. You take orders on phone and to deliver the used underwear, you ride your bicycle. On average, you make 20 deliveries a day. For each sale, you make $20 in profit. Your used underwears are becoming popular in the the city.

On average, you receive 50 calls a day, but you only take 20 orders each day because you can only deliver so much on your cheap bicycle. You are thinking of investing in a used scooter for $2000. This will enable you to make 50 deliveries a day. But before you go ahead and buy a scooter, like a smart businessman you are, you want to evaluate this investment. Evaluation is relatively simple for this: The scooter will let you make 30 additional sales each day. So additional profit per day = 30 x $20 = $600. Going at this rate, you will recover the cost of scooter in three and a half days, and from that point on, you will be swimming in money — easy decision, based on an easy evaluation.

Now you decide to open up a website for your store. The developer is asking for $2000 to build the website. How do you evaluate returns on this investment? You cannot — at least, not easily. Yes, there is a lot of value in it. People searching for user underwear will end up on your website and call you and buy the stuff from you. But can you quantify it? Can you estimate how many calls you will get in a day on the basis of the existence of your website alone? No. But you still know there is value in building your website. That value or return is non-measurable, and it is called indirect return on investment. Most IT projects follow this rule of indirect return on investments.

Fast forward one year. You built a website for your business and your business is booming. You have made lots of money delivering your stuff on your scooter. Now you want to expand your business. You are thinking of adding package tracking service on your website, which will cost you $3000. Can you predict the monetary returns on this investment? No. But you know its a valuable addition to your used-underwear business empire. You know that if your customers can track the shipment, they will be happy. A satisfied customer will recommend you to others, which will further improve your income. Although you cannot measure and predict the monetary returns, you are convinced that this investment will bring you immense indirect value.

This is the nature of investing in IT projects. Often times, the value of a technology project is realized much longer after a project is completed. At the time the investment is made, those returns are out of one’s imagination. Case in point — when you first invested in building your website, you had no idea you would want to add tracking service to it later becuase it will increase the value you provide to your customers.

In business circles, there is a formal way to evaluate investments in IT projects. It is based on Garbage Can Model made popular by Cohen, March, and Olsen in 1972. Henry Lucas, a professor at Smith School of Business
at University of Maryland adapted the garbage can model for IT projects. According to Lucas, the overall probability of successful return on an IT project investment can be measured based on the type of the project and its conversion effectiveness. Conversion effectiveness is the probability at which an IT project is executed successfully.

Probability of success of a technology project = (Probability of Return on Investment) X (Probability of completing the project on time and budget.)

The probability of completing a project is also known as Conversion Effectiveness of a project

Now let’s plug in the numbers:

Probability of return on investment: What is the probability of returns on adding shipment tracking service on your website? Based on your experience and some guesswork, you think the probability of this feature to bring you returns is 0.7.

Converstion Effectiveness: It is a relatively straight-forward project, and easy to build and maintain. You are confident its conversion effectiveness is 0.9. Thus, the overall probability of successful return on this project is 0.7 x 0.9 = 0.63.

Based on some math, you anticipate adding the package tracking service for $3000 today will bring you a total of $20,000 in the next 5 years. The probability of success of project, which we calculated, is 0.63. Therefore, in reality, you will be making $20,000 x 0.63 = $12,600 in the next five years. Not a bad investment. And don’t forget the unexpected returns you may realize down the road, which is very typical of a technology application.

Now back to Garbage can model. The model says that all project ideas in an organization are funneled through a pipeline. The first checkpoint is conversion effectiveness. If the probability of converting the project is very low, that project should be immediately dismissed. In other words, that idea of building a new music streaming service sounds good, but if you don’t have the resources or the budget, there is no point considering the project. Now comes the probability of return based on the project type. The probability of return on a typical infrastructure is low, whereas probability of return on an e-commerce project is high. Take the two probabilities and multiply them to get the overall probability of successful return on the project.

Finally, multiply the anticipated cash returns with the overall probability and you will get the near-real future cash projections.