No other term is more overused in the business world than Return on Investment (ROI), except perhaps “The Bottom Line.” Armies of well-meaning managers throw out superfluous business jargon like ROI to demonstrate they are good stewards of corporate finances. ROI was originally used to evaluate financial transactions by calculating the return of profit versus the capital investment. Over time, ROI has become the de facto measure used to evaluate one-time capital projects. When it comes to software, ROI is often a fantasy number used to make a decision maker to feel good about a decision.
ROI has been broadly applied to information technology projects. Often, ROI is used to decide if the investment in a software system is worth the return. Let’s take a look at ROI when applied to custom software development. In financial speak, software development costs are aggregated and booked in an accounting system as a capital expenditure. However, software development costs are an expense item on the P&L (Profit & Loss Statement), not a Balance Sheet item. The reality is that until the software is complete, it is not a capital item, it is just a collection of expenses. The question is: When is the software done?
Enter Technical Debt
The drive to “call” a piece of software complete leads to decisions that incur technical debt, which ultimately your organization will have to pay and drives up the true cost of the software. C-Level executives and middle managers should be more concerned with Technical Debt. Wikipedia defines technical debt as:
“
Technical debt (also known as design debt[1] or code debt) is a recent metaphor referring to the eventual consequences of any system design, software architecture or software development within a codebase. The debt can be thought of as work that needs to be done before a particular job can be considered complete or proper. If the debt is not repaid, then it will keep on accumulating interest, making it hard to implement changes later on. Unaddressed technical debt increases software entropy.” -
Technical Debt Wikipedia 2015
Technical debt, while not a quantitative calculation, is of bigger concern than ROI.
Understanding the "Bogacity" of ROI
First, to understand why ROI is a bogus measure and technical debt is more important we need to break down ROI. Consider the formula for ROI:
Figuring out the cost of developing software is difficult. Software is inherently complex. Software complexity is hidden in human interaction, systems that have different interfaces, differences in technology, algorithms, and the 1s and 0s that ultimately tell the computer what to do. The human factors in software are complex enough alone, without the myriad of other considerations. Having a good software vendor, one who doesn’t just want to sell to you, but rather is highly skilled and looking to help you with a solution is paramount to successful software projects. On the financial side, not only do you have to consider the cost of development, you have to consider operations and maintenance costs as well. These two things comprise your cost of investment.
The most ambiguous and most subjective part of calculating ROI is figuring the gain from the investment. Usually, building a piece of software is meant to increase productivity, replace an existing system, or achieve some efficiency. In most cases, the person doing the calculation figures the number of hours and costs per hour of doing some activity without the software. By comparison, they then estimate how much time, or money that building and implementing the new software will save. The problem is that this number is subjective and can be used to skew the results in favor of achieving or not achieving an ROI. Most organizations do not have the data necessary to accurately determine productivity. Therefore, their estimates are marred by personal bias. This effectively makes ROI a useless measure. There are those in the C-Suite that swear by ROI calculations because of their financial training during their MBAs, but with technology ROI is fleeting. When it comes to a financial return, unless you have hard numbers on productivity such as those that might be derived from manufacturing equipment that counts real widgets – the ROI number is unlikely to provide a business any more information than an Ouija Board. What ROI will do for you is help identify where you might get a return. This will allow you to focus on which areas or features of the software might be more important than others. For example, if you think the return may come from reducing data errors based on manual data entry, then you might consider making features like data validation or drop lists a priority feature of the system.
Assumptions: Making an Ass Out of Me & You
Most companies use an enterprise architect or software engineer to design a software application. That architect will make many decisions and many assumptions depending on the time and depth they are given to create the design and cost of a system. Most companies do not allocate enough time to the design and analysis phases. Worse yet, in today’s fast-paced business environment developers are actively engaged in Agile Development and are driven to build minimum viable products (MVP). This rushed delivery model can cause them to believe incorrectly that MVP means they should jump in and code without planning or that the software that they will deliver does not have to be “fully baked.”
These misconceptions and tendencies to forgo analysis and design lead to decisions that can incur a technical debt with compounding accrued interest. This kind of serious debt would bring a smile to the face of the biggest mafia loan shark, and bring the world’s best CFOs to tears. So how much design and analysis is necessary? Like all things in information technology, “It Depends.”
How Much Design & Analysis is Enough?
This is a topic that is fiercely debated in the software development community. In the late 90s and early 2000s, an IT Industry Research company called Meta Group tracked data for IT projects across industries around the world. They reported that data in an annual report called "Meta Group - IT World Benchmark Report." The following is a breakdown of the software development lifecycle (SDLC) based on that data.
Notice that 37% of time spent on a software project is in the requirements definition, analysis, and design phases. Only 23% of the time is actual coding time. There are those in the QA field that would tell you that 16.5% for testing is low for today’s browser-centric mobile world.
The less time an architect spends on the requirements gathering, analysis, and design phases means he has to make more assumptions. The more decisions made “along the way” instead of being planned out – the more potential there is for technical debt. Because of the complexity of software, those assumptions and tradeoffs amplify technical debt. The Standish Group, who monitors and collects data on software projects, reports that the average cost overrun of software projects is as high as 214%. Then if you factor in technical debt – which is often offloaded to operations and maintenance – over time, can lead to a 6X multiple of that percentage!
Will Fixed-Price Projects Save Me?
Many companies want fixed-price contracts when it comes to developing software. Unfortunately, this is yet another way to incur even greater technical debt. Sadly, many companies are rarely willing to make the investment to properly analyze, scope and design a piece of software. The time to properly analyze, scope, and design a piece of software requires skill and a deep understanding of both the software solution and the business need. That skill and effort are not free. They come at a cost, but it can save you in the long run. Without an adequate investment into these activities, a software architect is forced to make even greater assumptions. More assumptions = greater technical debt.
"The Bottom Line" (see what I did there)
The bottom line for the C-Suite and middle managers who are looking to software solutions for new efficiencies; is that they need to depend less on ROI analysis and more on defining clear requirements, and allocating enough budget for the analysis and design phases of their software projects. This alone won’t eliminate technical debt, but it will reduce it. Failure to understand and heed the perils of technical debt will result in your company becoming upside down and wondering why you are pouring money down a black hole.