Picture a dark moonless starry night filled with the infinite lights. When looking at a sky like that, you realized just how small and insignificant you are. Despite your insignificance, have you pondered how just how complex and unique you are?
You are a multitude of thoughts and feelings. Your body is an intricate biological machine, working in a perfectly orchestrated concert. Composed of hundreds of muscles, hundreds of bones, thousands of nerves, and 7*1027 atoms. That is a 7 followed by 27 zeros. Think about it. It is mind boggling! You are but one complex entity in an infinite universe of complex entities.
Like the universe, you, as insignificant as you are in comparison, are a complex system. A first glance, you appear simple. But you are unique and, unlike any other human being. Inside you are a complex mesh of thought, emotion, and biology. You are like other human beings, but you are not the same.
Software Is a Universe in Itself
Like you, each piece of software is its own universe. The nature of any software system is complexity. Outwardly, software looks simple. Internally, it is a unique universe of unlimited complexity. Software systems may have similar components such as a database or a user interface. But each software system is unique. All Software is Complex. It does not matter if it is a mobile application, a web site, an integration, or a line of business (LOB) application used in a Fortune 2000 company.
It is the job of the software engineer, to engineer the illusion of simplicity. Consider the cartoon below.
An ATM is a Simple Concept. Or is It?
An ATM is something everybody understands. You put your ATM Card in. You enter a pin and select a withdrawal amount. The system checks your balance and distributes your money. It then gives you a receipt and returns your card. That was just one transaction.
How does it change that transaction if:
- You check your balance first.
- You do not have sufficient funds.
- The ATM cannot communicate with the bank.
- You cancel the transaction.
- The ATM is not stocked with paper.
- The ATM is out of money.
The software that runs the ATM is just one component of a larger system. The ATM is an electro-mechanical machine. That machine connects to an intricate and complex computer network. It also connects to a sophisticated communications infrastructure, similar to the one in the picture below.
Complexity & Simplification
When you engage a software development firm, you are asking them to create an original software system. What is a Software System?
These are software systems:
- Web Application
- Client Server Application
- Mobile Application
- Integrating One or More Software Systems
- Line of Business (LOB) Application
Most software engineering companies do not engineer software from scratch. They use off the shelf components.
These are some off the shelf components:
- Application Programming Interfaces(API)
- Software Frameworks
- Software Development Kits (SDKs)
- Web Services
- Software Libraries
- Software Applications
Software components reduce the amount of code written. They are building blocks, used in creating software solutions. This reduces development costs and often reduces complexity. Developers use these components to create the illusion of simplicity. Simplicity achieved while solving your business problem and meeting business needs. To visualize, consider the complexity of a software system, as illustrated in the diagram below.
Creating a software system requires planning, designing, engineering, and testing. The leading cause of delayed, stalled, and failed software projects isn’t the technology. It is not the software development team, the tools, or even the platform. It’s a lack of clear understanding of the requirements and the complexity that must be overcome, combined with realistic expectations for the resulting solution. An organization can have a clear understanding of its requirements, but not grasp the complexity to put in place a solution. As a result, they have unrealistic expectations for the resulting solution.
Any organization who needs software engineered needs an estimate of the costs involved. An estimate is a series of educated guesses on the number of hours certain tasks will take to define a solution. To create an estimate requires a specification. A specification provides in detail exactly what an organization wants to have built.
Most organizations have some high-level requirements. Often these requirements lack enough detail to create a detailed specification. This is because they hide complexity. This makes it difficult to create an accurate estimate. The more complete the specification addressing the complexity, the more accurate an estimate. Why is addressing complexity so important? Because the complexity exponentially affects cost.
The Cost of Complexity
Complexity has an exponential effect on the cost and time to develop software. The engineer must take into account the breadth and depth of complexity.
Depth of complexity takes into account:
- Number of Functionalities.
- Number of Features within each functionality.
- Number of internal and external interfaces.
- Number of Users and variety of Users.
- Amount and Type of Data and Data Structures.
Breadth of Complexity considers:
- The data shared between functions and logic.
- The transfer of control among functionalities.
The more complex the essential features, the more time and money it takes to build software. Software engineers must use simplification, technology, processes, and methodologies. They use these things to weigh the variables, to create a software system that provides the illusion of simplicity. Building software takes engineering. It takes knowledge. And it takes expertise to make the trade-offs necessary to create a system. It takes time and cooperative effort between the software engineer and the organization that wants the software built. This at the core; this is why estimates are not free
How Much Does a Specification and Estimate Cost?
Let’s consider the extremes. The National Aeronautics and Space Administration (NASA) sends human beings into space. As a result, NASA creates specifications, estimates, and designs in painstaking detail. Why, because human lives are at stake. If they do not take all the complexities into account, the system could fail. Astronauts and scientists could die. Software and Hardware systems are mission critical at NASA. NASA spares no expense in writing specifications. They carefully plan for every contingency; dotting every “i” and crossing every “t”. NASA spends months, even years, getting every detail right before they build a system.
The traditional business system does need “NASA level” precision. Yet, every software system requires a modicum of due diligence. There is a trade-off of time, money, and risk. The more time and detail used for a specification, the more complexities are addressed. This focus on time and detail yields a more accurate estimate. It reduces the risk of unforeseen and unaddressed complexities. If you spend less time on the specification, the risk increases. Remember complexity exponentially affects cost. Not addressing complexity, means exponentially blowing your budget.
So how much does a specification and an estimate cost? This question is answered with a question. How much risk are you willing to bare? Once you understand your risk tolerance, you need to understand how a software firm works.
Understanding How Software Firms Work
Software development firms sell time and expertise. That is their product. They provide professional services. Their model is like a doctor or an attorney. Most of them bill for their time by the hour. An engineer’s time is a fixed cost. It is a fixed cost regardless of whether they are billing or not.
Additionally, there are other costs:
- overhead costs
- computer software
- computer hardware
- sales costs
- management costs
- building rent
- training costs
- cost of benefits
Let’s consider a fictitious software development firm. We will assume they have a bill rate of $1.20 per hour.
One dollar per hour is unrealistic and is only for illustration purposes.
So for every non-billable hour how many hours to break even on one non-billable hour?
One non-billable hour is $1.20 in lost revenue.
One non-billable hour is $1.00 in cost.
If we have $1.00 in cost per hour when an engineer is not billable, it would take five billable hours to break even on one non-billable hour. And it would take six billable hours before any profit is made. Billable time is why most firms charge for estimates.
When estimates are created for free, they are often done a quickly as possible. Building estimates at this level of speed means that many details will be overlooked. Additionally, inherent complexity is not taken into account; and this increases the risk for you the customer. That risk also increases the likelihood you will not get the software solution that you wanted in the first place. Additionally, it will likely result in unforeseen cost overruns.
Some firms will provide a free estimate and then charge you a higher hourly rate to factor in those costs. With the complexity of software, there is always an inherent risk. If the firm provides free estimates or if they do any engineering of a solution on your behalf, they assume all the risk for the project. As the customer you may choose not to have them take on your project and therefore, the estimate ended up costing them money and time that yielded no return for their effort.
The bottom line is there is no such thing as free. Estimates are not free to you the customer, and they are not free to the software development firm who creates them. This is why it is in your interest to pay for an estimate. But ultimately to have a successful project you must work collaboratively with the company you have chosen. Collaboration always reduces your risk and yields better results from your service provider. The time you pay for to have them research, analyze, design, and estimate your project is valuable to you. It is valuable to you to have a well-designed and scoped solution. It gives you a more accurate view of the cost of building a solution. It also gives the firm you are working with a clear blueprint to build you the solution you desire.
Software Development is Not Accomplished in a Vacuum
Software development is not done in a vacuum. Software development requires the active participation of customers, stakeholders, and users to be successful. The more information and understanding the software team has, the more accurate the end solution. The software development team will be more precise in meeting your needs and the estimate will be reliable to both you and the development team. Using the specification and estimate to build the application, you both can ensure there is no scope creep. This level of collaboration helps facilitate an on-time and on-budget delivery.
Throughout this blog, I have used the term software. However, this term applies to software development projects, digital marketing projects, and infrastructure projects alike. All these technology projects have great complexity and the principles discussed in this blog apply perfectly to these project types. Outcomes are a shared responsibility between the software development firm and you the customer.
Bringing It All Together
Remember even the simplest software systems are complex. Think about the thousands of interactions that you have in a day with people, machines, and objects. Then consider the number of thoughts and potential actions and possible alternate paths you could have had on a given day.
Now you have some idea of the complexity of building software. You should have an appreciation for why estimates and specifications are valuable and why their cost is never “free”. When contracting a firm to build a software solution, there is always a cost in risk and real dollars.