Master Estimation Process Like a Boss

If you are in the business - there's no way to avoid the topic of estimation. Either by estimating upcoming work with a detailed analysis or without, client will always need to know the budget.

Budget estimation.

We all hate doing it, but there’s no way to avoid it.

If you’re running a software development business your potential clients will always ask for the budget on any project you pitch. Sometimes they will want to see a detailed analysis. Sometimes they won’t. But they will always want to see an accurate estimation of total costs before they will sign off on your project.

These estimations are, of course, 100% reasonable of your clients to ask for.

But they are really hard to produce.

Software development just has so many hard-to-pin-down costs to take into consideration. You need to figure out what open source solutions you will use. You need to ID what custom business requirements you will have to develop. You need to determine which components you can re-use, and which you won’t. And these three factors are just the tip of the iceberg.

Finding a budget estimation framework that accurately processes all of the big cost variables is challenging, to say the least. At StartupCraft Inc. it took us a lot of research, investigation, and trial-and-error to piece together our budget estimation framework. But once we did, we were able to churn out accurate estimates in a manner that made the whole issue of budget, costs, and value demonstration much smoother for both us and our clients.

In this article, we’ll share how we developed our budget estimation framework, what it looks like, and how we automated the whole process to make it a seamless element of our workflow.

Let’s dive in!

Our First (Failed) Attempts at Budget Estimation

Our first attempt at budget estimation was simple– we used to just ask our clients what their budget was. This rarely worked. Most of our clients hadn’t set a budget. And when they did have a number in mind, it usually wasn’t realistic.

From there, we accepted that we’d have to come up with all budgets ourselves, and we devised a simple equation.

Seems pretty foolproof, right?

Not quite. It turns out this simple equation only works in a “vacuum environment” where everything goes according to plan. But software is developed in a complex, dynamic environment with a lot of variables that don’t fit neatly into the above equation. And these variables always shifted our final costs far away from our initial budget estimates.

Frustrated, but undeterred, we tried a third approach. We calculated our baseline equation. But then we also noted each of those extra variables that appeared and inflated our budget, as they came up.

Over time, we came to see that there are only four big variables that make software development budget estimation so challenging. We figured out how to factor each variable into our final formulas. And we were finally able to generate accurate budget estimations— despite the highly-dynamic nature of software development.

Let’s take a look at what these big four budget variables are, and how we accurately worked them into our estimations.

The Four Big Budget Variables in Software Development

Now, we’re not saying these are the only four unpredictable variables in software development budgets. We simply found these were the big four that really impacted our budgets.

Let’s look at each in a little more depth.

Variable One: Communications and Admin

Before developing our framework, we never considered the time we spent on communication, both inside our teams and with our clients. We also didn’t consider the time we spent on administrative tasks, in particular the time we spent on the estimation process itself.

Variable Two: Varied Developer Experience and Edge Cases

We found that developer experience played a much bigger role in our final budgets than we expected.

Sometimes our specialists were overloaded with other projects, and we had to give their tasks to other—less experienced—engineers who took longer to complete the task than our Rockstar would have. Other times, our seasoned engineers ran into edge cases that they unexpectedly stumbled over, because they lacked some highly specific expertise we didn’t expect would be an issue.

For example, we were once adding coupon functionality to a piece of software. The functionality was made by a senior engineer who lacked intense experience with the software’s specific billing system. He took twice as long to complete the task as an intermediary engineer—with the right experience—would have taken.

Variable Three: Quality Assurance

Natural asymmetries and inconsistencies make humans beautiful, but they’re completely unacceptable in the strict world of calculations and code. Your clients want you to give them perfect, bugless, resilient software, which you can only offer by ferreting out errors in your code.

We used to account for this by telling our clients that there was a chance our software would have an error rate that would cost up to 30% more to solve. But this was never precise enough, and our final report—that didn’t incorporate professional test results—always looked poor. It always looked like we were defending the “human factor” in our development process, instead of giving a solid solution that ensured high quality software.

Variable Four: The Unknown Unknowns

The most interesting tasks are usually also the most complicated tasks. Every good engineer loves these challenges. The harder the problem you give them, the more joy and happiness they feel when they solve it.

Unfortunately, these “monster tasks” are also really hard to estimate ahead of time. They are filled with twists and turns that are impossible to predict. That makes them really hard to accurately estimate in your initial budget. Even “Rockstar” developers usually fail to give a proper estimate on these big tasks.

Once we identified these four big variables, we quickly got to work developing methods to accurately account for each within our budget estimation process. Here’s what we came up with.

How We Budgeted for Communications and Admin

Before we budgeted for this variable, we put in a few simple changes to our communications and admin processes. First, we reduced communications time by conforming to a single requirements template protocol. Second, we defined a reasonable connection between how much communication and admin was required for a project, given that project’s level of requirement details. And finally, we began to set boundaries with our clients regarding what constituted “extra time” required for communications and admin beyond what was required for the implementation itself.

Taking these factors into consideration, we were able to simply add a 10% cost to the total task estimation to cover our communication and admin efforts, (including the efforts required to produce the estimation itself).

How We Budgeted for Varied Developer Experience and Edge Cases

This solution is a little more complex and incorporates a set of factors and calculations which we can’t fully reveal (as we have to keep them internal company secrets). But we can reveal that our solution involves becoming more efficient with our tasks by bringing in part-time guest engineers, and by learning who to properly assign certain task estimations too.

Ultimately, our calculations led us to budget an extra 10-20% to solve the problem of varied developer experience, and whatever edge cases cropped up. This is not a magic number—you should find your own, based on the expertise of your own team—but in our case capping this at 20% allows us to stay within our budget, while unlocking a lot of flexibility to get our specialized development tasks completed quickly.

How We Budgeted for Quality Assurance

To begin, we admitted to ourselves—and our clients—that errors happen, and that they don’t have to be seen as pure evil. We explained that, properly managed, a piece of software’s errors provide a clear path towards improving it.

After opening this dialogue, we began to set, with our clients, what constituted an acceptable level of error for each task or feature. This gave us all a crucial point of evaluation for the final work provided.

Now that we had clear quality targets to hit, we began to integrate a mandatory Quality Assurance process that would be conducted by a dedicated member of our team. This process included locating and fixing bugs, securing ourselves for the future, and looking fair to our clients by having a solid routine in place to ensure the quality of their software.

We timed what this process took and added an extra 40% to our budgets to account for it. An extra 40% might sound like a big number, but that’s what this Quality Assurance process requires, and the process makes a huge difference between whether or not you produce a fully working, or only partially working, solution.

How We Budgeted for The Unknown Unknowns

Finally, we had to deal with our “unknown unknowns”— the most interesting and complex tasks that require their own micro-investigations to complete, and which feel impossible to accurately estimate before we actually began the project.

Instead of pretending we could accurately estimate these tasks, we found the best way to handle them is to just tell our client straight away that these items are impossible to estimate ahead of time, and then we explain to our clients what work it will take to come up with that value.

And at StartupCraft Inc., we still struggle to articulate this work. We often start by looking for a quick solution, or at least a good direction to begin our investigation. We then split our investigation into micro-steps in that direction and push the task deadline back (i.e. the deadline is defined by the task itself). By re-evaluating the feature over time and performing retrospective analysis, we often find a better solution, or at least a clear answer on how much resources and time the task will take to implement. This is always a dialogue with the client, and there’s no simple answer.

Ultimately, the business value of the task always becomes the main factor here. We do our best to carefully measure the value of the task, analyze its results, and begin the conversation with the client. But even here, it’s a challenge. It’s impossible to predict the precise cost and value of the most innovative products we develop. The only thing you can really do is keep an open dialogue with the client, and either determine if they are willing to give the task the time it requires or cancel the task entirely.

Our New Budget Estimation Framework: How it Works, and Its Results

We bundled each of these steps into a simple framework that adds additional costs and “hidden” fees into reasonable padding:

Using this framework, you will have to add 60-70% of additional costs—as well as potential unknown costs—to your original estimation, nearly doubling your initial estimation.

If that sounds like a lot, it is. But a high-quality implementation requires these extra stages and roles. And ultimately it is always cheaper and more considerate of your clients to charge for these items ahead of time, instead of adding them in later in the process by “explaining bugs”.

After we began to use this framework, the quality and accuracy of our estimations increased dramatically. And at the same time, we experienced a set of benefits we didn’t expect…

The Deeper Benefits of Our Budget Estimation Framework

Our budget estimation framework shows your clients that you have a real plan in place to give them the best software possible. The framework describes the details of your entire process, defining you as a professional, and showing your clients that you’re considering their project’s risks in a transparent and honest manner.

Which means this framework not only produces more accurate estimations, it also transforms your dialogue with your clients, giving them comprehensive knowledge of their project, and making them more likely to treat you like a true partner in their project.

We loved experiencing these benefits. But before we began to use our new budget estimation framework with all of our clients, we first took a few extra steps to weave the whole process seamlessly into our workflow.

How to Streamline on Our Budget Estimation Framework

After a quick search, we found a set of tools that keep our budgets sharp and transparent with both our team and our clients. These tools won’t address every issue, but they form a solid foundation. (We’ve begun to develop our own internal tool that will cover every issue, but that’s a story for another day.)

The main two tools we use are Asana—to track our tasks—and Hubstaff—to track our time. Here’s how we use them:

Asana and HubStaff integrate with each other, and can confirm your estimation by creating a reporting system that you cannot fake.

Reporting in Our Budget Estimation Framework

We develop different reports depending on whether we use a Fixed Price for the project, or a “Time & Material” model.

For Fixed Price projects, we generate reports that confirm implementation, but we don’t report actual resources spent. With Fixed Price contracts, we spend more time on estimation, and set this as a pre-work milestone. We use this extra time to produce a full specifications list that includes details and the final price for the project (instead of a range). However, it also must include hidden fees that act as a buffer, and insurance, because we have to take additional risks with a Fixed Price contract.

For Time & Material contracts, we provide our clients with full access to our tools. This lets our clients see the screenshots captured by HubStaff, and to see how our teams spent its time (which is essential for remote teams).

We always prefer to set Time & Material contracts. They are fair and transparent, and in them our clients only pay for actual work completed—with no hidden fees. Our clients gain a complete understanding of the full cost breakdown for their project, which makes them feel more comfortable, and gives them a greater sense of control over their budget. They stop questioning how our teams spend their time, as they can log into HubStaff and Asana at anytime and see how many resources were spent at any given time, and for which task they were allocated to. In addition, our clients who use a Time & Material contract tend to save resources by focusing on only developing the features their customers value the most.

What You’ll Get if You Adopt Our Budget Estimation Framework

Our budget estimation framework has offered huge new benefits to our clients. But it’s created an even greater positive impact on our own internal operations.

After adopting our framework, Asana, and HubStaff, we immediately gained clear and self-explanatory internal reporting. This reporting let us connect the money earned by our team to the budget set by the client in order to easily see our profits. HubStaff’s reports on spent budget show you how much your team earned. We further centralized accounting by making hourly rates mandatory for every member of our team.

Through this method, we have been able to increase the total money earned by each of our team members, and link it 1:1 to the work they completed. And after we complete work, we’re able to charge any budget left over from our estimation as our margin (justified from associated software tools, miscellaneous expenses, and estimated profit).

How to Adopt Our Budget Estimation Framework

Over the last few months we’ve successfully tested this framework with all of our clients, and feel it’s ready for prime-time.

Now, there are still some kinks to work out. And there are some elements you will likely need to customize to your unique business context. (We plan to highlight these when we present the full product we’re developing.)

But ultimately, we feel you’ll benefit from adopting this framework. When you do, your team will begin to give sharper, more precise estimations. These estimations will form your first steps towards focusing on growth, avoiding rookie mistakes, and creating a successful and profitable agency. The honest and transparent client communication this framework provides gives you the key to building long-term, partner-first relationships. (And if you are a client and not an agency, then we hope this article gives you a clear picture of what any agency you’re looking to work with should provide you with.)

If you do use this framework, please share your experience and experiments in the comments below. And if you have a better process, we’d love to hear it! The only thing we love more than sharing our own experience, is to learn even more from all of you.