Response to "Can we put an end to this ‘Estimate’ game of fools?"

This is a response to the blog post: http://blog.frankel.ch/can-we-put-an-end-to-this-estimate-game-of-fools
I think writing a blog myself is the best way to share my alternate opinion on the matter. (The original blog post is quoted in this post for easier reading).

When I was a young software programmer, I had to develop features with estimates given by more senior programmers. If more time was required for the task, I had to explain the reasons – and I’d better be convincing about that. After some years, I became the one who had to provide feature estimates, but this did no mean it was easier: if the development team took more time to develop, I had to justify it to my management. Now, after even more years, I have to provide estimates for entire projects, not just fine-grained features.

This is a management dysfunction. The ones estimating must be the ones developing the thing estimated. Or add margins in the estimates for "junior" (or inexperienced, we all are from time to time) developers. Never blame developers for "wrong" estimates. It's how poor managers use them. That's what's wrong, not the estimates.

But in essence, what do we need estimates for? For big enough companies, those are required by internal processes. Whatever the process, it goes somewhat along these lines: in order to start a project, one needs estimate in order to request a budget, then approved (or not) by management.

We need them for making business decisions. I'll come to this later. But, just as you say: business needs to budget investments (projects) and prioritize them with other (even non software) projects.

Guess what? It doesn't work, it never did and I’m pretty sure it never will.

Why does some companies do this well then? Maybe look at how they are doing it?
The business needs budgets. If you've ever been a member of the board at any organization or cooperative, you know the importance if this. Very often the board needs to know what to budget for something that needs to be invested, and how can they do that without asking us for estimates? Even if they know the value of an investment they need to know the cost as well, it's called ROI (Return Of Investment).
One thing we usually miss is to add margins when needed (for uncertain items). Read this post: http://www.leadingagile.com/2014/02/use-agile-build-next-home/
I always add margins when I invite guests for dinner or when I go traveling or remodel my house. Do the same with software development.

Maybe we can keep price fixed and estimate the scope. Not as good in my opinion. Why not build the best thing? Maybe your customers are happy adding budget (or use margins) for some things (added scope). It's all about customer collaboration. Tell them where they should add margins/buffers.
If estimates are still "wrong", in the end? Well, "execute your risk plan" for this (which you discussed with the customer/stakeholders before project started: "What if the estimates are wrong?"). Don't have any "risk plan", fix one next time, but for now remember: Customer collaboration.

Note: some organizations are smart enough to realize this and couple estimates with a confidence factor. Too bad this factor has no place in an Excel sheets and that it is lost at some point during data aggregation

Exactly. But, why do you think it has no place in the excel? And why is it lost? Fix.

Unfortunately, my experience is the following: estimates are nearly always undervalued! Most of the time, this has the following consequences, (that might not be exclusive):

Yes, stop lying to yourself. Optimism is a disease when estimating.

In general, the first option is to cancel all planned vacations of team members. The second step is to make members work longer hours, soon followed by cutting on week-ends so they work 6/7. While effective in the very short-term, it brings down the team productivity very soon afterwards. People need rest and spirit – and developers are people too!After pressure on the development team, it’s time to negotiate. In this phase, project management goes to the customer and try to strike a deal to remove parts of the project scope (if it was ever defined…). However, even if the scope is reduced, it generally is not enough to finish on budget and on time.The final and last step is the most painful: go back to the powers that be, and ask for more budget. It is painful for the management, because it’s acknowledging failure. At this point, forget the initial schedule.Meanwhile and afterwards, management will communicate everything is fine and goes according to the plan. Human nature…

 Management dysfunctions. Tell how you feel. Don't accept working overtime because someone is using estimates as promises. Fix the dysfunctions. Customer collaboration. Quit, at worst.

In some cases (e.g. a bid), that’s even worse, as the project manager will frequently (always?) complain that those estimates are too high and pressuring you to get lower ones, despite the fact that lowering estimates never lowers workload.

Same thing as my previous response. What's the purpose of using estimates as pressure? They are used for making decisions and understanding. Don't be optimistic. Don't accept pressuring. Ask what the estimates are going to be used for. Seems like they are used for something else here. If things don't get better, quit at worst.

You could question why estimates are always wrong. Well, this is not the point of this post but my favorite answer is that Civil Engineering is around 5,000 years old and civil engineers also rarely get their estimates right. Our profession is barely 50 years old and technology and methodologies keep changing all the time.

Estimates are never right. It's in the nature of estimates. It's *estimates*. But my estimates are almost always close, at least close enough. And if things change so much that the initial estimate is off, tell customers or stakeholders. Remember *Customer collaboration*.

I’m not a methodologist, not a Project Manager, not a Scrum Master… only a Software Architect. I don’t know if Agile will save the world; however, I witnessed first-hand every upfront estimate attempt as a failure. I can only play this game of fools for so long, because we’re all doomed to loose by participating in it.

Don't play games. Be professional. That goes for everyone in the organization.