Friday 1 August 2014

My Rules of Estimation

In this post I'm just going to write down some of the things I've found about estimates. That is: when estimates fill a purpose (some say they *always* do, and I'm more into the ideas of #NoEstimates to lessen the dependency on estimates... But this is off topic in this post).

These are things that apply to me and the situation and context I'm in (types of customers I have, size of projects etc). I'm not saying these are universal truths or are general guidelines. It's merely my opinions. There's no real news in here either, it's rather old stuff. I just want to share them and keep this as a reminder to my future self.

I won't go into lots of details on these items, just list them as shorter notes. I might revisit these items later on and update them.

  1. Estimate or commitment?
    Someone buying software (or anything really) aren't that interested in estimates. They seek commitment (promises). So they are probably not asking for estimates, but for a commitment. But don't confuse estimates with commitments, two different things and requires different approaches. Here's a metaphor on the difference I like (by Mike Cohn): http://mountaingoatsoftware.com/blog/separate-estimating-from-committing
  2. Find out the budget
    It's there for sure. Then try to discuss what can be built for that amount of money. If they don't want to tell their budget (it's a trust issue really but...) it's rather easy to find it out anyway; Tell them something clearly too high and work your way down ;-)
  3. Find out the "roughness"
    Maybe "This year" is okay as an answer? Or "Less than a month"? Don't just assume they want "estimates as you usually do". Probably they don't need precise answers - you just provide it anyway - they need accurate answers (I'll get to this soon). But, if they really seek a precise answer it's a sign of something that is probably a bit risky and you should look out for it.
  4. Find out the value
    If the cost is to be estimated, the value must also be estimated. Calculate the Return Of Investment (ROI). If value and cost is close (low ROI value), it's probably time to cut scope. It's a risk anyway, and it should be discussed with customer. Sometimes this can reveal the level of "roughness" on the cost estimates as well (item 3). Maybe cost isn't that relevant (high ROI value)?
    Value always trumps cost!
    Sometimes they can't (or won't - a sign of trust issue?) estimate the value. If so, one could really ask why this project should be carried out at all..? Anyway, go back to item 2 and you probably have the value somewhere...
  5. Estimate in collaboration
    Who says we can't estimate together? I've written a blog post that touches that subject: http://kodkreator.blogspot.se/2014/03/what-noestimates-has-taught-me.html
    "Customer collaboration" right? I see no problem in this if there is trust (and maybe even if there isn't?). A customer might have much valuable information to provide when estimating. At least they will learn - and feel they are a part of and/or have opinions about - the estimation process you have and the estimates you come up with. Great value, in my opinion. They might claim it's a waste of their time. But I think it can be very valuable for them. We have nothing to hide, right?
    If it's not possible to estimate in collaboration at least don't present your estimate as a "truth" or "this is how it's going to be". Collaborate! Help. Guide. Tell where there are uncertainties or where you feel there should be margins. Tell them some margins might be used, some don't. It's like inviting people for dinner - you want people to be satisfied so you tend to buy too much food, and it's okay. Discuss risks. Read more here: http://www.leadingagile.com/2014/02/use-agile-build-next-home/
  6. Confidence level
    Try to provide your estimates with a confidence level. This is actually not used at all where I work, but claimed to be utterly insane not to in other domains.
    Create a graph (usually in a Weibull distribution form), it doesn't have to take much time to create or use maths. Just use your "best bet" and then min-max values and create a distribution curve by free hand, it's enough to give you an overview of your initial "best bet" estimate. And discuss what you should "commit" to.
  7. Don't be an optimist!
    There might be a good reason for you to be... But usually - and especially with estimates - there might not be a reason to be. Ask yourself: "Who will benefit from us being optimistic here and not be pessimistic instead?" There might be reasons, as I said, but usually not. One reason I can come up with is if there is no true "benefit" from doing a software project, like updating to a new version of some CMS or something. But one could really ask why we should do it, if there is no clear "value" in doing it.
    Read some of what I've written here: http://kodkreator.blogspot.se/2012/01/my-estimates-are-too-optimistic.html But always remember "Hofstadter's law": http://en.wikipedia.org/wiki/Hofstadter's_law
  8. Preciseness is not accuracy! 
    For instance: "952 hours" is not a good estimate. If anything it might clearly show that it's a sum from a number of other estimates. But, please, stop provide estimates like that!
    It's better to be correct than precise. It's better to say "By the end of this year" or "In august" than to say "2 August". Let the specific date be decided later on. Maybe you don't have to present the exact number of hours you estimated at all? Maybe you even shouldn't? Customer is probably only interested in what it's going to cost and roughly when to expect it.
  9. Always keep a deadline - however it was set!
    Okay, this is not true. There might be good reasons to postpone a deadline, but it's a provocative title right :-)
    Anyway, I actually thinks deadlines - often - should be held. Even if they are off. Postponing deadlines can be discouraging for both the team and your customer. And it always feels good to make the "finish line". And you probably have *something* to deploy.
    What do you do? Trim the tail! Read more here: http://www.crosstalkonline.org/storage/issue-archives/2014/201407/201407-Cockburn.pdf
  10. Say No (or: don't always say Yes)!
    Even if I like item 9 it doesn't always work (as I've said). There's still something else you should keep doing during project: say "No". Or actually - and better explained - don't always say "Yes!" to everything. If things change (and they will) we have to say "This will make the initial estimate a bit obsolete" and re-estimate (if needed). There's nothing wrong (in my opinion) to re-estimate if things change, even small things might affect schedule (and surely if you provide those precise estimates I talked about). Read more here: http://www.mountaingoatsoftware.com/blog/to-re-estimate-or-not-that-is-the-question
    Real user input is always good and should be encouraged. But all changes comes at a price (when you drive with estimates (or you could think about trying to lessen the estimate driving i.e. #NoEstimates. Damn! Now I'm there again. Sorry.)).
  11. Intervals
    Presenting estimates as intervals is okay. And sometimes even preferable to e.g. mark some kind of uncertainty or confidence. It will keep your back safe at least :-)
    But remember that some customers might tend to hear the lower interval while you tend to hear the higher. And you run a risk that the lower value will start traveling in the organisation... (dysfunctional, yes, but there's always a risk).
  12. Focus on iterations
    Do everything you can to always try to have something "usable" to deliver. Don't build (arbitrary) parts of the system/application but ask instead: "If we abort this project after the next sprint/iteration and deliver - what will it look like then?" That is, focus on the most important things first.
    Try to have a "beta site" or "beta application" and continuously deploy to it. Let real users see it and use it - get real feedback. Don't listen to *all* feedback, but more is better than less (and one is infinitely better than zero).
  13. Prototype!
    You don't have to stop prototyping (and by "prototype" I include things like mockups e.g. in Balsamiq) just because project has started (and please use prototypes before you start the project as well). Don't finish all the nifty little details 100% (see previous item as well); give users something they can see and interact with. It doesn't have to work all the way to database and handle all "edge cases" etc. Read more here: https://medium.com/@ppolsinelli/from-noestimates-to-yesprototypes-1b51f6a63e5d
    Remember that real user input is something good! Change is better than following a plan, right?
Remember, this list is mainly written for my own purpose, and it fits me and my context. But I'll be happy if you would like to provide some comments on your thoughts and opinions.

3 comments:

  1. 1. We certainly are interested in estimate. Just left a meeting with a vendor where we "estimated" the "total cost of ownership" of an agile development tool over the life of the program (18 months). We can't "know" or have a "commitment" in the absence of the installation and use of the tool and it's integration with the other enterprise tools.
    2. If the buyer doesn’t want to share their budget, they’re not being rational about the expected value.
    3. Accuracy is not possible in the absence of a confidence interval
    4. You need a target ROI. It could be low, it could be high. Find that out first. And then put a confidence interval and an acceptable range on both numbers. Very cheap EXCEL plug-ins available (<$100) for modeling the “numbers” for both cost and returned value. Use a cost for the cost model, then the sensitivity to cost can be modeled
    5. The Basis of Estimate is the conversation starter. “How did you arrive at this number?” Woody loves to use the phrase challenge everything in the absence of any domain or context. This is a domain and context to challenge.
    6. Weibull is for parts reliability. Don’t assume it is of any use. If you don’t know the underlying probability distributing function, a Triangle is the best starting point. Google will find you the reason why this is so. Ditch the Weibull, it’s just a buzz word people like to toss around like Fibonacci numbers, which have no connection to the underlying PDF.
    7. Hofstadler’s was not a mathematician, don’t over quote him. Optimism bias is there, learn to seek “basis of estimate,” and read how, and don’t use platitudes to explain the problems of estimating – do the math.
    8. Yes, know the different.
    9. Deadlines without schedule margin is a “fools errand” and fodder for Dilbert cartons. If you’re going to do the math, do the margin estimates as well. Plans without margins are late on day one.
    10. Re-estimate every iteration, release, epic. Update the ETC and EAC, that’s how the pros do it. Be a pro, don’t let customers “misunderstand,” it’s your job as an estimator to make sure they understand.
    11/12/13. This is called systems architecture. Build the parts in the proper order to arrive at the end with the “all in” needed capabilities, while delivering needed capabilities along the way. I’d conjecture this can’t be done by asking the customer. This is the role of the architecture everywhere I work. No architecture needed, problem is too simple then.

    ReplyDelete
    Replies
    1. Hi Glen! Thanks for comment. I liked it.
      I'm going to take time and respond in more detail, I think you make some good points.
      But, mainly, I think you missed the *my* context/situation :-)

      Delete
    2. 1. You maybe. In my domain, customers aren't interested in estimates really. Not more then that they want to know how we came up with the cost.
      2. Agree.
      3. Confidence intervals is overdoing things in my domain. Doesn't say much to my customers either.
      4. Good advice to have a target ROI. The people I talk to at my customers don't know what ROI is even :-)
      5. No comments.
      6. Good advice. I should update my post maybe. I don't think the Weibull should be used as the probability distribution. I just assumed that drawing a curve by free hand would probably be an outcome in a weibull curve. In my domain mostly true.
      7. I wasn't referring to maths, just my experience that correlates well with that statement.
      8. Yes.
      9. Yes.
      10. Now where touching #NoEstimates... If we re-estimate every iteration, how good was it to make a decision from the initial estimate?
      11-13. I think you might have missed my point?

      Delete