Estimation in Software Development
Posted January 27, 2005 in miscSee McConnell for another list (better).
Cost models aren’t perfect
The official “engineering” way of estimating proposed by CMM and others work in three steps:
- estimate size,
- calculate effort using a model (COCOMO is quite common),
- use staffing profile and historical productivity to derive schedule.
Kemerer has shown that these models have a huge error factor (up to 500%) even if you know the size (step 1 is perfect).
Guessing program size
There is no objective way to estimate the size of a program given the requirements
This article by Lewis provides the reasoning behind this statement. Even using something like Parnas’ tables to specify all inputs and outputs of the program, the range between minimum size and the upper bound is huge.
Productivity is dificult to quantify
Even if you measure size and development time after the project is finished you still don’t know the productivity, as the program might be unnecessarily big (i.e. could have been done in 200 lines instead of 5000).
Personal factors have a big effect
COCOMO advertises an effort/size estimate error of 50% after requirements and 10% after detailed design. Even so, as McConnell points out:
The seven personnel-oriented factors collectively … exert a staggering influence range of 25.8!
Better people! Try it out here
See this honest article at the "7th International Forum on COCOMO and Software Cost Modeling ":http://sunset.usc.edu/events/2002/cocomo17/agenda_cocomo17.html to see the problems of applying something like COCOMO in your organisation.
So, what can we and what should we do?
- Hire the best people you can find. It is better to have a smaller team with good people than a big team of mediocre people.
- No fancy models. Given their accuracy I would only use them to compare. Just ask the engineer to say easy, average or hard and map that to actual effort at the end of the project.
Some references