Estimation in Software Development

Summary
My list of issues with estimation of software effort, costs and schedules.

See McConnell for another list (better).

Cost models aren’t perfect

The official “engineering” way of estimating proposed by CMM and others work in three steps:

1. estimate size,
2. calculate effort using a model (COCOMO is quite common),
3. use staffing profile and historical productivity to derive schedule.

Kemerer has shown that these models have a huge error factor (up to 500%) even if you know the size (step 1 is perfect).

Guessing program size

There is no objective way to estimate the size of a program given the requirements

This article by Lewis provides the reasoning behind this statement. Even using something like Parnas’ tables to specify all inputs and outputs of the program, the range between minimum size and the upper bound is huge.

Productivity is dificult to quantify

Even if you measure size and development time after the project is finished you still don’t know the productivity, as the program might be unnecessarily big (i.e. could have been done in 200 lines instead of 5000).

Personal factors have a big effect

COCOMO advertises an effort/size estimate error of 50% after requirements and 10% after detailed design. Even so, as McConnell points out:

The seven personnel-oriented factors collectively … exert a staggering influence range of 25.8!

Better people! Try it out here

See this honest article at the 7th International Forum on COCOMO and Software Cost Modeling to see the problems of applying something like COCOMO in your organisation.

So, what can we and what should we do?

• Hire the best people you can find. It is better to have a smaller team with good people than a big team of mediocre people.

• No fancy models. Given their accuracy I would only use them to compare. Just ask the engineer to say easy, average or hard and map that to actual effort at the end of the project.

Some references