![]()
![]()
![]()
Next: About
this document ... Up:
Estimating Operating System Code Previous: Results
We caution against interpreting these results as an absolute measure of quality, for these reasons.
Limited Checks The kinds of errors checked for clearly affect the final defect density. As an extreme example, suppose that a static analysis tool warns for potentially harmful, but usually correct program constructs, such as using an assignment operator in a conditional expression. This may generate a large number of warnings which skew the defect density.
Statistical Significance Even if we could agree on a fixed set of tests, there do not appear to exist formal studies - only “rules of thumb” - to determine whether the numbers above are essentially equivalent. One such rule of thumb seems to be to expect about 1 error per thousand lines of code.
Robustness Even if such a study could be performed, it is unclear whether there is a correlation between the number of detected errors and the robustness of the program. For instance, one could imagine a situation where one program has a large number of insignificant errors, while another program has few errors but significant ones. A defect density calculation, incorrectly interpreted, would claim that the second program is more robust than the first. That said, it is clear that a density of, say, 10 errors per thousand lines of code indicates serious coding problems. The interpretation problem arises mainly when trying to distinguish between low defect density numbers such as the ones above.