There is quite a debate going on about the evidence for why requirements are needed. The Standish ‘Chaos’ Report has come under fire and its owners have so far kept an eerie silence. What the Standish Corp. and others have tried to do is to gather statistical evidence on how many projects fail, and for what reasons. This is pretty hard to do, as failing projects may well want to keep quiet, and companies do not want to give their competitors ammunition in the struggle to win new work.
However,
the Chaos report did not assert that, for instance, bad requirements work caused
x% of all projects to fail. Instead, it asked large corporations how many of
their projects failed, and then which factors it considered important in causing
failure.
There
are plenty of things you could say about this as an inquiry method – for
instance, suppose that 3 out of 4 corporations just threw the survey form in the
bin, and that of the rest, most of the forms were filled in by the process
improvement department. The 3 out of 4 might have been those who didn’t
acknowledge that requirements were even an issue, and who consequently had the
worst projects: we don’t know, and nor presumably did Standish.
Process
improvement departments might deliberately colour their responses to show how
vital their work was, and therefore how problem-ridden the unimproved projects
were. Shock! Horror! There are heaps of problems! Quick, spend money on process
improvement!
Furthermore,
a list of factors that someone thinks might be causes of failure is not the same
as an analysis of what actually went wrong on any given project, nor as it might
seem to be, an analysis of the relative impact of the different causes of
failure of a set of projects: it’s a generalisation at arm’s length, a piece
of ‘tender-minded’ thinking (see the book review below).
Quite
different statistical evidence could be collected by experiments on students,
but the applicability of such results is also a moot point.
So,
if statistics cannot deliver, is there any way to gather evidence that bad
requirements work is a problem?
Yes
there is.
Remember
that all you need to disprove someone’s claim that X does not exist is one
instance of X, or two or three if you are cautious. You don’t have to do
statistics to try to show that lack of requirements is always bad; you just have
to find one case where it was a serious problem, and the claim that requirements
work doesn’t matter is disproved. In other words, showing that
Exists X: problematic(X)
is
sufficient, and a great deal easier than showing that
Forall X: problematic(X).
Let’s
be concrete about this, then.
·
Do I know of
an actual project where lack of requirements work caused disaster?
·
Yes.
A
scientific analysis-and-modelling package project was knocked together based on
a chat in a pub, as a ‘good idea’ that was ‘obviously useful’ and which
‘could easily be developed by the graduates’. No-one was given
responsibility for the overall specifications, and the system lacked a coherent
architecture and interface specifications. In fact there were scarcely any
written requirements. Nobody looked at the user interface either. The new
programmers did their best, and the individual components, some of them doing
very complicated three-dimensional geometry and interpolation, certainly worked.
But the interfaces were a continuing and very costly nightmare, and when the
product was finally shown to clients, it remained very buggy.
At
least as importantly, it was very difficult to use: nobody had thought how it
would be applied to problems, or indeed what problems it would actually be used
to solve. The system was ‘inside-out’, making sense to the designers of its
component parts, but almost no sense to its users. A whole lot of the
complicated work had to be done in the user’s head. Not surprisingly, it
didn’t exactly sell like hot cakes.
Let’s
go further.
·
Do I know of
an actual project where a single bad requirement was enough to cause serious
trouble?
·
Yes.
An
image processing and distribution service in the days of slow modems and
plain-old-telephone-system phone lines allowed users to view a catalogue with
thumbnail images, and to place orders for high-resolution images. This meant
that users had to be allowed to ask to see what they were about to buy before
committing themselves.
The
requirement should have said something like
“Users shall be able to view thumbnail images.”
Unfortunately it said
“…to view images.”
Yes
it did. At the acceptance tests, the customer asked to be shown how to download
an image. The head programmer showed him how to get a thumbnail on screen in a
few moments.
“Yes, that’s fine, but I want to see a full image please.”
There
was some umming and erring.
“Here it is in the requirements specification, ‘view images’.”
“Yes, but that will take 24 hours or so over the slow modem link; the phone line will go down every few hours, so you’ll never be able to transfer a whole image.”
“Never mind the ‘yes, buts’, it says I can have it and I want it.”
Really.
So the project went over time and over budget to do its best to make it possible
for the customer to do something that was never going to work properly given the
available technology.
The
developers of the image distribution service got paid eventually, which is more
than happened in another company which foolishly included the requirement
“Each search shall be completed within 2.5 seconds”
in
their telephone system specifications. At the acceptance tests, the
developers showed off the pre-arranged searches which all easily passed.
Unluckily,
“each search”
could
include the surname
“Chang”
which
is very common in countries with a large Chinese population. The customer asked
for a demonstration, which, as he presumably anticipated, failed by a wide
margin.
The search was painstakingly optimised, but
the time never did come down below 3 seconds, and the customer got a free
system. He’d found an existence proof of a broken requirement, and didn’t
need statistical evidence to see him through a lawsuit: he had a watertight
case, and he knew it.
Could
you give similar examples? Yes, I thought so. Let’s not be shy: we know bad requirements can be
disastrous.
© Ian Alexander 2005