Reply to: Computer error

[Mich the Tester]

in most systems proving the system works more or less as specified would be a massive improvement.

Culpability and a minimum level of basic testing would be what I would aspire to for legislation.

You'd need to start by demonstrating that what's specified is what's right, and that opens a whole new can of worms.

[doodab]

I think it's difficult to talk about legislating for software quality without requiring software vendors to provide warranties for shrink-wrapped products. Open source would be a massive can of worms as well.

I think problems like the one with the RM system are best handled by making company officers liable for criminal prosecution. There are clearly problems beyond bad software that need to be addressed when something like this happens.

[vetran]

in most systems proving the system works more or less as specified would be a massive improvement.

Culpability and a minimum level of basic testing would be what I would aspire to for legislation.

[Mich the Tester]

If there's a bug, responsibility doesn't clearly lie with one area (unless, for example you said you'd run test script A which would have identified the bug, but didn't) One of the first premises of testing is that you're never going to prove a system is bug free.

Indeed. And you're also never going to prove that professional standards or legislation are bug free.

I just have a feeling that if we go down the route of professional standards, the big corporates and the training companies are going to dictate; that means the usual suspects, plus Rex Black and the ISTQB, who IMO are sitting on the summit of Mount Stupid.

[mudskipper]

So why did Agile emerge? Process thinking obviously didn't solve the problems. Quite simply, we can't agree on standards; if that means we're constantly challenging each other I'd say that's actually better than standards set in stone.

I wasn't suggesting ISO 9001 as a solution, more as an illustration that people have tried to solve the problem. Legislating on testing alone isn't the answer.

[mudskipper]

And push testers' rates up Carry on with this reasoning, please !!!

On the other hand, it would push our professional indemnity premiums up too

If there's a bug, responsibility doesn't clearly lie with one area (unless, for example you said you'd run test script A which would have identified the bug, but didn't) One of the first premises of testing is that you're never going to prove a system is bug free.

[Mich the Tester]

There would be little point in legislating good testing if you don't legislate good requirements, good design, good coding etc...

Surely this was what ISO9001 et al were intended to do.

So why did Agile emerge? Process thinking obviously didn't solve the problems. Quite simply, we can't agree on standards; if that means we're constantly challenging each other I'd say that's actually better than standards set in stone.

[Mich the Tester]

... making senior management culpable would improve testing in most firms a hundred fold.

And push testers' rates up Carry on with this reasoning, please !!!

On the other hand, it would push our professional indemnity premiums up too

[mudskipper]

When I was younger I thought I knew what constituted good testing but I don't any more. I'm a bit concerned that those who would write legislation on what good testing is would be those sitting on the first peak of Mount Stupid.

There would be little point in legislating good testing if you don't legislate good requirements, good design, good coding etc...

Surely this was what ISO9001 et al were intended to do.

[vetran]

I agree on giving 'senior people' the accountability that fits their remuneration, something that's painfully missing in the corporate world, but who's going to write the legislation? Have standards prevented crises? Did Basel 2 prevent banking crises? Did SOX prevent corruption?

What is a 'reasonable standard' of testing? I'd love to know; I've been in testing for more than 15 years and the more I learn, the less certainty I have of what constitutes good testing.

SOX has caused perceptible improvements, whilst it has not eradicated misbehaviour it has reduced it and made those responsible more accountable:

Sarbanes-Oxley law has been a pretty clean sweep - ABC News
Learning To Love Sarbanes-Oxley - Businessweek

It has also made many businesses that embraced it more secure and successful.

A reasonable level of testing? Agree its a challenge but I'm sure you recognise that in this case they obviously didn't do a reasonable level of testing. There must be a set of golden rules like making sure the numbers add up when random values are entered into every input and every process.

At each prosecution they should have reviewed the system to make sure.

Of course I don't blame the testers(if there were any professional ones hired) as no doubt they were underfunded and pressurised but making senior management culpable would improve testing in most firms a hundred fold.

[eek]

When I was younger I thought I knew what constituted good testing but I don't any more. I'm a bit concerned that those who would write legislation on what good testing is would be those sitting on the first peak of Mount Stupid.

I think in the case of the most computer systems in the public sector Mount Stupid is far further to the right than that diagram shows.

Most managers think we've paid millions for this so it most be perfect......

[Mich the Tester]

When I was younger I thought I knew what constituted good testing but I don't any more. I'm a bit concerned that those who would write legislation on what good testing is would be those sitting on the first peak of Mount Stupid.

[Mich the Tester]

Private Eye have been talking about this issue for years - not only have people lost their homes and been sent to prison, but IIRC at least one was driven to suicide.

Wouldn't surprise me; all rather Kafkan. A Russian asylum seeker in Holland was driven to suicide a couple of months ago thanks to faults in a justice ministry system and uncritical users.

Will perhaps add Private Eye to my 'list of publications that every tester should read', which includes but does not entirely consist of;

An Introduction to General Systems Thinking, G Weinberg
Perfect Software: And Other Illusions about Testing, G Weinberg
The Edge of Chaos, M Waldrop
The Quark and the Jaguar, M Gell-Mann
The Demon Haunted World, C Sagan
The Black Swan, N Taleb
The Trial, F Kafka
...and much more

[TheFaQQer]

Private Eye have been talking about this issue for years - not only have people lost their homes and been sent to prison, but IIRC at least one was driven to suicide.

[Mich the Tester]

because it is tested to a reasonable standard and the code is reviewed to ensure accuracy, of course none of these happen and errors occur.

If we had a Sarbanes Oxley style legislation (written by people who had actually seen a computer) for functionality we could see massive benefits.

You cannot trust companies to do a good job unless you put senior people at risk of censure/conviction.

I agree on giving 'senior people' the accountability that fits their remuneration, something that's painfully missing in the corporate world, but who's going to write the legislation? Have standards prevented crises? Did Basel 2 prevent banking crises? Did SOX prevent corruption?

What is a 'reasonable standard' of testing? I'd love to know; I've been in testing for more than 15 years and the more I learn, the less certainty I have of what constitutes good testing.