Software Testing

[domcobb180]

The infratructure dudes here finally set up a test area at ClientCos client, yippee, we now have a test system. three months after the system went live with bugger all UAT

Does few issues from UAT = a good system or just a poor UAT..... (or course you could have both... !)

Agree on the benefits of automated tools but not available in many places and not available to run through many combinations as various constraints in practise.

Thanks for the replies all - be interested to hear how any seasoned testers approach things and if you find it any different from perm testing?

[EternalOptimist]

It went live so wearing my consultant hat that's sign off. Everything is now a chargeable change request (even if it is a bug).

thats nothing, they asked me on wednesday to document the system. So I quoted 3 months

they had brown adrenalin running down their legs. hehhe

that'll learn them to get a system build with no spec, no test area, no uat and no plan.

anyway, they must like it, they asked me to do another

[EternalOptimist]

Does few issues from UAT = a good system or just a poor UAT..... (or course you could have both... !)

Agree on the benefits of automated tools but not available in many places and not available to run through many combinations as various constraints in practise.

Thanks for the replies all - be interested to hear how any seasoned testers approach things and if you find it any different from perm testing?

well, I am the developer, not the tester. so few user complaints, issues, change requests or bugs, is good , but its probably more luck and tolerant users than anything else

[mudskipper]

It is a toughie. As a developer who's currently testing (and bored tulipless) I know I find defects that someone who hasn't got the technical background wouldn't pick up.

Having said that, most of them are obscure scenarios that would be unlikely to happen in the real world. And ClientCo seem to have a very tolerant approach to even quite major defects getting through to their customer facing websites and accept it as 'one of those things'.

[russell]

It is a toughie. As a developer who's currently testing (and bored tulipless) I know I find defects that someone who hasn't got the technical background wouldn't pick up.

Having said that, most of them are obscure scenarios that would be unlikely to happen in the real world. And ClientCo seem to have a very tolerant approach to even quite major defects getting through to their customer facing websites and accept it as 'one of those things'.

Why are you doing this? Would you also clean the toilets? Or do a shift in the coffee shop.

[mudskipper]

Why are you doing this? Would you also clean the toilets? Or do a shift in the coffee shop.

Current work fits nicely with lifestyle / family. Work 3 or 4 days a week, mostly from home and getting paid well for it. Finishes at end of June, so might actually have to get proper work after that.

[russell]

Current work fits nicely with lifestyle / family. Work 3 or 4 days a week, mostly from home and getting paid well for it. Finishes at end of June, so might actually have to get proper work after that.

Ah sorry, I thought you were in a developer role and they had asked you to do testing. Sounds good.

[Freamon]

Agree in theory but in practise you find on a lot of projects this isn't always the case and you have to roll with a sub optimal documentation set or a set that's fairly high level with a lot of ambiguity - is it practical to refuse to work on test cases until a set of perfect or near perfect documentation exists.... As a perm you can negotiate on this but mostly there's a compromise. How do contract testers find this goes down - is it a case of downing tools until a perfect doc set is produced? In addition, you can test something and it works with one dataset or at one time slot (or in conjunction with other processes) but won't work with another but to test all possible data instances/process combinations would be impossible. Then you'd end up with a requirement failing under a certain context but probably out of 100 testers, none would find the issue or think to specify anything other than a fairly generic test case.

When you write your test cases, document any assumptions you've had to make about the prior state of the application and then get those signed off. The assumption could be "application will be set up with X set of test data and test will be executed at A,B and C times". These should all be documented in the behaviours, but if they are not then you just have to make assumptions (and get them agreed upon).

[meridian]

Agree in theory but in practise you find on a lot of projects this isn't always the case and you have to roll with a sub optimal documentation set or a set that's fairly high level with a lot of ambiguity - is it practical to refuse to work on test cases until a set of perfect or near perfect documentation exists.... As a perm you can negotiate on this but mostly there's a compromise. How do contract testers find this goes down - is it a case of downing tools until a perfect doc set is produced?

You'll never get perfect documentation, but as a contractor neither will you be downing tools. You're (presumably) being hired for your expertise in a particular area - you'll have all sorts of references to create test scenarios from, whether this is use case scenarios from the business, or your own domain experience as the test oracle. Raise issues and risks in your strategy/plan, create test cases where possible, note that you'll be doing exploratory testing to complement your structured approach.

In addition, you can test something and it works with one dataset or at one time slot (or in conjunction with other processes) but won't work with another but to test all possible data instances/process combinations would be impossible. Then you'd end up with a requirement failing under a certain context but probably out of 100 testers, none would find the issue or think to specify anything other than a fairly generic test case.

You're not going to be able to capture all defects. What you can do, though, is design tests that capture as many permutations and scenarios as possible for your test coverage. If the execution time of your tests is not possible within the time allocated for testing, you'll need to work with your PM/Dev/business to analyse for risk and priority.
Data type tests are good candidates for automation - many tools are free or included within dev apps.

[domcobb180]

You'll never get perfect documentation, but as a contractor neither will you be downing tools. You're (presumably) being hired for your expertise in a particular area - you'll have all sorts of references to create test scenarios from, whether this is use case scenarios from the business, or your own domain experience as the test oracle. Raise issues and risks in your strategy/plan, create test cases where possible, note that you'll be doing exploratory testing to complement your structured approach.


You're not going to be able to capture all defects. What you can do, though, is design tests that capture as many permutations and scenarios as possible for your test coverage. If the execution time of your tests is not possible within the time allocated for testing, you'll need to work with your PM/Dev/business to analyse for risk and priority.
Data type tests are good candidates for automation - many tools are free or included within dev apps.

Thanks. Agree wrt your suggested approach and also on the exploratory testing - that's how I'd look on any perm test assignment. (Wasn't serious on the downing tools suggestion!)

In terms of working as a test contractor, is it just a case of working through a limited company, getting contracts reviewed for possible liability and getting some PI then or do I need to print out and archive all documents/e-mails/test plans/approvals/marked up docs/results daily so I have an archive on leaving the client so that I can justify any future issues......(And I couldn't do this with many organisations as it would breach security rules.....) I've worked in IBs in London where vendors are treated well and have a great collaberative approach but I've also seen a lot of vendor companies having to expend considerable effort on negotiation where issues are found.