What to say?

[Hill Station Murthy]

Problem: Due to a bad installation (not made by me) part of the system is corrupt on the dev environment.

Firstly I would say as a security expert that you should always ensure that installation regime is written with this auditing functionality to point the finger of blame and cover your backside.
ALways make sure you have a paper trail. In my opinion I think you have failed on this basic level.

I recommended 3.

I recommend 1

I have been in your shoes many times in the past and let me tell you very clever way to ameliorate this tricky situation you are finding yourself in:

Production of very verbose documentation with many ambiguities in release candidate to your UAT team. They will keep rejecting RC because they can't folow documentation and by time they have accepted documentation the fix will be fully implemented.

This is good way of buying time in my opinion.

HTH Joshi

[d000hg]

Depends on your definition of super important. In short it would mean that any complaints worldwide about bookcases by the name of Billy or any other flatpack furniture might not be ever received or dealt with after mid December, hardly Boeing or LSE but still a big player in their market field.

I suppose it then depends the worst-case down-time... if you did do it and it breaks, how easily you can roll-back.

In your original options, what about:

4)hold the release until Q1

Is that a total non-starter?

[chef]

Firstly I would say as a security expert that you should always ensure that installation regime is written with this auditing functionality to point the finger of blame and cover your backside.
ALways make sure you have a paper trail. In my opinion I think you have failed on this basic level.

Ok, I'll bite, first up WTF are you talking about? at what basic level have I failed at? if you mean not having a paper trail, then I never ever keep things discussed verbally, I always follow it up with an email to ensure I have full recollection of what was said or agreed, by whom and when. Not only as it is good business practice but my memory is pretty p!ss poor at the best of times and so having it written down helps (see Memento for an example)

Second, the installation was performed months before i even walked on site. The change documentation at client co is shocking and it is very clear that the corruption is in no way related to me and was only discovered by me as part of me developing the requested change, client co recognise and agree that (in writing). So my arse is well and truly covered, even more so by highlighting this problem to them BEFORE migrating it to a 2nd environment and spreading the cancer corruption further.

I recommend 1

I have been in your shoes many times in the past and let me tell you very clever way to ameliorate this tricky situation you are finding yourself in:

Production of very verbose documentation with many ambiguities in release candidate to your UAT team. They will keep rejecting RC because they can't folow documentation and by time they have accepted documentation the fix will be fully implemented.

This is good way of buying time in my opinion.

HTH Joshi

Ok hotshot, you would knowingly migrate a partially corrupt system from dev (where it currently affects nobody as little is dev is done) to test where the effects would be further realised to the live production system where it is used globally 24/7 ???? And you would "delay" this by delivering shoddy documentation in the release candidate to confuse UAT and therefore have client co. think you are a muppet who isn't the expert you were brought in as and cant do what was asked of him by producing a legible release candidate???? remind me not to work on your dev team.

Furthermore, you would suggest imploying these delay tactics for approx 3 months for a release that was/is scheduled to take 6 weeks from start of dev to production release??? Wouldn't you think that client co would look at you as utterly unprofessional that you cant deliver something that was scheduled for 6 weeks in over 4 months? FFS.


So to summarise you would:
-deny it happened and continue as planned
-release a confusing release candidate to delay the inevitable
-when at the greatly delayed date the sh!t hits the fan, look for someone to point the blame to to cover your own arse

reminds me of several stereotypical subcontinent workers I have previously had the misfortune to be working alongside..

rather than
-raising the found problem to business before it becomes a possible BIG problem, i.e during development testing before it gets to UAT.
-put forward suggestions that minimises impact but also doesn't delay the release of several other unrelated changes
-strongly resist developing directly on production as it goes against what you have been brought up to learn that it is both unprofessional and high risk

[chef]

I suppose it then depends the worst-case down-time... if you did do it and it breaks, how easily you can roll-back.

In your original options, what about:

4)hold the release until Q1

Is that a total non-starter?

the release is made up of multiple changes that are not related therefore the release can go ahead in December but I recommended that we pull this particular change from the release and implement it end of Q1 shortly after the upgrade. So essntially your option 4 and my option 3 are the same.

[TheFaQQer]

the release is made up of multiple changes that are not related therefore the release can go ahead in December but I recommended that we pull this particular change from the release and implement it end of Q1 shortly after the upgrade. So essntially your option 4 and my option 3 are the same.

That's what I would recommend.

Keep stressing "in my expert opinion..."

[Mich the Tester]

Ok hotshot, you would knowingly migrate a partially corrupt system from dev (where it currently affects nobody as little is dev is done) to test where the effects would be further realised to the live production system where it is used globally 24/7 ???? And you would "delay" this by delivering shoddy documentation in the release candidate to confuse UAT and therefore have client co. think you are a muppet who isn't the expert you were brought in as and cant do what was asked of him by producing a legible release candidate???? remind me not to work on your dev team.

Any test manager worth his salt is alert to the tactic of trying to confuse testers with long, shoddy documents and will advise his client not to accept until those documents are clarified; he'll also investigage further to find out why a dev manager is using an obvious obfuscation tactic. There is a reason why good test managers earn good money; we're not as thick as some people like to think.

[chef]

Any test manager worth his salt is alert to the tactic of trying to confuse testers with long, shoddy documents and will advise his client not to accept until those documents are clarified; he'll also investigage further to find out why a dev manager is using an obvious obfuscation tactic. There is a reason why good test managers earn good money; we're not as thick as some people like to think.

I totally agree. Hill Station Murthy has gone down greatly in my estimation with that advice. I didn't realise it was rodeo season already.. yeeehaaa.

[wobbegong]

Firstly I would say as a security expert that you should always ensure that installation regime is written with this auditing functionality to point the finger of blame and cover your backside.
ALways make sure you have a paper trail. In my opinion I think you have failed on this basic level.



I recommend 1

I have been in your shoes many times in the past and let me tell you very clever way to ameliorate this tricky situation you are finding yourself in:

Production of very verbose documentation with many ambiguities in release candidate to your UAT team. They will keep rejecting RC because they can't folow documentation and by time they have accepted documentation the fix will be fully implemented.

This is good way of buying time in my opinion.

HTH sasguru

FTFY

[Hill Station Murthy]

Ok, I'll bite, first up WTF are you talking about? at what basic level have I failed at? if you mean not having a paper trail, then I never ever keep things discussed verbally, I always follow it up with an email to ensure I have full recollection of what was said or agreed, by whom and when. Not only as it is good business practice but my memory is pretty p!ss poor at the best of times and so having it written down helps (see Memento for an example)

Second, the installation was performed months before i even walked on site. The change documentation at client co is shocking and it is very clear that the corruption is in no way related to me and was only discovered by me as part of me developing the requested change, client co recognise and agree that (in writing). So my arse is well and truly covered, even more so by highlighting this problem to them BEFORE migrating it to a 2nd environment and spreading the cancer corruption further.

I wasn't of the understanding that you were not present on client site at install time. For this, excuse my imperinence.

But when I talk of paper trail perhaps I misled you. What I was meaning was that I always have full audit trail covering any operational aspects of any application I am working on, including install. With install, I would generate log and wehn this problem installation took place I would take my log to the highest level and show them proof that I wasn't responsible ofr installation because my log will contain windows user account of the installing user.

Ok hotshot, you would knowingly migrate a partially corrupt system from dev (where it currently affects nobody as little is dev is done) to test where the effects would be further realised to the live production system where it is used globally 24/7 ???? And you would "delay" this by delivering shoddy documentation in the release candidate to confuse UAT and therefore have client co. think you are a muppet who isn't the expert you were brought in as and cant do what was asked of him by producing a legible release candidate???? remind me not to work on your dev team.

But if RC keeps getting rejected by QA it's never going to make it to Live isn't it?

HTH

Joshi

[Mich the Tester]

But if RC keeps getting rejected by QA it's never going to make it to Live isn't it?

HTH

Joshi

Why do QA reject things?