• Visitors can check out the Forum FAQ by clicking this link. You have to register before you can post: click the REGISTER link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. View our Forum Privacy Policy.
  • Want to receive the latest contracting news and advice straight to your inbox? Sign up to the ContractorUK newsletter here. Every sign up will also be entered into a draw to WIN £100 Amazon vouchers!

You are not logged in or you do not have permission to access this page. This could be due to one of several reasons:

  • You are not logged in. If you are already registered, fill in the form below to log in, or follow the "Sign Up" link to register a new account.
  • You may not have sufficient privileges to access this page. Are you trying to edit someone else's post, access administrative features or some other privileged system?
  • If you are trying to post, the administrator may have disabled your account, or it may be awaiting activation.

Previously on "What is the Biggest IT Cockup You've Ever Made?"

Collapse

  • m0n1k3r
    replied
    In the very late eighties, with 2400 bps modems the norm rather than the exception. I was asked to get a Usenet feed with only a few groups. I managed to get the subscription wrong, causing half of the modems to be occupied permanently and racking up quite large phone bills, while at the same time running out of space on the 340 MB Fujitsu HDD (proper Winchester physical size) that was supposed to store it all. I was on holiday when disaster struck. Oops.

    Leave a comment:


  • BlasterBates
    replied
    The best one I experienced was when we were installing a computer controlled computer system for a hot strip steel mill in South Korea. During commissioning the mill was run up and a few tests, the rollers went up and down, after the tests were finished, the Lead Engineer switched off the computer, as you do , after which we heard a rather odd crunching sound, followed by the swift entry of a Korean Engineeer into the Computer room who uttered something to the effect of WTF. The sight from the control room was rather dramatic with the rollers skewed at a funny angle, unable to rotate and the mill covered in white hot steel, alas the hydraulic valves installed had lost power from the computer and defaulted to wide open thus crushing the rollers and causing the hot metal to pile up like spaghetti.

    Having repaired the mill the tests were repeated. The lead Engineer spotted a slight problem with the control electronics, did a couple of measuerments, having realised the resistor on the card was wrong, pulled the card out to replace the resistor with his soldering iron, and....exactly ...., you pull the card out it's like switching the computer off.

    He was not a happy lead Engineer, and it was not a happy Steel company.

    I earnt a lot of money in overtime fixing the steel mill, but gaw was I tired.

    Last edited by BlasterBates; 6 June 2017, 10:46.

    Leave a comment:


  • SeededLoaf
    replied
    In my earlier days I did a software upgrade of a date centre management network and took down a large chunk of prod. The upgrade process, as per the manual, stated no downtime but it lied. Prod was far too dependent on the backend and there rapid re-design after that incident.

    Leave a comment:


  • fool
    replied
    Originally posted by MrMarkyMark View Post
    Correct and its funny you have just reminded me about something I did when I had just started in IT .
    I deleted a folder on a NW drive by being clumsy

    Luckily was a small IT Dept, at the time, so got it restored without any issue.
    I was always a lot more careful after that
    Yeah it actually crippled my productivity when dealing with production systems for a good wee while. Now it serves as a story I tell the kids when they **** up, so they don't feel like it's over.

    At the same place a guy deleted the main volume on a netapp that had all the LUNs for all the customer VMs that was sold as "Europes first cloud". No backups either. Had to call the vendor and ask them how to extract LUNs direct from the block storage.

    Leave a comment:


  • northernladyuk
    replied
    Originally posted by MarillionFan View Post
    The thread was about IT.

    HTH

    MF
    Please don't tell us you're a real person.

    Leave a comment:


  • MarillionFan
    replied
    Originally posted by northernladyuk View Post
    Apart from your life, of course.
    The thread was about IT.

    HTH

    MF

    Leave a comment:


  • MrMarkyMark
    replied
    Originally posted by fool View Post
    Indeed, but it was a good time to learn the lesson that anyone can screw up...
    Correct and its funny you have just reminded me about something I did when I had just started in IT .
    I deleted a folder on a NW drive by being clumsy

    Luckily was a small IT Dept, at the time, so got it restored without any issue.
    I was always a lot more careful after that

    Leave a comment:


  • northernladyuk
    replied
    Originally posted by MarillionFan View Post
    I am afraid to,admit I don't recall anytime I've fecked something up, minor or major. I'm the one who tends to catch everyone's elses possible feckups and don't make them myself.
    Apart from your life, of course.

    Leave a comment:


  • MarillionFan
    replied
    I am afraid to,admit I don't recall anytime I've fecked something up, minor or major. I'm the one who tends to catch everyone's elses possible feckups and don't make them myself.

    Leave a comment:


  • fool
    replied
    Originally posted by MrMarkyMark View Post
    Seems like you lived up to your user name on this occasion
    Indeed, but it was a good time to learn the lesson that anyone can screw up...

    Leave a comment:


  • MrMarkyMark
    replied
    Originally posted by fool View Post
    In my first dev role about 10 years ago, the Lead (only other) Dev was editing files in production and expected that I would do the same.

    I said, sod that I'm making a development environment. I exported production (my first job lads and I had no other access to the schema or any way of producing test data quickly) and imported into a dev version.

    It failed because of some MySQL configuration values which after a quick google suggested I needed to add a flag, fair enough, I'll just drop this...

    CEO - "Why is production down"
    ME - *Gulp*

    To be fair we restored from backup and I finished setting up dev environments.
    Seems like you lived up to your user name on this occasion

    Leave a comment:


  • fool
    replied
    In my first dev role about 10 years ago, the Lead (only other) Dev was editing files in production and expected that I would do the same.

    I said, sod that I'm making a development environment. I exported production (my first job lads and I had no other access to the schema or any way of producing test data quickly) and imported into a dev version.

    It failed because of some MySQL configuration values which after a quick google suggested I needed to add a flag, fair enough, I'll just drop this...

    CEO - "Why is production down"
    ME - *Gulp*

    To be fair we restored from backup and I finished setting up dev environments, SCM, etc.

    Leave a comment:


  • OwlHoot
    replied
    Originally posted by fullyautomatix View Post
    Ages ago I wrote a small web app for a local sandwich company which was essentially ordering a customised sandwich. The data was then emailed to a third party who would fax it to the kitchen. No database involved since they wanted it done cheaply and quickly.

    Anyway, all done, tested on all environments and before go live I wanted to run some automated tests and decided that the best way was to comment out the line of code that sent the email, since I did not want the kitchen to start making sandwiches that I was testing.

    On go live, I completely forgot about this and the site was launched and appeared to be busy. But no orders reached the kitchen and the phones went off the hook with customers angry that their sandwich never was delivered.
    Was it you who tested a direct mail promotion to customers of a prestigious bank, and when it went live forgot to change the salutation from "Dear Rich Bastard" ?

    Leave a comment:


  • xoggoth
    replied
    Not had any major ones
    Not that have come out yet anyway. Done quite a bit of work on aircraft control systems. If you are on a 747 just make sure you don't flush the toilet at exactly 24301ft 7.5inches

    Leave a comment:


  • VectraMan
    replied
    Not an IT person, but I remember whilst working for a small company we'd got a new server with RAID5 which we hadn't seen before. I thought it was a good idea to test it by pulling a drive out whilst the system was running, which apparently you weren't meant to do. No data was lost or anything, but I think we ended up returning the drive and getting a warranty replacement.

    I can remember leaving something that logged to c:\log.txt in a release build that went out to customers. And I'm sure I've once or twice made the mistake of commenting out a line that enabled some major piece of functionality, forgetting, committing it and then getting back a bug report that major-new-piece-of-functionality wasn't working. Doh.

    Leave a comment:

Working...
X