You are currently viewing our forum as a guest, which gives you limited access to view most discussions and access our other features. By joining our community, at no cost, you will have access to post topics, communicate privately with other members (PM), respond to polls, upload content and access many other special features. Registration is free, fast and simple, so please join our community today!
Apparently someone decied to go into the cabinet where the servers are and then kick the power on the server. Then they decided to rewire their server and somehow plugged out network cable back into the wrong port.
Sorry for the downtime, we'll be looking for a new host now.
I thought maybe you guys had changed DNS or something. Yeh, that's bad. I had a server colo'd here in Virginia a few years back. Some very dumb datacenter employee was tasked with reloading the OS on a customer's box. Said customer just happened to have their box next to mine, long story short, tech reloaded OS on wrong server, mine.
They took 24 production websites off line and offered me not even a month of free service. Their CEO told me to go F myself basically. That guy is still on my list...
__________________ 2004 Honda CBR 600RR current 2009 Suzuki DL650 V-Strom current 2009 Subaru WRX - current 2011 Nissan Pathfinder 4x4 current 1975 Datsun Fairlady Z RHD Project 2004.5 Nissan 350Z (gone, but not forgotten) 2008.5 Nissan Titan(Goodbye my monster truck)
Apparently someone decied to go into the cabinet where the servers are and then kick the power on the server. Then they decided to rewire their server and somehow plugged out network cable back into the wrong port.
Sorry for the downtime, we'll be looking for a new host now.
LOL that is a common problem with datacenter operations...
__________________ 2004 Honda CBR 600RR current 2009 Suzuki DL650 V-Strom current 2009 Subaru WRX - current 2011 Nissan Pathfinder 4x4 current 1975 Datsun Fairlady Z RHD Project 2004.5 Nissan 350Z (gone, but not forgotten) 2008.5 Nissan Titan(Goodbye my monster truck)
I thought maybe you guys had changed DNS or something. Yeh, that's bad. I had a server colo'd here in Virginia a few years back. Some very dumb datacenter employee was tasked with reloading the OS on a customer's box. Said customer just happened to have their box next to mine, long story short, tech reloaded OS on wrong server, mine.
They took 24 production websites off line and offered me not even a month of free service. Their CEO told me to go F myself basically. That guy is still on my list...
Shoot, I'm glad that my company has its own rows of locked racks... avoid something like this...
__________________ SHIFT_norcal ~ SF Bay Area ~ Breaking Bad, Homeland, Dexter, GSW (SFW), SF Giants, Jdorama, SFII, IT croo
LOL that is a common problem with datacenter operations...
I think a lot of datacenter employees get fatigued from the constant monotony of the job and do stuff like that. Not really an excuse though, when you consider they are entrusted with caring for the lifeblood of a lot of companies.
Shoot, I'm glad that my company has its own rows of locked racks... avoid something like this...
Yeh but don't the datacenter employees have access through your locks. If not, you have no remote hands assistance. I've had servers in Texas and California before, I couldn't lock their people out unless I'm going to hop on a plane everytime there's a problem.
Yeh but don't the datacenter employees have access through your locks. If not, you have no remote hands assistance. I've had servers in Texas and California before, I couldn't lock their people out unless I'm going to hop on a plane everytime there's a problem.
You said that another customer's box was [presumably] in your rack. And that they mixed up another customer's with yours...? All of our rows of racks have our own servers... so mimimal risk of mixing up our servers with other customers.
Also, our colo is only a few minutes away from our centralized [primary] operations.
__________________ SHIFT_norcal ~ SF Bay Area ~ Breaking Bad, Homeland, Dexter, GSW (SFW), SF Giants, Jdorama, SFII, IT croo
Kyle was in range before he moved, we are looking to relocate closer to one of us for this exact reason.
Quote:
Originally Posted by yobri
You said that another customer's box was [presumably] in your rack. And that they mixed up another customer's with yours...? All of our rows of racks have our own servers... so mimimal risk of mixing up our servers with other customers.
Also, our colo is only a few minutes away from our centralized [primary] operations.
__________________ 04 G Sedan Greddy Twin Turbo and Built Motor by UMS Tuning 649whp/651wtq Rotora (12pot) 20" iForged Aeros Quad Projector Retro (S2k/TL) Full Custom, Dual 3" Dual Muffler Exhaust by Nichols AutoFab
I thought you guys did not pay your electric bill . . .
__________________
MY STABLE: 1991 Red Mazda Miata | 1996 White BMW 318i | 1996 Red BMW 328i | 2003 Red Nissan Frontier | 2005 Red Nissan Titan | 2006 Silverstone Nissan 350Z | 2006 Red Porsche Cayenne S
Sort of reverse happed to my boss before I got hired on, he called the cage monkeys to hit the power on a router and after they said they had done it, he still could not attach to it, after about four rounds of "we just recycled the power again"/"I still can't get on" he drove to the datacenter and the cage monkey walked him over to his rack, but it was not his rack, his was one row over. This guy had been rebooting someone else's router.
Once I got on board, we moved all our stuff over to a different facility and have been trouble free for over four years. The real beauty for me is our new datacenter is less than a mile from my house
you would not believe the guy that I had on the phone last night helping me.. oh the pain that was.. You would think they would hire someone with more brains than that..
Outage: 7/30
