Oops

Sorry about that, folks. Looks like we had a drive fail. Second one this week.

Fortunately, this server has hardware RAID, so we didn't lose anything. Unfortuntely, the drive had a whole lot of crazy to get out of its system before going into that good night, and took the server down with it.

We might need another brief outage to get the drive replaced, but other than that, we should be good.

Update: YES, I DO KNOW ABOUT HOT-SWAP DRIVES THANKS! Whether that will actually work perfectly with these drives, this controller, and this kernel, is something I don't know and can't easily test, since I only have one of these servers, which is why I'm allowing for a maintenance window.

We now return you to our regular program of... Uh, stuff.

Otherwords [Gabe]: Hoowee, tsk tsk tsk, guys. Don't want to piss a boy named Pixy off. He controls your sun and stars (also known as this website), y'know. ;)

Update: The RAID controller is reporting a "bad stripe", which apparently means that the whole thing could die at any time, so I get to back up sixteen million files* to another server, then reformat the entire thing, and then copy them all back again. It might be fixable without doing that, but attempting the fix might destroy the drive array, so I have to do the backup regardless.

Enjoy your long weekend, folks, because we might be in for a few hours of downtime.

* Yes, really.

posted by Pixy Misa at 02:49 PM

| Access Comments