Recent downtime

[Brad Reddekopp]

Two recent emails from the hosting company:

I am sorry to inform you that the server is currently facing a filesystem issue. We are running a manual fsck to bring everything back to normal. This should be done in a while.

Your data is safe at the moment so please do not worry about it.

Our apologies for any inconvenience and we will keep you updated on the status.

A few hours later:

First off, our apologies again for the downtime occured just now. We have finished running the filesystem check and the server is now back and running fine. We have been monitoring the situation and it seems to be stable now.

However to avoid more downtime we will continue to monitor the server for longer.

Please do not hesitate to contact us if you need any assistance.

[tism]

Argh.

Maybe I'm wrong, but I thought the days of "manual fsck" were long, long over. Modern filesystems have journals to protect it from corruption by powerloss. RAID protects filesystems from corruption by damaged disks. A situation which requires more than just replaying the journal (which takes seconds) means something has gone horribly wrong. A corrupted filesystem means data is *no longer safe*, unless a backup of it has been made prior to the error!