recovery

First day of the year in the office, and a call came up "site isn't working". Browsing to the site, I noticed it couln't connect to the database. Thinking it was probably some glitch, I logged in remotely to the server, to find that the filesystem was readonly. After trying to figure out why me, being root, could not do anything to the filesystem... it rebooted. My first filesystem crash.