Hardware failure

Look on the bright side: as the place grows, singular anything just won't handle the load, so you'll eventually have at least two of everything. Four is a good number: dead, replacement live, standby for the replacement in case it dies, the one to copy from as you bring the dead one back again. But it takes a while to get there...

Next stop after that is geographic redundancy so you can handle your data center burning down, flooding or both and carry on going. Or the more banal dead transit provider or key colo switch. That's even more of a challenge, particularly for forums, where the software just isn't built to provide web-scale robust enterprise-critical service at the best in the world load range. Not yet. You might change that for vBulletin.

You have kids to drive to school so you get a car. As your family grows you might need two or three cars to get them there on time. If one car breaks down you can handle that load of kids in the others. It's the people with a few children who are most vulnerable because they only need and have one car. Same for web sites and computers: as you grow you need more and become less harmed by one failing.

Eventually as your family grows you might think that it'd be nice to have a second house so that you can fit with less crowding. That also gets you protection if one of them burns down. Might be crowded for a while but at least you still have somewhere to sleep. Same for web sites: as reliability becomes more important you start to look at putting some of your computers in a different place so that a failure at one place won't leave everyone unable to get to the site. It's called geographic redundancy: data centers spread around the country or the world. The sort of thing Google, Yahoo and Amazon do.

MSE is still a bit small to be very worried about geographic redundancy. It tends to be quite expensive and require lots of really careful and often expensive programming work to be able to do it.

Eventually the cost of downtime or harmed reputation will make the investment seem worthwhile to Martin as the site grows and people start to just expect it to always be there. Only he knows what his thresholds will be for this. And when the technical people will have enough time free to get it done while dealing with all of the day to day problems that happen behind the scenes.

The MSE people (and Martin spending money!) are doing really well just to keep up as they have. It's really technically tough to do when growing rapidly.

HariboJunkie wrote: »

Pardon?

He lost me at "dead transit provider"

Well done, MSE, in continuing to stay available in what are clearly difficult circumstances.

I don't know if you want feedback on any problems people have noticed. If you do, I thought I'd let you know that some threads aren't displaying properly. For instance, if you click on the stickies in the Bankruptcy forum, the content displayed doesn't relate to the appropriate page number - it's as if the content of the first page is missing. Will I need to re-do the content of the sticky I created? I hope not.:eek: There's a similar problem with the Poundstretcher thread - posts done earlier this morning display as being on page 7, when they should be on the last page.

Anyway, keep up the good work. Hopefully you'll be back to normal soon.

I checked that too, & the computer I did it on died recently, so I'm not sure I've got the file archived. I'll have a good look today.