katrinacheckin.org fallout

What a strange 24 hours.
As the last post says, I launched KatrinaCheckIn.org yesterday morning, around 2:30 a.m. By the time I woke up, around 8:00, I had CNN on the phone asking me to do an on-air interview the next morning. I also had visitors pouring in at the at-the-time-unbelievable rate of 1,500 an hour.
People were posting about their conditions and their loved ones — heartwrenching stuff in many cases. I was happy, because the site was fulfilling its purpose. There were a few other similiar sites set up at about the same time, but I think that KatrinaCheckIn.org was the largest of the independent ones. (Sites set up by the Times-Picayune and WWL-TV were probably the same size or a bit bigger.)
Anyway, I was frustrated when the site started crashing around 9:30 a.m. I wrote to my web host — which, for the record, is not the crappy one I recently tossed to the curb — to complain. The server ended up fixed, but it failed again a couple hours later. When the site was up, it was grindingly slow. I was a little mad.
Which was dumb. Because, as it turns out, I was the one crashing the server. Because KatrinaCheckIn.org was drawing in an absolutely insane number of hits.
In the end, my host — completely reasonably — had to take the site down. Turns out the site was on pace to pull in 84 million hits in a single day. (To put that in context, here’s a note from my host: “Companies like realtor.com have 60 load-balanced web servers, a large staff (and they spend 40 million a month) to do 20 million a day without issues. Yahoo’s Sept 11th memorial 3 day ‘event’ did 64 million hits over 3 days, took months to plan, was done on 500 servers and was fully staffed. That’s the kind of traffic that showed up here.”)
The wholesale bandwidth alone for a day’s traffic on KatrinaCheckIn.org was going to be $18,000. The site was generating 1,000 requests to the web server per second right before being shut down. (Again, from my host, “Wikipedia for example does a 1,000 requests/second on a 64-server cluster with about a dozen of those being squid proxy caches.” I don’t know what a squid proxy cache is — although it sounds totally awesome — but Wikipedia’s 64 servers compare quite favorably to the one server KatrinaCheckIn.org was on. And it shared that server with hundreds of other sites — crabwalk.com among them.)
So — what now? The terrific people at Textdrive are working on a few possible routes to bringing it back without melting servers and emptying my 401(k). May have word soon. I’m hopeful. But I wanted those who used the site to know I’m sorry it disappeared, and that I’m working to bring it back.
(FYI, CNN called late last night and cancelled the interview — presumably because they realized there was no site left to talk about.)

4 thoughts on “katrinacheckin.org fallout”

  1. Akamai, Amazon, etc., might be interested in hosting. I’m not sure precisely how to reach the right people there to ask for help. Amazon is already hosting the Red Cross hurricane relief site.

Comments are closed.