I read blogs, I listen to podcasts, I look at conference talks, all about deployment, and the impression I’m getting is that everyone’s problem is how to serve one billion requests per nanosecond. What the heck, am I the only one who makes small sites? I suspected that the people who blog are usually the ones who achieve more impressive feats, and therefore that the blogs aren’t representative of the developer population. So I made a web survey and asked people whether their deployments serve more or less than ten requests per second on the busiest second of the day. I chose ten because usually you can easily serve ten requests per second from a single virtual server; but still, if you reach ten, you will start thinking at least about the possibility of needing to scale.
A web survey is not, however, representative of the population; the sample consists only of people who chose to respond to the survey. So when I was at DjangoCon Europe last week, I took the opportunity to ask the attendants to raise hands during a lightning talk. It’s still not representative, as the sample in that case consists only of people who chose to go to the conference. But it’s still an indication.
Both the web survey and the lightning talk question suggest that about half of the Django developers deal with small sites in terms of traffic (fewer than ten requests per second).
In both cases I also asked whether they use any kind of comet / server push / long polling / WebSockets, or any kind of similar technology that keeps a connection open. In the lightning talk I got about half yes and half no, whereas in the web survey I got about one seventh yes and the rest no.
My conclusion from this investigation is that very often all this talk about “scaling” can be misleading. If your sites have only a little traffic, thinking about scaling is usually going to be premature optimization.
Thanks to Paolo Melchiorre for the photo of my talk