History of the URL, or why you are confused by serving many domains with a single server

I’ve noticed that many people attempting to deploy Django have trouble understanding how a single web server installation can serve many domains. One reason is that words matter. What Apache calls a “virtual host” is exactly what nginx calls a “server” and what HTTP calls a “host”. It is neither a host (let alone a virtual one) nor a server; it’s a domain. Let’s clear this up.

The first version of the HyperText Transfer Protocol was very simple. If your browser wanted to visit page http://djangodeployment.com/apage, it would connect to the server djangodeployment.com, port 80, and after the TCP connection was established it would send this line to the server, terminated with a newline:

GET /apage

The server would then respond with the content of that page and close the connection. The content of the page was usually something like this:

<html>
  <head>
    <title>Hello</title>
  </head>
  <body>
    <p>hello, world</p>
  </body>
</html>

In that era, a single computer could not distinguish different domains in the request. Whether you visited http://djangodeployment.com/apage or http://71.19.145.109/apage you’d get the same response. This is because the whole syntax of the URL, protocol://host[:port]/path, assumed you would get a path from a server. In that context, “host” is synonymous to “server”. At that time, no-one had thought of serving many domains from a single computer (having even a single web site was such a novelty that no-one was dreaming of having a second one).

Soon HTTP was enhanced to what we still use today. The GET directive is followed by a list of “HTTP headers”, terminated with an empty line.

GET /apage HTTP/1.1
Host: djangodeployment.com
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0

And the response is like this:

HTTP/1.1 200 OK
Date: Wed, 15 Feb 2017 15:04:50 GMT
Server: Apache/2.4.10 (Debian)
Last-Modified: Wed, 15 Feb 2017 15:02:36 GMT
Content-Length: 102
Content-Type: text/html

<html>
  <head>
    <title>Hello</title>
  </head>
  <body>
    <p>hello, world</p>
  </body>
</html>

It’s the “Host” header in the request that makes it possible for a single server installation to respond to each domain differently. However, the word that was chosen for that header, “Host”, was wrong since its conception—the correct one would have been “Domain”. Since the format of the URL was protocol://host[:port]/path, the HTTP designers apparently thought that you were asking the server for a specific “host”. And because, in that context, “host” is synonym for “server”, they thought that a single physical host can serve many “virtual hosts”. Hence the confusion that persists to this day.

The moral is that you should mentally translate the currently used keywords into others. Specifically, you should understand the HTTP “Host” header as “domain”; the Apache “VirtualHost”, “ServerName” and “ServerAlias” directives as “domain”, “domain name” and “domain alias”; and the nginx “server” and “server_name” directives as “domain” and “domain name”.

Incidentally, it’s not only the words that are wrong. In fact, the whole design of the URL is suboptimal. I don’t, of course, hold anything against its designers; I wouldn’t have done better in their place, and it’s Tim Berners Lee himself who said the syntax is clumsy. In a URL like https://www.djangoproject.com/start/overview/, the “host” part, that is, the domain, goes from the most specific (djangoproject) to the most general (com), whereas the path goes from the most general to the most specific. What’s more, why should the user care about the protocol, the host, and the port? The URL should be something like “/com/djangoproject/start/overview”; the web browser would ask the DNS where to find it, and the DNS would reply “to get this resource, connect to 146.20.110.22, port 80, with HTTP”. While in 1990 the URL design looked cool, it’s the root of today’s confusion.

External link: Tim Berners Lee on the design of the URL