Fork me on GitHub

Web servers

Web servers respond to Hypertext Transfer Protocol (HTTP) requests from clients and send back a response containing a status code and often content such as HTML, XML or JSON as well.

Why are web servers necessary?

Web servers are the ying to the web client's yang. The server and client speak the standardized language of the World Wide Web. This standard language is why an old Mozilla Netscape browser can still talk to a modern Apache or Nginx web server, even if it cannot properly render the page design like a modern web browser can.

The basic language of the Web with the request and response cycle from client to server then server back to client remains the same as it was when the Web was invented by Tim Berners-Lee at CERN in 1989. Modern browsers and web servers have simply extended the language of the Web to incorporate new standards.

Web server implementations

The conceptual web server idea can be implemented in various ways. The following web server implementations each have varying features, extensions and configurations.

  • The Apache HTTP Server has been the most commonly deployed web server on the Internet for 20+ years.

  • Nginx is the second most commonly used server for the top 100,000 websites and often serves as a reverse proxy for Python WSGI servers.

  • Caddy is a newcomer to the web server scene and is focused on serving the HTTP/2 protocol with HTTPS.

Client requests

A client that sends a request to a web server is usually a browser such as Internet Explorer, Firefox, or Chrome, but it can also be a

  • headless browser, commonly use for testing, such as phantomjs
  • commandline utility, for example wget and cURL
  • text-based web browser such as Lynx
  • web crawler.

Web servers process requests from the above clients. The result of the web server's processing is a response code and commonly a content response. Some status codes, such as 204 (No content) and 403 (Forbidden), do not have content responses.

In a simple case, the client will request a static asset such as a picture or JavaScript file. The file sits on the file system in a location the web server is authorized to access and the web server sends the file to the client with a 200 status code. If the client already requested the file and the file has not changed, the web server will pass back a 304 "Not modified" response indicating the client already has the latest version of that file.

Web server and web browser request-response cycle

A web server sends files to a web browser based on the web browser's request. In the first request, the browser accessed the "www.fullstackpython.com" address and the server responded with the index.html HTML-formatted file. That HTML file contained references to other files, such as style.css and script.js that the browser then requested from the server.

Sending static assets (such as CSS and JavaScript files) can eat up a large amount of bandwidth which is why using a Content Delivery Network (CDN) to serve static assets is important when possible.

Web server resources

  • HTTP/1.1 Specification

  • A reference with the full list of HTTP status codes is provided by W3C.

  • 4 HTTP Security Headers You Should Always Be Using

  • If you're looking to learn about web servers by building one, here's part one, part two and part three of a great tutorial that shows how to code a web server in Python.

  • rwasa is a newly released web server written in Assembly with no external dependencies that tuned to be faster than Nginx. The benchmarks are worth taking a look at to see if this server could fit your needs if you need the fastest performance trading off for as of yet untested web server.

Web servers learning checklist

  1. Choose a web server. Nginx is often recommended although Apache is also a great choice.

  2. Create an SSL certificate. For testing use a self-signed certificate and for a production app buy one from Digicert. Configure the web server to serve traffic over SSL. You'll need SSL for serving only HTTPS traffic and preventing security issues that occur with unencrypted user input.

  3. Configure the web server to serve up static files such as CSS, JavaScript and images.

  4. Once you set up the WSGI server you'll need to configure the web server as a pass through for dynamic content.

What do you want to learn after the web server is configured?

What runs a Python application execute on the server?

How should Python libraries be installed on a server?

I have a server, how do I set up the operating system?

Sign up here to receive a monthly email with major updates to this site, tutorials and discount codes for Python books.