O'Reilly logo

Introduction to Tornado by Brendan Berg, Allison Parrish, Michael Dory

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 1. Introduction

Over the last half decade, the tools available to web developers have grown by leaps and bounds. As technologists continue to push the limits of what web applications can do for users everywhere, we’ve had to upgrade our toolkit and create frameworks that let us build better applications. We would like to be able to use new toolkits that make it easier for us to write clean and maintainable code that scales efficiently when deployed to users all across the globe.

This brings us to talking about Tornado, a fantastic choice for writing powerful web applications that are simple to create, extend, and deploy. The three of us had all fallen in love with Tornado for its speed, simplicity, and scalability, and after trying it out on a few personal projects, we’ve put it to work in our day jobs. We’ve seen it increase developer speed (and happiness!) on projects large and small, and at the same time have been impressed time and again by its robustness and lightweight footprint.

This book is meant to be an overview of the Tornado web server, and will walk readers through the basics of the framework, some sample applications, and best practices for use in the real world. We’ll use examples to detail how Tornado works, what you can do with it, and what you’d be best avoiding as you build your first applications with it.

In this book, we’ll be assuming that you have at least a rough understanding of Python, a sense of how web services work, and a basic familiarity with databases. For more on any of those, there are some great books to consult (including Learning Python, Restful Web Services, and MongoDB: The Definitive Guide).

And so you can follow along, the code for the examples in this book is available on GitHub. If you have any thoughts on these samples or anything else, we’d love to hear from you there.

So, without further ado, let’s dive in!

What Is Tornado?

Tornado is a powerful, scalable web server written in Python. It’s robust enough to handle serious web traffic, yet is lightweight to set up and write for, and can be used for a variety of applications and utilities.

The Tornado we now know is based on a web server framework that was first developed by Bret Taylor and others for FriendFeed, and later open sourced by Facebook when they acquired FriendFeed. Unlike traditional web servers that maxed out at around 10,000 simultaneous connections, Tornado was written with performance in mind, aiming to solve the C10K problem, so by design it’s an extremely high-performance framework. It’s also packed with tools for dealing with security and user authentication, social networks, and asynchronous interaction with external services like databases and web APIs.

Since its release on September 10, 2009, Tornado has garnered a lot of community support, and has been adopted to fit a variety of purposes. In addition to FriendFeed and Facebook, a host of companies have turned to Tornado in production, including Quora, Turntable.fm, Bit.ly, Hipmunk, and MyYearbook, to name a few.

In short, if you’re looking for a replacement for your giant CMS or monolithic development framework, Tornado is probably not the way to go. Tornado doesn’t require that you have giant models set up a particular way, or handle forms in a certain fashion, or anything like that. What it does do is let you write super fast web applications quickly and easily. If you want to create a scalable social application, real-time analytics engine, or RESTful API—all with the power and simplicity of Python—then Tornado (and this book) is for you!

Getting Started with Tornado

Installing Tornado on most *nix systems is easy—you can either get it from PyPI (and install via easy_install or pip), or download the source from GitHub and build it like this:

$ curl -L -O http://github.com/downloads/facebook/tornado/tornado-2.1.1.tar.gz
$ tar xvzf tornado-2.1.1.tar.gz
$ cd tornado-2.1.1
$ python setup.py build
$ sudo python setup.py install

Tornado is not officially supported on Windows, but it can be installed via ActivePython’s PyPM package manager like so:

C:\> pypm install tornado

Once Tornado is installed on your machine, you’re good to go! A bunch of demos are included with the package, which include examples for building a blog, integrating with Facebook, running a chat server, and more. We’ll be walking through some sample applications step by step later in this book, but be sure to have a look at these later for reference as well.

Caution

We’re assuming for these examples that you are using a Unix-based system and have Python 2.6 or 2.7 installed. If so, you won’t need anything aside from the Python standard library. You can run Tornado under Python 2.5 provided you have installed pycURL, simpleJSON, and the Python development headers, and on Python 3.2 with the distribute package. However, you should note that Python 3+ support is new as of Tornado 2.0, and the Tornado team has advised developers to continue to keep an eye out for bugs on that front.

Community and Support

For questions, examples, and general how-to’s, the official Tornado documentation is a great place to start. There’s a variety of examples and breakdowns of features at tornadoweb.org, and more specific details and changes can be seen at Facebook’s Tornado repository on GitHub. For more specific concerns, the Tornado Web Server Google Group is active and full of folks who use Tornado on a daily basis.

Simple Web Services

Now that we’ve covered what Tornado is, let’s look at what it can do. To start, we’ll go over the basics of writing a simple web service with Tornado.

Hello Tornado

Tornado is a framework for writing responses to HTTP requests. Your job as a programmer is to write “handlers” that respond to HTTP requests that match particular criteria. Here’s a basic example of a fully functional Tornado application:

Example 1-1. The basics: hello.py
import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web

from tornado.options import define, options
define("port", default=8000, help="run on the given port", type=int)

class IndexHandler(tornado.web.RequestHandler):
    def get(self):
        greeting = self.get_argument('greeting', 'Hello')
        self.write(greeting + ', friendly user!')

if __name__ == "__main__":
    tornado.options.parse_command_line()
    app = tornado.web.Application(handlers=[(r"/", IndexHandler)])
    http_server = tornado.httpserver.HTTPServer(app)
    http_server.listen(options.port)
    tornado.ioloop.IOLoop.instance().start()

Most of the work in making a Tornado application is to define classes that extend the Tornado RequestHandler class. In this case, we’ve made a simple application that listens for requests on a given port, and responds to requests to the root resource ("/").

Try running the program yourself on the command line to test it out:

$ python hello.py --port=8000

Now you can go to http://localhost:8000/ in a web browser, or open up a separate terminal window to test out the application with curl:

$ curl http://localhost:8000/
Hello, friendly user!
$ curl http://localhost:8000/?greeting=Salutations
Salutations, friendly user!

Let’s break this example down into smaller chunks and analyze them one by one:

import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web

At the top of the program, we import various Tornado libraries. There are other helpful libraries included with Tornado, but you’ll need to import at least these four to get this example running:

from tornado.options import define, options
define("port", default=8000, help="run on the given port", type=int)

Tornado includes a helpful library (tornado.options) for reading options from the command line. We make use of that library here to let us specify which port our application will listen on for HTTP requests. Here’s how it works: any option in a define statement will become available as an attribute of the global options object, if an option with the same name is given on the command line. If the user runs the program with the --help parameter, the program will print out all of the options you’ve defined, along with the text you specified with the help parameter in the call to define. If the user fails to provide a value for an option we specified, the default value for that option will be used instead. Tornado uses the type parameter to do basic type checking on the parameter, throwing an error if a value of an inappropriate type is given. Our line, therefore, allows the user to use an integer port argument, which we can access in the body of the program as options.port. If the user doesn’t specify a value, it defaults to 8000.

class IndexHandler(tornado.web.RequestHandler):
    def get(self):
        greeting = self.get_argument('greeting', 'Hello')
        self.write(greeting + ', friendly user!')

This is a Tornado request handler class. When handling a request, Tornado instantiates this class and calls the method corresponding to the HTTP method of the request. In this example, we’ve defined only a get method, meaning that this handler will respond only to HTTP GET requests. We’ll look at handlers that implement more than one HTTP method later.

greeting = self.get_argument('greeting', 'Hello')

Tornado’s RequestHandler class has a number of useful built-in methods, including get_argument, which we use here to get an argument greeting from the query string. (If no such argument is present in the query string, Tornado will use the second argument provided to get_argument, if any, as a default.)

self.write(greeting + ', friendly user!')

Another method of the RequestHandler class is write, which takes a string as a parameter and writes that string into the HTTP response. Here, we take the string supplied in the request’s greeting parameter, interpolate it into a greeting, and write it back in the response.

if __name__ == "__main__":
    tornado.options.parse_command_line()
    app = tornado.web.Application(handlers=[(r"/", IndexHandler)])

These are the lines that actually make the Tornado application run. First, we use Tornado’s options library to parse the command line. Then we create an instance of Tornado’s Application class. The most important argument to pass to the __init__ method of the Application class is handlers. This tells Tornado which classes to use to handle which requests. More on this in a moment.

http_server = tornado.httpserver.HTTPServer(app)
http_server.listen(options.port)
tornado.ioloop.IOLoop.instance().start()

From here on out, this code is boilerplate: once it has been created, we can pass the Application object to Tornado’s HTTPServer object, which then listens to the port we specified on the command line (retrieved through the options object). Finally, we create an instance of Tornado’s IOLoop, after which point the program is ready to accept HTTP requests.

The handlers Parameter

Let’s take a look at one line from the hello.py example again:

app = tornado.web.Application(handlers=[(r"/", IndexHandler)])

The handlers parameter here is important, and worth looking at in further detail. It should be a list of tuples, with each tuple containing a regular expression to match as its first member and a RequestHandler class as its second member. In hello.py, we specified only one regular expression RequestHandler pair, but you can put as many of these pairs into the list as needed.

Specifying paths with regular expressions

Tornado uses the regular expression in the tuples to match the path of the HTTP request. (The path is the portion of the URL that follows the hostname, excluding the query string and fragment.) Tornado treats these regular expressions as though they contain beginning-of-line and end-of-line anchors (i.e., the string "/" is assumed to mean "^/$").

When a regular expression has a capture group in it (i.e., a portion of the regular expression is enclosed in parentheses), the matching contents of that group will be passed to the RequestHandler object as parameters to the method corresponding to the HTTP request. We’ll see how this works in the next example.

String Service

Example 1-2 is a more sophisticated example program that illustrates what we’ve gone over so far and introduces a few more basic Tornado concepts.

Example 1-2. Handling input: string_service.py
import textwrap

import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web

from tornado.options import define, options
define("port", default=8000, help="run on the given port", type=int)

class ReverseHandler(tornado.web.RequestHandler):
    def get(self, input):
        self.write(input[::-1])

class WrapHandler(tornado.web.RequestHandler):
    def post(self):
        text = self.get_argument('text')
        width = self.get_argument('width', 40)
        self.write(textwrap.fill(text, width))

if __name__ == "__main__":
    tornado.options.parse_command_line()
    app = tornado.web.Application(
        handlers=[
            (r"/reverse/(\w+)", ReverseHandler),
            (r"/wrap", WrapHandler)
        ]
    )
    http_server = tornado.httpserver.HTTPServer(app)
    http_server.listen(options.port)
    tornado.ioloop.IOLoop.instance().start()

As with the first example, you can run this program on the command line by typing the following:

$ python string_service.py --port=8000

The program is a basic framework for an all-purpose web service for string manipulation. Right now, you can do two things with it. First, GET requests to /reverse/string returns the string specified in the URL path in reverse:

$ curl http://localhost:8000/reverse/stressed
desserts

$ curl http://localhost:8000/reverse/slipup
pupils

Second, POST requests to the /wrap resource will take text specified in an argument text and return that text, wrapped to the width specified in an argument named width. The following request specifies a string but no width, so the output is wrapped to the default width specified in the program’s get_argument call, 40 characters:

$ curl http://localhost:8000/wrap »
-d text=Lorem+ipsum+dolor+sit+amet,+consectetuer+adipiscing+elit.
Lorem ipsum dolor sit amet, consectetuer
adipiscing elit.
Note

The cURL command just shown was broken onto two lines for formatting reasons, but should be typed as a single line. As a convention, we will use the right double quote character (») to indicate a line continuation.

The string service example shares most of its code with the example presented in the previous section. Let’s zero in on some parts of the code that are new. First, let’s look at the value passed in the handlers parameter to the Application constructor:

app = tornado.web.Application(handlers=[
    (r"/reverse/(\w+)", ReverseHandler),
    (r"/wrap", WrapHandler)
])

In the previous code, the Application class is instantiated with two RequestHandlers in the “handlers” parameter. The first directs Tornado to send requests whose path matches the following regular expression:

/reverse/(\w+)

This regular expression tells Tornado to match any path beginning with the string /reverse/ followed by one or more alphanumeric characters. The parentheses tell Tornado to save the string that matched inside the parentheses, and pass that string to the RequestHandler’s request method as a parameter. Check out the definition of ReverseHandler to see how it works:

class ReverseHandler(tornado.web.RequestHandler):
    def get(self, input):
        self.write(input[::-1])

You can see here that the get method takes an additional parameter input. This parameter will contain whatever string was matched inside the first set of parentheses in the regular expression that matched the handler. (If there are additional sets of parentheses in the regular expression, the matched strings will be passed in as additional parameters, in the same order as they occurred in the regular expression.)

Now, let’s take a look at the definition of WrapHandler:

class WrapHandler(tornado.web.RequestHandler):
    def post(self):
        text = self.get_argument('text')
        width = self.get_argument('width', 40)
        self.write(textwrap.fill(text, width))

The WrapHandler class handles requests that match the path /wrap. This handler defines a post method, meaning that it accepts requests with an HTTP method of POST.

We’ve previously used the RequestHandler object’s get_argument method to grab parameters off of a request’s query string. It turns out we can use the same method to get parameters passed into a POST request. (Tornado understands POST requests with URL-encoded or multipart bodies.) Once we’ve grabbed the text and width arguments from the POST body, we use Python’s built-in textwrap library to wrap the text to the specified width, and write the resulting string to the HTTP response.

More About RequestHandlers

So far, we’ve explored the bare basics of RequestHandler objects: how to get information from an incoming HTTP request (using get_argument and the parameters passed to get and post) and how to write an HTTP response (using the write method). There’s a lot more to learn, which we’ll get to in subsequent chapters. In the meantime, here are a few things to keep in mind about RequestHandler and how Tornado uses it.

HTTP methods

In the examples discussed so far, each RequestHandler class has defined behavior for only one HTTP method. However, it’s possible—and useful—to define multiple methods in the same handler. This is a good way to keep conceptually related functionality bundled into the same class. For example, you might write one handler for both a GET and a POST to an object in a database with a particular ID. Here’s an imaginary example, in which the GET method for a widget ID returns information about that widget, and the POST method makes changes to the widget with that ID in the database:

# matched with (r"/widget/(\d+)", WidgetHandler)
class WidgetHandler(tornado.web.RequestHandler):
    def get(self, widget_id):
        widget = retrieve_from_db(widget_id)
        self.write(widget.serialize())

    def post(self, widget_id):
        widget = retrieve_from_db(widget_id)
        widget['foo'] = self.get_argument('foo')
        save_to_db(widget)

We’ve used only GET and POST in our examples so far, but Tornado supports any valid HTTP method (GET, POST, PUT, DELETE, HEAD, OPTIONS). You can define behavior for any of these methods simply by defining a method in your RequestHandler class with a matching name. The following is another imaginary example, in which a HEAD request for a particular frob ID gives information only concerning whether or not the frob exists, while the GET method returns the full object:

# matched with (r"/frob/(\d+)", FrobHandler)
class FrobHandler(tornado.web.RequestHandler):
    def head(self, frob_id):
        frob = retrieve_from_db(frob_id)
        if frob is not None:
            self.set_status(200)
        else:
            self.set_status(404)
    def get(self, frob_id):
        frob = retrieve_from_db(frob_id)
        self.write(frob.serialize())

HTTP status codes

As shown in the previous example, you can explicitly set the HTTP status code of your response by calling the set_status() method of the RequestHandler. It’s important to note, however, that Tornado will set the HTTP status code of your response automatically under some circumstances. Here’s a rundown of the most common cases:

404 Not Found

Tornado will automatically return a 404 (Not Found) response code if the path of the HTTP request doesn’t match any pattern associated with a RequestHandler class.

400 Bad Request

If you call get_argument without a default, and no argument with the given name is found, Tornado will automatically return a 400 (Bad Request) response code.

405 Method Not Allowed

If an incoming request uses an HTTP method that the matching RequestHandler doesn’t define (e.g., the request is POST but the handler class only defines a get method), Tornado will return a 405 (Method Not Allowed) response code.

500 Internal Server Error

Tornado will return 500 (Internal Server Error) when it encounters any errors that aren’t severe enough to cause the program to exit. Any uncaught exceptions in your code will also cause Tornado to return a 500 response code.

200 OK

If the response was successful and no other status code was set, Tornado will return a 200 (OK) response code by default.

When one of the errors above occurs, Tornado will by default send a brief snippet of HTML to the client with the status code and information about the error. If you’d like to replace the default error responses with your own, you can override the write_error method in your RequestHandler class. For example, Example 1-3 shows our initial hello.py example, but with custom error messages.

Example 1-3. Custom error responses: hello-errors.py
import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web

from tornado.options import define, options
define("port", default=8000, help="run on the given port", type=int)

class IndexHandler(tornado.web.RequestHandler):
    def get(self):
        greeting = self.get_argument('greeting', 'Hello')
        self.write(greeting + ', friendly user!')
    def write_error(self, status_code, **kwargs):
        self.write("Gosh darnit, user! You caused a %d error." % status_code)

if __name__ == "__main__":
    tornado.options.parse_command_line()
    app = tornado.web.Application(handlers=[(r"/", IndexHandler)])
    http_server = tornado.httpserver.HTTPServer(app)
    http_server.listen(options.port)
    tornado.ioloop.IOLoop.instance().start()

The following response is what happens when we attempt to POST to this handler. Normally, we would get Tornado’s default error response, but because we’ve overridden write_error, we get something else:

$ curl -d foo=bar http://localhost:8000/
Gosh darnit, user! You caused a 405 error.

Next Steps

By now you’ve got the basics under your belt, and we hope you’re hungry for more. In the upcoming chapters, we’ll show features and techniques that will help you use Tornado to build full-blown web services and web applications. First up: Tornado’s template system.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required