MongoDB and Python

Chapter 4. MongoDB with Web Frameworks

While MongoDB can be used in all sorts of applications, its most obvious role is as the database backend for a web application. These days, a great many mobile and tablet applications are functioning as “fat clients” to the same HTTP-based API’s as browser-based web applications; hence mobile and tablet apps need the same sort of backend database infrastructure as more traditional web apps.

Many organizations and engineers are finding the advantages of MongoDB’s document-oriented architecture compelling enough to migrate parts or even entire applications from traditional RDBMS such as MySQL to MongoDB. Numerous well-known companies have built their whole application from the ground up on MongoDB.

It is my opinion that for the vast majority of web, mobile and tablet applications, MongoDB is a better starting point than RDBMS technology such as MySQL. This chapter is an attempt to get you off the ground using MongoDB with three common Python web frameworks: Pylons, Pyramid and Django.

Pylons 1.x and MongoDB

Pylons is one of the older WSGI-based Python web frameworks, dating back to September 2005. Pylons reached version 1.0 in 2010 and is considered very stable at this point. In fact, not much development is planned for Pylons 1.x any more; all new development is happening in Pyramid (see Pyramid and MongoDB for details). The Pylons philosophy is the precise opposite of “one-size-fits-all.” Application developers are free to choose from the various database, templating, session store options available. This kind of framework is excellent when you aren’t exactly sure what pieces you will need when you are starting work on your application. If it turns out you need to use an XML-based templating system, you are free to do so.

The existence of Pyramid aside, Pylons 1.x is a very capable and stable framework. As Pylons is so modular, it is easy to add MongoDB support to it.

First you need to create a virtual environment for your project. These instructions assume you have the virtualenv tool installed on your system. Install instructions for the virtualenv tool are provided in the first chapter of this book.

To create the virtual environment and install Pylons along with its dependencies, run the following commands:

virtualenv --no-site-packages myenv
cd myenv
source bin/activate
easy_install pylons

Now we have Pylons installed in a virtual environment. Create another directory named whatever you like in which to create your Pylons 1.x project, change your working directory to it, then execute:

paster create -t pylons

You will be prompted to enter a name for your project, along with which template engine you want to use and whether or not you want the SQLAlchemy Object-Relational Mapper (ORM). The defaults (“mako” for templating engine, False to SQLAlchemy) are fine for our purposes—not least since we are demonstrating a NoSQL database!

After I ran the paster create command, a “pylonsfoo” directory (I chose “pylonsfoo” as my project name) was created with the following contents:

MANIFEST.in
README.txt
development.ini
docs
ez_setup.py
pylonsfoo
pylonsfoo.egg-info
setup.cfg
setup.py
test.ini

Next you need to add the PyMongo driver as a dependency for your project. Change your working directory to the just-created directory named after your project. Open the setup.py file present in it with your favourite editor. Change the install_requires list to include the string pymongo. Your file should look something like this:

try:
    from setuptools import setup, find_packages
except ImportError:
    from ez_setup import use_setuptools
    use_setuptools()
    from setuptools import setup, find_packages

setup(
    name='pylonsfoo',
    version='0.1',
    description='',
    author='',
    author_email='',
    url='',
    install_requires=[
        "Pylons>=1.0", "pymongo",
    ],
    setup_requires=["PasteScript>=1.6.3"],
    packages=find_packages(exclude=['ez_setup']),
    include_package_data=True,
    test_suite='nose.collector',
    package_data={'pylonsfoo': ['i18n/*/LC_MESSAGES/*.mo']},
    #message_extractors={'pylonsfoo': [
    #        ('**.py', 'python', None),
    #        ('templates/**.mako', 'mako', {'input_encoding': 'utf-8'}),
    #        ('public/**', 'ignore', None)]},
    zip_safe=False,
    paster_plugins=['PasteScript', 'Pylons'],
    entry_points="""
    [paste.app_factory]
    main = pylonsfoo.config.middleware:make_app

    [paste.app_install]
    main = pylons.util:PylonsInstaller
    """,
)

Now you need to fetch the PyMongo driver into your virtual environment. It is easy to do this by executing:

python setup.py develop

Your Pylons app is now ready to be configured with a MongoDB connection. First, we shall create a config file for development

cp development.ini.sample development.ini

Next open the file development.ini in your favourite editor. Underneath the section [app:main] add the following two variables, changing the URI and database names to whatever works for your set up:

mongodb.url = mongodb://localhost
mongodb.db_name = mydb

You can now try starting your project with the following command:

paster serve --reload development.ini

You should see the following output:

Starting subprocess with file monitor
Starting server in PID 82946.
serving on http://127.0.0.1:5000

If you open the URL http://localhost:5000/ in a web browser, you should see the default Pylons page. This means that you have correctly set up your project. However, we do not yet have a way to talk to MongoDB.

Now that the configuration is in place, we can tell Pylons how to connect to MongoDB and where to make the PyMongo connection available to our application. Pylons provides a convenient place for this in <project_name>/lib/app_globals.py. Edit this file and change the contents to the following:

from beaker.cache import CacheManager
from beaker.util import parse_cache_config_options
from pymongo import Connection
from pylons import config

class Globals(object):
    """Globals acts as a container for objects available throughout the
    life of the application

    """

    def __init__(self, config):
        """One instance of Globals is created during application
        initialization and is available during requests via the
        'app_globals' variable

        """
        mongodb_conn = Connection(config['mongodb.url'])
        self.mongodb = mongodb_conn[config['mongodb.db_name']]
        self.cache = CacheManager(**parse_cache_config_options(config))

Once this has been set up, a PyMongo Database instance will be available to your Pylons controller actions through the globals object. To demonstrate, we will create a new controller named “mongodb” with the following command:

paster controller mongodb

You should see a file named mongodb.py in the <project_name>/controllers directory. For demonstration purposes, we shall modify it to increment a counter document in MongoDB every time the controller action is run.

Open this file with your editor. Modify it to look like the following (remembering to change the from pylonsfoo import line into whatever you named your project):

import logging

from pylons import app_globals as g, request, response, session, tmpl_context as c, url
from pylons.controllers.util import abort, redirect

from pylonsfoo.lib.base import BaseController, render

log = logging.getLogger(__name__)

class MongodbController(BaseController):

    def index(self):
        new_doc = g.mongodb.counters.find_and_modify({"counter_name":"test_counter"},
            {"$inc":{"counter_value":1}}, new=True, upsert=True , safe=True)
        return "MongoDB Counter Value: %s" % new_doc["counter_value"]

Once you have saved these changes, in a web browser open the URL http://localhost:5000/mongodb/index. Each time you load this page, you should see a document in the counters collection be updated with its counter_value property incremented by 1.