Test-Driven Development with Python

Chapter 16. Server-Side Authentication and Mocking in Python

Let’s crack on with the server side of our new auth system. In this chapter we’ll do some more mocking, this time with Python. We’ll also find out about how to customise Django’s authentication system.

A Look at Our Spiked Login View

At the end of the last chapter, we had a working client side that was trying to send authentication assertions to our server’s login view. Let’s start by building that view, and then move inwards to build the backend authentication functions.

Here’s the spiked version of our login view:

def persona_login(request):
    print('login view', file=sys.stderr)
    #user = PersonaAuthenticationBackend().authenticate(request.POST['assertion'])
    user = authenticate(assertion=request.POST['assertion'])  #
    if user is not None:
        login(request, user)  #
    return redirect('/')

authenticate is our customised authentication function, which we’ll de-spike later. Its job is to take the assertion from the client side and validate it.

login is Django’s built-in login function. It stores a session object on the server, tied to the user’s cookies, so that we can recognise them as being an authenticated user on future requests.

Our authenticate function is going to make calls out, over the Internet, to Mozilla’s servers. We don’t want that to happen in our unit test, so we’ll want to mock out authenticate.

Mocking in Python

The popular mock package was added to the standard library as part of Python 3.3.^[27] It provides a magical object called a Mock, which is a bit like the Sinon mock objects we saw in the last chapter, only much cooler. Check this out:

>>> from unittest.mock import Mock
>>> m = Mock()
>>> m.any_attribute
<Mock name='mock.any_attribute' id='140716305179152'>
>>> m.foo
<Mock name='mock.foo' id='140716297764112'>
>>> m.any_method()
<Mock name='mock.any_method()' id='140716331211856'>
>>> m.foo()
<Mock name='mock.foo()' id='140716331251600'>
>>> m.called
False
>>> m.foo.called
True
>>> m.bar.return_value = 1
>>> m.bar()
1

A mock object would be a pretty neat thing to use to mock out the authenticate function, wouldn’t it? Here’s how you can do that.

Testing Our View by Mocking Out authenticate

(I trust you to set up a tests folder with a dunderinit. Don’t forget to delete the default tests.py, as well.)

accounts/tests/test_views.py. 

from django.test import TestCase
from unittest.mock import patch


class LoginViewTest(TestCase):

    @patch('accounts.views.authenticate')  #
    def test_calls_authenticate_with_assertion_from_post(
        self, mock_authenticate  #
    ):
        mock_authenticate.return_value = None  #
        self.client.post('/accounts/login', {'assertion': 'assert this'})
        mock_authenticate.assert_called_once_with(assertion='assert this')  #

The decorator called patch is a bit like the Sinon mock function we saw in the last chapter. It lets you specify an object you want to “mock out”. In this case we’re mocking out the authenticate function, which we expect to be using in accounts/views.py.

The decorator adds the mock object as an additional argument to the function it’s applied to.

We can then configure the mock so that it has certain behaviours. Having authenticate return None is the simplest, so we set the special .return_value attribute. Otherwise it would return another mock, and that would probably confuse our view.

Mocks can make assertions! In this case, they can check whether they were called, and what with.

So what does that give us?

$ python3 manage.py test accounts
[...]
AttributeError: <module 'accounts.views' from
'/workspace/superlists/accounts/views.py'> does not have the attribute
'authenticate'

We tried to patch something that doesn’t exist yet. We need to import authenticate into our views.py:^[28]

accounts/views.py. 

from django.contrib.auth import authenticate

Now we get:

AssertionError: Expected 'authenticate' to be called once. Called 0 times.

That’s our expected failure; to implement, we’ll have to wire up a URL for our login view:

superlists/urls.py. 

urlpatterns = patterns('',
    url(r'^$', 'lists.views.home_page', name='home'),
    url(r'^lists/', include('lists.urls')),
    url(r'^accounts/', include('accounts.urls')),
    # url(r'^admin/', include(admin.site.urls)),
)

accounts/urls.py. 

from django.conf.urls import patterns, url

urlpatterns = patterns('',
    url(r'^login$', 'accounts.views.persona_login', name='persona_login'),
)

Will a minimal view do anything?

accounts/views.py. 

from django.contrib.auth import authenticate

def persona_login():
    pass

Yep:

TypeError: persona_login() takes 0 positional arguments but 1 was given

And so:

accounts/views.py (ch16l008). 

def persona_login(request):
    pass

Then:

ValueError: The view accounts.views.persona_login didn't return an HttpResponse
object. It returned None instead.

accounts/views.py (ch16l009). 

from django.contrib.auth import authenticate
from django.http import HttpResponse

def persona_login(request):
    return HttpResponse()

And we’re back to:

AssertionError: Expected 'authenticate' to be called once. Called 0 times.

We try:

accounts/views.py. 

def persona_login(request):
    authenticate()
    return HttpResponse()

And sure enough, we get:

AssertionError: Expected call: authenticate(assertion='assert this')
Actual call: authenticate()

And then we can fix that too:

accounts/views.py. 

def persona_login(request):
    authenticate(assertion=request.POST['assertion'])
    return HttpResponse()

OK so far. One Python function mocked and tested.

Checking the View Actually Logs the User In

But our authenticate view also needs to actually log the user in by calling the Django auth.login function, if authenticate returns a user. Then it needs to return something other than an empty response—since this is an Ajax view, it doesn’t need to return HTML, just an “OK” string will do:

accounts/tests/test_views.py (ch16l011). 

from django.contrib.auth import get_user_model
from django.test import TestCase
from unittest.mock import patch
User = get_user_model()  #


class LoginViewTest(TestCase):
    @patch('accounts.views.authenticate')
    def test_calls_authenticate_with_assertion_from_post(
        [...]

    @patch('accounts.views.authenticate')
    def test_returns_OK_when_user_found(
        self, mock_authenticate
    ):
        user = User.objects.create(email='[e-mail]')
        user.backend = ''  # required for auth_login to work
        mock_authenticate.return_value = user
        response = self.client.post('/accounts/login', {'assertion': 'a'})
        self.assertEqual(response.content.decode(), 'OK')

I should explain this use of get_user_model from django.contrib.auth. Its job is to find the project’s user model, and it works whether you’re using the standard user model or a custom one (like we will be).

That test covers the desired response. Now test that the user actually gets logged in correctly. We can do that by inspecting the Django test client, to see if the session cookie has been set correctly.

Tip

Check out the Django docs on authentication at this point.

accounts/tests/test_views.py (ch16l012). 

from django.contrib.auth import get_user_model, SESSION_KEY
[...]

    @patch('accounts.views.authenticate')
    def test_gets_logged_in_session_if_authenticate_returns_a_user(
        self, mock_authenticate
    ):
        user = User.objects.create(email='[e-mail]')
        user.backend = ''  # required for auth_login to work
        mock_authenticate.return_value = user
        self.client.post('/accounts/login', {'assertion': 'a'})
        self.assertEqual(self.client.session[SESSION_KEY], user.pk)  #


    @patch('accounts.views.authenticate')
    def test_does_not_get_logged_in_if_authenticate_returns_None(
        self, mock_authenticate
    ):
        mock_authenticate.return_value = None
        self.client.post('/accounts/login', {'assertion': 'a'})
        self.assertNotIn(SESSION_KEY, self.client.session)  #

The Django test client keeps track of the session for its user. For the case where the user gets authenticated successfully, we check that their user ID (the primary key, or pk) is associated with their session.

In the case where the user should not be authenticated, the SESSION_KEY should not appear in their session.

Django Sessions: How a User’s Cookies Tell the Server She Is Authenticated

Being an attempt to explain sessions, cookies, and authentication in Django.

Because HTTP is stateless, servers need a way of recognising different clients with every single request. IP addresses can be shared, so the usual solution is to give each client a unique session ID, which it will store in a cookie, and submit with every request. The server will store that ID somewhere (by default, in the database), and then it can recognise each request that comes in as being from a particular client.

If you log in to the site using the dev server, you can actually take a look at your session ID by hand if you like. It’s stored under the key sessionid by default. See Figure 16-1.

Figure 16-1. Examining the session cookie in the Debug toolbar

These session cookies are set for all visitors to a Django site, whether they’re logged in or not.

When we want to recognise a client as being a logged-in and authenticated user, again, rather asking the client to send their username and password with every single request, the server can actually just mark that client’s session as being an authenticated session, and associate it with a user ID in its database.

A session is a dictionary-like data structure, and the user ID is stored under the key given by django.contrib.auth.SESSION_KEY. You can check this out in a manage.py console if you like:

$ python3 manage.py shell
[...]
In [1]: from django.contrib.sessions.models import Session

# substitute your session id from your browser cookie here
In [2]: session = Session.objects.get(
    session_key="8u0pygdy9blo696g3n4o078ygt6l8y0y"
)

In [3]: print(session.get_decoded())
{'_auth_user_id': '[e-mail]', '_auth_user_backend':
'accounts.authentication.PersonaAuthenticationBackend'}

You can also store any other information you like on a user’s session, as a way of temporarily keeping track of some state. This works for non-logged-in users too. Just use request.session inside any view, and it works as a dict. There’s more information in the Django docs on sessions.

That gives us two failures:

$ python3 manage.py test accounts
[...]
    self.assertEqual(self.client.session[SESSION_KEY], user.pk)
KeyError: '_auth_user_id'

[...]
AssertionError: '' != 'OK'
+ OK

The Django function that takes care of logging in a user, by marking their session, is available at django.contrib.auth.login. So we go through another couple of TDD cycles, until:

accounts/views.py. 

from django.contrib.auth import authenticate, login
from django.http import HttpResponse

def persona_login(request):
    user = authenticate(assertion=request.POST['assertion'])
    if user:
        login(request, user)
    return HttpResponse('OK')

…

OK

We now have a working login view.

Testing Login with Mocks

An alternative way of testing that the Django login function was called correctly would be to mock it out too:

accounts/tests/test_views.py. 

from django.http import HttpRequest
from accounts.views import persona_login
[...]

    @patch('accounts.views.login')
    @patch('accounts.views.authenticate')
    def test_calls_auth_login_if_authenticate_returns_a_user(
        self, mock_authenticate, mock_login
    ):
        request = HttpRequest()
        request.POST['assertion'] = 'asserted'
        mock_user = mock_authenticate.return_value
        login(request)
        mock_login.assert_called_once_with(request, mock_user)

The upside of this version of the test is that it doesn’t need to rely on the magic of the Django test client, and it doesn’t need to know anything about how Django sessions work—all you need to know is the name of the function you’re supposed to call.

Its downside is that it is very much testing implementation, rather than testing behaviour—it’s tightly coupled to the particular name of the Django login function and its API.

De-spiking Our Custom Authentication Backend: Mocking Out an Internet Request

Our custom authentication backend is next. Here’s how it looked in the spike:

class PersonaAuthenticationBackend(object):

    def authenticate(self, assertion):
        # Send the assertion to Mozilla's verifier service.
        data = {'assertion': assertion, 'audience': 'localhost'}
        print('sending to mozilla', data, file=sys.stderr)
        resp = requests.post('https://verifier.login.persona.org/verify', data=data)
        print('got', resp.content, file=sys.stderr)

        # Did the verifier respond?
        if resp.ok:
            # Parse the response
            verification_data = resp.json()

            # Check if the assertion was valid
            if verification_data['status'] == 'okay':
                email = verification_data['email']
                try:
                    return self.get_user(email)
                except ListUser.DoesNotExist:
                    return ListUser.objects.create(email=email)


    def get_user(self, email):
        return ListUser.objects.get(email=email)

Decoding this:

• We take an assertion and send it off to Mozilla using requests.post.
• We check its response code (resp.ok), and then check for a status=okay in the response JSON.
• We then extract an email address, and either find an existing user with that address, or create a new one.

1 if = 1 More Test

A rule of thumb for these sorts of tests: any if means an extra test, and any try/except means an extra test, so this should be about four tests. Let’s start with one:

accounts/tests/test_authentication.py. 

from unittest.mock import patch
from django.test import TestCase

from accounts.authentication import (
    PERSONA_VERIFY_URL, DOMAIN, PersonaAuthenticationBackend
)

class AuthenticateTest(TestCase):

    @patch('accounts.authentication.requests.post')
    def test_sends_assertion_to_mozilla_with_domain(self, mock_post):
        backend = PersonaAuthenticationBackend()
        backend.authenticate('an assertion')
        mock_post.assert_called_once_with(
            PERSONA_VERIFY_URL,
            data={'assertion': 'an assertion', 'audience': DOMAIN}
        )

In authenticate.py we’ll just have a few placeholders:

accounts/authentication.py. 

import requests

PERSONA_VERIFY_URL = 'https://verifier.login.persona.org/verify'
DOMAIN = 'localhost'


class PersonaAuthenticationBackend(object):

    def authenticate(self, assertion):
        pass

At this point we’ll need to:

(virtualenv)$ pip install requests

Note

Don’t forget to add requests to requirements.txt too, or the next deploy won’t work…

Then let’s see how the tests get on!

$ python3 manage.py test accounts
[...]
AssertionError: Expected 'post' to be called once. Called 0 times.

And we can get that to passing in three steps (make sure the Goat sees you doing each one individually!):

accounts/authentication.py. 

    def authenticate(self, assertion):
        requests.post(
            PERSONA_VERIFY_URL,
            data={'assertion': assertion, 'audience': DOMAIN}
        )

Grand:

$ python3 manage.py test accounts
[...]

Ran 5 tests in 0.023s

OK

Next let’s check that authenticate should return None if it sees an error from the request:

accounts/tests/test_authentication.py (ch16l020). 

    @patch('accounts.authentication.requests.post')
    def test_returns_none_if_response_errors(self, mock_post):
        mock_post.return_value.ok = False
        backend = PersonaAuthenticationBackend()

        user = backend.authenticate('an assertion')
        self.assertIsNone(user)

And that passes straight away—we currently return None in all cases!

Patching at the Class Level

Next we want to check that the response JSON has status=okay. Adding this test would involve a bit of duplication—let’s apply the “three strikes” rule:

accounts/tests/test_authentication.py (ch16l021). 

@patch('accounts.authentication.requests.post')  #
class AuthenticateTest(TestCase):

    def setUp(self):
        self.backend = PersonaAuthenticationBackend()  #


    def test_sends_assertion_to_mozilla_with_domain(self, mock_post):
        self.backend.authenticate('an assertion')
        mock_post.assert_called_once_with(
            PERSONA_VERIFY_URL,
            data={'assertion': 'an assertion', 'audience': DOMAIN}
        )


    def test_returns_none_if_response_errors(self, mock_post):
        mock_post.return_value.ok = False  #
        user = self.backend.authenticate('an assertion')
        self.assertIsNone(user)


    def test_returns_none_if_status_not_okay(self, mock_post):
        mock_post.return_value.json.return_value = {'status': 'not okay!'}  #
        user = self.backend.authenticate('an assertion')
        self.assertIsNone(user)

You can apply a patch at the class level as well, and that has the effect that every test method in the class will have the patch applied, and the mock injected.

We can now use the setUp function to prepare any useful variables which we’re going to use in all of our tests.

Now each test is only adjusting the setup variables it needs, rather than setting up a load of duplicated boilerplate—it’s more readable.

And that’s all very well, but everything still passes!

OK

Time to test for the positive case where authenticate should return a user object. We expect this to fail:

accounts/tests/test_authentication.py (ch16l022-1). 

from django.contrib.auth import get_user_model
User = get_user_model()
[...]

    def test_finds_existing_user_with_email(self, mock_post):
        mock_post.return_value.json.return_value = {'status': 'okay', 'email': '[e-mail]'}
        actual_user = User.objects.create(email='[e-mail]')
        found_user = self.backend.authenticate('an assertion')
        self.assertEqual(found_user, actual_user)

Indeed, a fail:

AssertionError: None != <User: >

Let’s code. We’ll start with a “cheating” implementation, where we just get the first user we find in the database:

accounts/authentication.py (ch16l023). 

import requests
from django.contrib.auth import get_user_model
User = get_user_model()
[...]

    def authenticate(self, assertion):
        requests.post(
            PERSONA_VERIFY_URL,
            data={'assertion': assertion, 'audience': DOMAIN}
        )
        return User.objects.first()

That gets our new test passing, but still, none of the other tests are failing:

$ python3 manage.py test accounts
[...]

Ran 8 tests in 0.030s

OK

They’re passing because objects.first() returns None if there are no users in the database. Let’s make our other cases more realistic, by making sure there’s always at least one user in the database for all our tests:

accounts/tests/test_authentication.py (ch16l022-2). 

    def setUp(self):
        self.backend = PersonaAuthenticationBackend()
        user = User(email='[e-mail]')
        user.username = 'otheruser'  #
        user.save()

By default, Django’s users have a username attribute, which has to be unique, so this value is just a placeholder to allow us to create multiple users. Later on, we’ll get rid of usernames in favour of using emails as the primary key.

That gives us three failures:

FAIL: test_finds_existing_user_with_email
AssertionError: <User: otheruser> != <User: >
[...]
FAIL: test_returns_none_if_response_errors
AssertionError: <User: otheruser> is not None
[...]
FAIL: test_returns_none_if_status_not_okay
AssertionError: <User: otheruser> is not None

Let’s start building our guards for cases where authentication should fail—if the response errors, or if the status is not okay. Suppose we start with this:

accounts/authentication.py (ch16l024-1). 

def authenticate(self, assertion):
    response = requests.post(
        PERSONA_VERIFY_URL,
        data={'assertion': assertion, 'audience': DOMAIN}
    )
    if response.json()['status'] == 'okay':
        return User.objects.first()

That actually fixes two of the tests, slightly surprisingly:

AssertionError: <User: otheruser> != <User: >

FAILED (failures=1)

Let’s get the final test passing by retrieving the right user, and then we’ll have a look at that surprise pass:

accounts/authentication.py (ch16l024-2). 

    if response.json()['status'] == 'okay':
        return User.objects.get(email=response.json()['email'])

…

OK

Beware of Mocks in Boolean Comparisons

So how come our test_returns_none_if_response_errors isn’t failing?

Because we’ve mocked out requests.post, the response is a Mock object, which as you remember, returns all attributes and properties as more Mocks.^[29] So, when we do:

accounts/authentication.py. 

    if response.json()['status'] == 'okay':

response is actually a mock, response.json() is a mock, and response.json()['status'] is a mock too! We end up comparing a mock with the string “okay”, which evaluates to False, and so we return None by default. Let’s make our test more explicit, by saying that the response JSON will be an empty dict:

accounts/tests/test_authentication.py (ch16l025). 

    def test_returns_none_if_response_errors(self, mock_post):
        mock_post.return_value.ok = False
        mock_post.return_value.json.return_value = {}
        user = self.backend.authenticate('an assertion')
        self.assertIsNone(user)

That gives:

    if response.json()['status'] == 'okay':
KeyError: 'status'

And we can fix it like this:

accounts/authentication.py (ch16l026). 

    if response.ok and response.json()['status'] == 'okay':
        return User.objects.get(email=response.json()['email'])

…

OK

Great! Our authenticate function is now working the way we want it to.

Creating a User if Necessary

Next we should check that, if our authenticate function has a valid assertion from Persona, but we don’t have a user record for that person in our database, we should create one. Here’s the test for that:

accounts/tests/test_authentication.py (ch16l027). 

def test_creates_new_user_if_necessary_for_valid_assertion(self, mock_post):
    mock_post.return_value.json.return_value = {'status': 'okay', 'email': '[e-mail]'}
    found_user = self.backend.authenticate('an assertion')
    new_user = User.objects.get(email='[e-mail]')
    self.assertEqual(found_user, new_user)

That fails in our application code when we try find an existing user with that email:

    return User.objects.get(email=response.json()['email'])
django.contrib.auth.models.DoesNotExist: User matching query does not exist.

So we add a try/except, returning an “empty” user at first:

accounts/authentication.py (ch16l028). 

    if response.ok and response.json()['status'] == 'okay':
        try:
            return User.objects.get(email=response.json()['email'])
        except User.DoesNotExist:
            return User.objects.create()

And that fails, but this time it fails when the test tries to find the new user by email:

    new_user = User.objects.get(email='[e-mail]')
django.contrib.auth.models.DoesNotExist: User matching query does not exist.

And so we fix it by assigning the correct email addresss:

accounts/authentication.py (ch16l029). 

    if response.ok and response.json()['status'] == 'okay':
        email = response.json()['email']
        try:
            return User.objects.get(email=email)
        except User.DoesNotExist:
            return User.objects.create(email=email)

That gets us to passing tests:

$ python3 manage.py test accounts
[...]
Ran 9 tests in 0.019s
OK

The get_user Method

The next thing we have to build is a get_user method for our authentication backend. This method’s job is to retrieve a user based on their email address, or to return None if it can’t find one. (That last wasn’t well documented at the time of writing, but that is the interface we have to comply with. See the source for details.)

Here’s a couple of tests for those two requirements:

accounts/tests/test_authentication.py (ch16l030). 

class GetUserTest(TestCase):

    def test_gets_user_by_email(self):
        backend = PersonaAuthenticationBackend()
        other_user = User(email='[e-mail]')
        other_user.username = 'otheruser'
        other_user.save()
        desired_user = User.objects.create(email='[e-mail]')
        found_user = backend.get_user('[e-mail]')
        self.assertEqual(found_user, desired_user)


    def test_returns_none_if_no_user_with_that_email(self):
        backend = PersonaAuthenticationBackend()
        self.assertIsNone(
            backend.get_user('[e-mail]')
        )

Here’s our first failure:

AttributeError: 'PersonaAuthenticationBackend' object has no attribute
'get_user'

Let’s create a placeholder one then:

accounts/authentication.py (ch16l031). 

class PersonaAuthenticationBackend(object):

    def authenticate(self, assertion):
        [...]

    def get_user(self):
        pass

Now we get:

TypeError: get_user() takes 1 positional argument but 2 were given

So:

accounts/authentication.py (ch16l032). 

    def get_user(self, email):
        pass

Next:

    self.assertEqual(found_user, desired_user)
AssertionError: None != <User: >

And (step by step, just to see if our test fails the way we think it will):

accounts/authentication.py (ch16l033). 

    def get_user(self, email):
        return User.objects.first()

That gets us past the first assertion, and onto

    self.assertEqual(found_user, desired_user)
AssertionError: <User: otheruser> != <User: >

And so we call get with the email as an argument:

accounts/authentication.py (ch16l034). 

    def get_user(self, email):
        return User.objects.get(email=email)

That gets us to passing tests:

Now our test for the None case fails:

ERROR: test_returns_none_if_no_user_with_that_email
[...]
django.contrib.auth.models.DoesNotExist: User matching query does not exist.

Which prompts us to finish the method like this:

accounts/authentication.py (ch16l035). 

    def get_user(self, email):
        try:
            return User.objects.get(email=email)
        except User.DoesNotExist:
            return None #

You could just use pass here, and the function would return None by default. However, because we specifically need the function to return None, explicit is better than implicit here.

That gets us to passing tests:

OK

And we have a working authentication backend!

$ python3 manage.py test accounts
[...]
Ran 11 tests in 0.020s
OK

Now we can define our custom user model.

A Minimal Custom User Model

Django’s built-in user model makes all sorts of assumptions about what information you want to track about users, from explicitly recording first name and last name, to forcing you to use a username. I’m a great believer in not storing information about users unless you absolutely must, so a user model that records an email address and nothing else sounds good to me!

accounts/tests/test_models.py. 

from django.test import TestCase
from django.contrib.auth import get_user_model

User = get_user_model()

class UserModelTest(TestCase):

    def test_user_is_valid_with_email_only(self):
        user = User(email='[e-mail]')
        user.full_clean()  # should not raise

That gives us an expected failure:

django.core.exceptions.ValidationError: {'username': ['This field cannot be
blank.'], 'password': ['This field cannot be blank.']}

Password? Username? Bah! How about this?

accounts/models.py. 

from django.db import models

class User(models.Model):
    email = models.EmailField()

And we wire it up inside settings.py using a variable called AUTH_USER_MODEL. While we’re at it, we’ll add our new authentication backend too:

superlists/settings.py (ch16l039). 

AUTH_USER_MODEL = 'accounts.User'
AUTHENTICATION_BACKENDS = (
    'accounts.authentication.PersonaAuthenticationBackend',
)

Now Django tells us off because it wants a couple of bits of metadata on any custom user model:

AttributeError: type object 'User' has no attribute 'REQUIRED_FIELDS'

Sigh. Come on, Django, it’s only got one field, you should be able to figure out the answers to these questions for yourself. Here you go:

accounts/models.py. 

class User(models.Model):
    email = models.EmailField()
    REQUIRED_FIELDS = ()

Next silly question?^[30]

AttributeError: type object 'User' has no attribute 'USERNAME_FIELD'

So:

accounts/models.py. 

class User(models.Model):
    email = models.EmailField()
    REQUIRED_FIELDS = ()
    USERNAME_FIELD = 'email'

The next error is a database error:

django.db.utils.OperationalError: no such table: accounts_user

That prompts us, as usual, to do a migration:

$ python3 manage.py makemigrations
System check identified some issues:

WARNINGS:
accounts.User: (auth.W004) 'User.email' is named as the 'USERNAME_FIELD', but
it is not unique.
        HINT: Ensure that your authentication backend(s) can handle non-unique
usernames.
Migrations for 'accounts':
  0001_initial.py:
    - Create model User

Let’s hold that thought, and see if we can get the tests passing again.

A Slight Disappointment

Meanwhile, we have a couple of weird unexpected failures:

$ python3 manage.py test accounts
[...]
ERROR: test_gets_logged_in_session_if_authenticate_returns_a_user
[...]
ERROR: test_returns_OK_when_user_found
[...]
    user.save(update_fields=['last_login'])
[...]
ValueError: The following fields do not exist in this model or are m2m fields:
last_login

It looks like Django is going to insist on us having a last_login field on our user model too. Oh well. My pristine, single-field user model is despoiled. I still love it though.

accounts/models.py. 

from django.db import models
from django.utils import timezone

class User(models.Model):
    email = models.EmailField()
    last_login = models.DateTimeField(default=timezone.now)
    REQUIRED_FIELDS = ()
    USERNAME_FIELD = 'email'

We get another database error, so let’s clear down the migration and re-create it:

$ rm accounts/migrations/0001_initial.py
$ python3 manage.py makemigrations
System check identified some issues:
[...]
Migrations for 'accounts':
  0001_initial.py:
    - Create model User

That gets the tests passing, although they are still giving us some warnings:

$ python3 manage.py test accounts
[...]
System check identified some issues:

WARNINGS:
accounts.User: (auth.W004) 'User.email' is named as the 'USERNAME_FIELD', but
it is not unique.
[...]

Ran 12 tests in 0.041s

OK

Tests as Documentation

Let’s go all the way and make the email field into the primary key^[31], and thus implicitly remove the auto-generated id column.

Although that warning is probably enough of a justification to go ahead and make the change, it would be better to have a specific test:

accounts/tests/test_models.py (ch16l043). 

    def test_email_is_primary_key(self):
        user = User()
        self.assertFalse(hasattr(user, 'id'))

It’ll help us remember if we ever come back and look at the code again in future.

    self.assertFalse(hasattr(user, 'id'))
AssertionError: True is not false

Note

Your tests can be are a form of documentation for your code—they express what your requirements are of a particular class or function. Sometimes, if you forget why you’ve done something a particular way, going back and looking at the tests will give you the answer. That’s why it’s important to give your tests explicit, verbose method names.

And here’s the implementation (feel free to check what happens with unique=True first):

accounts/models.py (ch16l044). 

    email = models.EmailField(primary_key=True)

That works:

$ python3 manage.py test accounts
[...]
Ran 13 tests in 0.021s
OK

One final cleanup of migrations to make sure we’ve got everything there:

$ rm accounts/migrations/0001_initial.py
$ python3 manage.py makemigrations
Migrations for 'accounts':
  0001_initial.py:
    - Create model User

No warnings now!

Users Are Authenticated

Our user model needs one last property before it’s complete: standard Django users have an API which includes several methods, most of which we won’t need, but there is one that will come in useful: .is_authenticated():

accounts/tests/test_models.py (ch16l045). 

    def test_is_authenticated(self):
        user = User()
        self.assertTrue(user.is_authenticated())

Which gives:

AttributeError: 'User' object has no attribute 'is_authenticated'

And so, the ultra-simple:

accounts/models.py. 

class User(models.Model):
    email = models.EmailField(primary_key=True)
    last_login = models.DateTimeField(default=timezone.now)
    REQUIRED_FIELDS = ()
    USERNAME_FIELD = 'email'

    def is_authenticated(self):
        return True

And that works:

$ python3 manage.py test accounts
[...]
Ran 14 tests in 0.021s
OK

The Moment of Truth: Will the FT Pass?

I think we’re just about ready to try our functional test! Let’s just wire up our base template. Firstly, it needs to show a different message for logged-in and non-logged-in users:

lists/templates/base.html. 

<nav class="navbar navbar-default" role="navigation">
    <a class="navbar-brand" href="/">Superlists</a>
    {% if user.email %}
        <a class="btn navbar-btn navbar-right" id="id_logout" href="#">Log out</a>
        <span class="navbar-text navbar-right">Logged in as {{ user.email }}</span>
    {% else %}
        <a class="btn navbar-btn navbar-right" id="id_login" href="#">Sign in</a>
    {% endif %}
</nav>

Lovely. Then we wire up our various context variables for the call to initialize:

lists/templates/base.html. 

<script>
    /*global $, Superlists, navigator */
    $(document).ready(function () {
        var user = "{{ user.email }}" || null;
        var token = "{{ csrf_token }}";
        var urls = {
            login: "{% url 'persona_login' %}",
            logout: "TODO",
        };
        Superlists.Accounts.initialize(navigator, user, token, urls);
    });
</script>

So how does our FT get along?

$ python3 manage.py test functional_tests.test_login
Creating test database for alias 'default'...
[...]
Ran 1 test in 26.382s

OK

Woohoo!

I’ve been waiting to do a commit up until this moment, just to make sure everything works. At this point, you could make a series of separate commits—one for the login view, one for the auth backend, one for the user model, one for wiring up the template. Or you could decide that, since they’re all interrelated, and none will work without the others, you may as well just have one big commit:

$ git status
$ git add .
$ git diff --staged
$ git commit -m "Custom Persona auth backend + custom user model"

Finishing Off Our FT, Testing Logout

We’ll extend our FT to check that the logged-in status persists, ie it’s not just something we set in JavaScript on the client side, but the server knows about it too and will maintain the logged-in state if she refreshes the page. We’ll also test that she can log out.

I started off writing code a bit like this:

functional_tests/test_login.py. 

    # Refreshing the page, she sees it's a real session login,
    # not just a one-off for that page
    self.browser.refresh()
    self.wait_for_element_with_id('id_logout')
    navbar = self.browser.find_element_by_css_selector('.navbar')
    self.assertIn('[e-mail]', navbar.text)

And, after four repetitions of very similar code, a couple of helper functions suggested themselves:

functional_tests/test_login.py (ch16l050). 

    def wait_to_be_logged_in(self):
        self.wait_for_element_with_id('id_logout')
        navbar = self.browser.find_element_by_css_selector('.navbar')
        self.assertIn('[e-mail]', navbar.text)

    def wait_to_be_logged_out(self):
        self.wait_for_element_with_id('id_login')
        navbar = self.browser.find_element_by_css_selector('.navbar')
        self.assertNotIn('[e-mail]', navbar.text)

And I extended the FT like this:

functional_tests/test_login.py (ch16l049). 

        [...]
        # The Persona window closes
        self.switch_to_new_window('To-Do')

        # She can see that she is logged in
        self.wait_to_be_logged_in()

        # Refreshing the page, she sees it's a real session login,
        # not just a one-off for that page
        self.browser.refresh()
        self.wait_to_be_logged_in()

        # Terrified of this new feature, she reflexively clicks "logout"
        self.browser.find_element_by_id('id_logout').click()
        self.wait_to_be_logged_out()

        # The "logged out" status also persists after a refresh
        self.browser.refresh()
        self.wait_to_be_logged_out()

I also found that improving the failure message in the wait_for_element_with_id function helped to see what was going on:

functional_tests/test_login.py. 

    def wait_for_element_with_id(self, element_id):
        WebDriverWait(self.browser, timeout=30).until(
            lambda b: b.find_element_by_id(element_id),
            'Could not find element with id {}. Page text was {}'.format(
                element_id, self.browser.find_element_by_tag_name('body').text
            )
        )

With that, we can see that the test is failing because the logout button doesn’t work:

$ python3 manage.py test functional_tests.test_login
  File "/workspace/superlists/functional_tests/test_login.py", line 36, in
wait_to_be_logged_out
[...]
selenium.common.exceptions.TimeoutException: Message: 'Could not find element
with id id_login. Page text was Superlists\nLog out\nLogged in as
[e-mail]\nStart a new To-Do list'

Implementing a logout button is actually very simple: we can use Django’s built-in logout view, which clears down the user’s session and redirects them to a page of our choice:

accounts/urls.py. 

urlpatterns = patterns('',
    url(r'^login$', 'accounts.views.persona_login', name='persona_login'),
    url(r'^logout$', 'django.contrib.auth.views.logout', {'next_page': '/'}, name='logout'),
)

And in base.html, we just make the logout into a normal URL link:

lists/templates/base.html. 

<a class="btn navbar-btn navbar-right" id="id_logout" href="{% url 'logout' %}">Log out</a>

And that gets us a fully passing FT—indeed, a fully passing test suite:

$ python3 manage.py test functional_tests.test_login
[...]
OK
$ python3 manage.py test
[...]
Ran 54 tests in 78.124s

OK

In the next chapter, we’ll start trying to put our login system to good use. In the meantime, do a commit, and enjoy this recap:

On Mocking in Python

The Mock library

Michael Foord (who used to work for the company that spawned PythonAnywhere, just before I joined) wrote the excellent “Mock” library that’s now been integrated into the standard library of Python 3. It contains most everything you might need for mocking in Python.

The patch decorator

unittest.mock provides a function called patch, which can be used to “mock out” any object from the module you’re testing. It’s commonly used as a decorator on a test method, or even at the class level, where it’s applied to all the test methods of that class.

Mocks are truthy and can mask errors

Be aware that mocking things out can cause counterintuitive behaviour in if statements. Mocks are truthy, and they can also mask errors, because they have all attributes and methods.

Too many mocks are a code smell

Overly mocky tests end up very tightly coupled to their implementation. Sometimes this is unavoidable. But, in general, try to find ways of organising your code so that you don’t need too many mocks.

---

^[27] If you’re using Python 3.2, upgrade! Or if you’re stuck with it, pip3 install mock, and use from mock instead of from unittest.mock.

^[28] Even though we’re going to define our own authenticate function, we still import from django.contrib.auth. Django will dynamically replace it with our function once we’ve configured it in settings.py. This has the benefit that, if we later switch to a third-party library for our authenticate function, our views.py doesn’t need to change.

^[29] Actually, this is only happening because we’re using the patch decorator, which returns a MagicMock, an even mockier version of mock that can also behave like a dictionary. More info in the docs.

^[30] You might ask, if I think Django is so silly, why don’t I submit a pull request to fix it? Should be quite a simple fix. Well, I promise I will, as soon as I’ve finished writing the book. For now, snarky comments will have to suffice.

^[31] Emails may not be the perfect primary key IRL. One reader, clearly deeply emotionally scarred, wrote me a tearful email about how much they’ve suffered for over a decade from trying to deal with the effects email primary keys, due to their making multi-user account management impossible. So, as ever, YMMV.