Chapter 4. What Are We Doing with All These Tests? (And, Refactoring)
Now that weâve seen the basics of TDD in action, itâs time to pause and talk about why weâre doing it.
Iâm imagining several of you, dear readers, have been holding back some seething frustrationâperhaps some of you have done a bit of unit testing before, and perhaps some of you are just in a hurry. Youâve been biting back questions like:
-
Arenât all these tests a bit excessive?
-
Surely some of them are redundant? Thereâs duplication between the functional tests and the unit tests.
-
I mean, what are you doing importing
django.urls.resolve
in your unit tests? Isnât that testing Djangoâthat is, testing third-party code? I thought that was a no-no? -
Those unit tests seemed way too trivialâtesting one line of declaration, and a one-line function that returns a constant! Isnât that just a waste of time? Shouldnât we save our tests for more complex things?
-
What about all those tiny changes during the unit-test/code cycle? Surely we could have just skipped to the end? I mean,
home_page = None
!? Really? -
Youâre not telling me you actually code like this in real life?
Ah, young grasshopper. I too was once full of questions like these. But only because theyâre perfectly good questions. In fact, I still ask myself questions like these, all the time. Does all this stuff really have value? Is this a bit of a cargo cult?
Programming Is Like Pulling a Bucket of Water Up from a Well
Ultimately, programming is hard. Often, we are smart, so we succeed. TDD is there to help us out when weâre not so smart. Kent Beck (who basically invented TDD) uses the metaphor of lifting a bucket of water out of a well with a rope: when the well isnât too deep, and the bucket isnât very full, itâs easy. And even lifting a full bucket is pretty easy at first. But after a while, youâre going to get tired. TDD is like having a ratchet that lets you save your progress, take a break, and make sure you never slip backwards. That way you donât have to be smart all the time.
OK, perhaps in general, youâre prepared to concede that TDD is a good idea, but maybe you still think Iâm overdoing it? Testing the tiniest thing, and taking ridiculously many small steps?
TDD is a discipline, and that means itâs not something that comes naturally; because many of the payoffs arenât immediate but only come in the longer term, you have to force yourself to do it in the moment. Thatâs what the image of the Testing Goat is supposed to illustrateâyou need to be a bit bloody-minded about it.
Using Selenium to Test User Interactions
Where were we at the end of the last chapter? Letâs rerun the test and find out:
$ python functional_tests.py F ====================================================================== FAIL: test_can_start_a_list_and_retrieve_it_later (__main__.NewVisitorTest) --------------------------------------------------------------------- Traceback (most recent call last): File "functional_tests.py", line 19, in test_can_start_a_list_and_retrieve_it_later self.fail('Finish the test!') AssertionError: Finish the test! --------------------------------------------------------------------- Ran 1 test in 1.609s FAILED (failures=1)
Did you try it, and get an error saying Problem loading page or
Unable to connect? So did I. Itâs because we forgot to spin up the dev
server first using manage.py runserver
. Do that, and youâll get the failure
message weâre after.
Note
One of the great things about TDD is that you never have to worry about forgetting what to do nextâjust rerun your tests and they will tell you what you need to work on.
âFinish the testâ, it says, so letâs do just that! Open up functional_tests.py and weâll extend our FT:
functional_tests.py
from
selenium
import
webdriver
from
selenium.webdriver.common.keys
import
Keys
import
time
import
unittest
class
NewVisitorTest
(
unittest
.
TestCase
)
:
def
setUp
(
self
)
:
self
.
browser
=
webdriver
.
Firefox
(
)
def
tearDown
(
self
)
:
self
.
browser
.
quit
(
)
def
test_can_start_a_list_and_retrieve_it_later
(
self
)
:
# Edith has heard about a cool new online to-do app. She goes
# to check out its homepage
self
.
browser
.
get
(
'
http://localhost:8000
'
)
# She notices the page title and header mention to-do lists
self
.
assertIn
(
'
To-Do
'
,
self
.
browser
.
title
)
header_text
=
self
.
browser
.
find_element_by_tag_name
(
'
h1
'
)
.
text
self
.
assertIn
(
'
To-Do
'
,
header_text
)
# She is invited to enter a to-do item straight away
inputbox
=
self
.
browser
.
find_element_by_id
(
'
id_new_item
'
)
self
.
assertEqual
(
inputbox
.
get_attribute
(
'
placeholder
'
)
,
'
Enter a to-do item
'
)
# She types "Buy peacock feathers" into a text box (Edith's hobby
# is tying fly-fishing lures)
inputbox
.
send_keys
(
'
Buy peacock feathers
'
)
# When she hits enter, the page updates, and now the page lists
# "1: Buy peacock feathers" as an item in a to-do list table
inputbox
.
send_keys
(
Keys
.
ENTER
)
time
.
sleep
(
1
)
table
=
self
.
browser
.
find_element_by_id
(
'
id_list_table
'
)
rows
=
table
.
find_elements_by_tag_name
(
'
tr
'
)
self
.
assertTrue
(
any
(
row
.
text
==
'
1: Buy peacock feathers
'
for
row
in
rows
)
)
# There is still a text box inviting her to add another item. She
# enters "Use peacock feathers to make a fly" (Edith is very
# methodical)
self
.
fail
(
'
Finish the test!
'
)
# The page updates again, and now shows both items on her list
[
.
.
.
]
Weâre using several of the methods that Selenium provides to examine web pages:
find_element_by_tag_name
,find_element_by_id
, andfindâ _eleâ mentâ sâ _byâ _âtag_name
(notice the extras
, which means it will return several elements rather than just one).We also use
send_keys
, which is Seleniumâs way of typing into input elements.The
Keys
class (donât forget to import it) lets us send special keys like Enter.1When we hit Enter, the page will refresh. The
time.sleep
is there to make sure the browser has finished loading before we make any assertions about the new page. This is called an âexplicit waitâ (a very simple one; weâll improve it in Chapter 6).
Tip
Watch out for the difference between the Selenium find_element_...
and find_elements_...
functions. One returns an element and raises
an exception if it canât find it, whereas the other returns a list, which
may be empty.
Also, just look at that any
function. Itâs a little-known Python built-in.
I donât even need to explain it, do I? Python is such a joy.
Although, if youâre one of my readers who doesnât know Python, whatâs happening
inside the any
is a generator expression, which is like a list
comprehension but awesomer. You need to read up on this. If you Google it,
youâll find Guido himself explaining it nicely.
Come back and tell me thatâs not pure joy!
Letâs see how it gets on:
$ python functional_tests.py [...] selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: h1
Decoding that, the test is saying it canât find an <h1>
element on the page.
Letâs see what we can do to add that to the HTML of our home page.
Big changes to a functional test are usually a good thing to commit on their own. I failed to do so in my first draft, and I regretted it later when I changed my mind and had the change mixed up with a bunch of others. The more atomic your commits, the better:
$ git diff # should show changes to functional_tests.py $ git commit -am "Functional test now checks we can input a to-do item"
The âDonât Test Constantsâ Rule, and Templates to the Rescue
Letâs take a look at our unit tests, lists/tests.py. Currently weâre looking for specific HTML strings, but thatâs not a particularly efficient way of testing HTML. In general, one of the rules of unit testing is Donât test constants, and testing HTML as text is a lot like testing a constant.
In other words, if you have some code that says:
wibble
=
3
Thereâs not much point in a test that says:
from
myprogram
import
wibble
assert
wibble
==
3
Unit tests are really about testing logic, flow control, and configuration. Making assertions about exactly what sequence of characters we have in our HTML strings isnât doing that.
Whatâs more, mangling raw strings in Python really isnât a great way of dealing with HTML. Thereâs a much better solution, which is to use templates. Quite apart from anything else, if we can keep HTML to one side in a file whose name ends in .html, weâll get better syntax highlighting! There are lots of Python templating frameworks out there, and Django has its own which works very well. Letâs use that.
Refactoring to Use a Template
What we want to do now is make our view function return exactly the same HTML, but just using a different process. Thatâs a refactorâwhen we try to improve the code without changing its functionality.
That last bit is really important. If you try to add new functionality at the same time as refactoring, youâre much more likely to run into trouble. Refactoring is actually a whole discipline in itself, and it even has a reference book: Martin Fowlerâs Refactoring.
The first rule is that you canât refactor without tests. Thankfully, weâre doing TDD, so weâre way ahead of the game. Letâs check that our tests pass; they will be what makes sure that our refactoring is behaviour preserving:
$ python manage.py test [...] OK
Great! Weâll start by taking our HTML string and putting it into its own file. Create a directory called lists/templates to keep templates in, and then open a file at lists/templates/home.html, to which weâll transfer our HTML:2
lists/templates/home.html
<html>
<title>
To-Do lists</title>
</html>
Mmmh, syntax-highlightedâ¦much nicer! Now to change our view function:
lists/views.py
from
django.shortcuts
import
render
def
home_page
(
request
):
return
render
(
request
,
'home.html'
)
Instead of building our own HttpResponse
, we now use the Django render
function. It takes the request as its first parameter (for reasons weâll go
into later) and the name of the template to render. Django will automatically
search folders called templates inside any of your appsâ directories. Then
it builds an HttpResponse
for you, based on the content of the template.
Note
Templates are a very powerful feature of Djangoâs, and their main
strength consists of substituting Python variables into HTML text. Weâre
not using this feature yet, but we will in future chapters. Thatâs
why we use render
and (later) render_toâ_string
rather
than, say, manually reading the file from disk with the built-in open
.
Letâs see if it works:
$ python manage.py test [...] ====================================================================== ERROR: test_home_page_returns_correct_html (lists.tests.HomePageTest) --------------------------------------------------------------------- Traceback (most recent call last): File "...python-tdd-book/lists/tests.py", line 17, in test_home_page_returns_correct_html response = home_page(request) File "...python-tdd-book/lists/views.py", line 5, in home_page return render(request, 'home.html') File "/usr/local/lib/python3.6/dist-packages/django/shortcuts.py", line 48, in render return HttpResponse(loader.render_to_string(*args, **kwargs), File "/usr/local/lib/python3.6/dist-packages/django/template/loader.py", line 170, in render_to_string t = get_template(template_name, dirs) File "/usr/local/lib/python3.6/dist-packages/django/template/loader.py", line 144, in get_template template, origin = find_template(template_name, dirs) File "/usr/local/lib/python3.6/dist-packages/django/template/loader.py", line 136, in find_template raise TemplateDoesNotExist(name) django.template.base.TemplateDoesNotExist: home.html --------------------------------------------------------------------- Ran 2 tests in 0.004s
Another chance to analyse a traceback:
We start with the error: it canât find the template.
Then we double-check what test is failing: sure enough, itâs our test of the view HTML.
Then we find the line in our tests that caused the failure: itâs when we call the
home_page
function.Finally, we look for the part of our own application code that caused the failure: itâs when we try to call
render
.
So why canât Django find the template? Itâs right where itâs supposed to be, in the lists/templates folder.
The thing is that we havenât yet officially registered our lists app with
Django. Unfortunately, just running the startapp
command and
having what is obviously an app in your project folder isnât quite enough. You
have to tell Django that you really mean it, and add it to settings.py as
well. Belt and braces. Open it up and look for a variable called
INSTALLED_APPS
, to which weâll add lists
:
superlists/settings.py
# Application definition
INSTALLED_APPS
=
[
'django.contrib.admin'
,
'django.contrib.auth'
,
'django.contrib.contenttypes'
,
'django.contrib.sessions'
,
'django.contrib.messages'
,
'django.contrib.staticfiles'
,
'lists'
,
]
You can see thereâs lots of apps already in there by default. We just need to
add ours, lists
, to the bottom of the list. Donât forget the trailing
commaâit may not be required, but one day youâll be really annoyed when you
forget it and Python concatenates two strings on different linesâ¦
Now we can try running the tests again:
$ python manage.py test [...] self.assertTrue(html.endswith('</html>')) AssertionError: False is not true
Darn, not quite.
Note
Depending on whether your text editor insists on adding newlines to the end of files, you may not even see this error. If so, you can safely ignore the next bit, and skip straight to where you can see the listing says OK.
But it did get further! It seems it managed to find our template, but
the last of the three assertions is failing. Apparently thereâs something wrong
at the end of the output. I had to do a little print(repr(html))
to debug this, but it turns out that the switch to templates has introduced an
additional newline (\n
) at the end. We can get them to pass like this:
lists/tests.py
self
.
assertTrue
(
html
.
strip
()
.
endswith
(
'</html>'
))
Itâs a tiny bit of a cheat, but whitespace at the end of an HTML file really shouldnât matter to us. Letâs try running the tests again:
$ python manage.py test [...] OK
Our refactor of the code is now complete, and the tests mean weâre happy that behaviour is preserved. Now we can change the tests so that theyâre no longer testing constants; instead, they should just check that weâre rendering the right template.
The Django Test Client
One
way we could test this is to manually render the template ourselves in the
test, and then compare that to what the view returns. Django has a function
called
render_to_string
which will let us do that:
lists/tests.py
from
django.template.loader
import
render_to_string
[
...
]
def
test_home_page_returns_correct_html
(
self
):
request
=
HttpRequest
()
response
=
home_page
(
request
)
html
=
response
.
content
.
decode
(
'utf8'
)
expected_html
=
render_to_string
(
'home.html'
)
self
.
assertEqual
(
html
,
expected_html
)
But thatâs a bit of an unwieldy way of testing that we use the right template.
And all this faffing about with .decode()
and .strip()
is distracting.
Instead, Django gives us a tool called the
Django
Test Client, which has built-in ways of checking what templates are used.
Hereâs how it looks:
lists/tests.py
def
test_home_page_returns_correct_html
(
self
)
:
response
=
self
.
client
.
get
(
'
/
'
)
html
=
response
.
content
.
decode
(
'
utf8
'
)
self
.
assertTrue
(
html
.
startswith
(
'
<html>
'
)
)
self
.
assertIn
(
'
<title>To-Do lists</title>
'
,
html
)
self
.
assertTrue
(
html
.
strip
(
)
.
endswith
(
'
</html>
'
)
)
self
.
assertTemplateUsed
(
response
,
'
home.html
'
)
Instead of manually creating an
HttpRequest
object and calling the view function directly, we callself.client.get
, passing it the URL we want to test.Weâll leave the old tests there for now, just to make sure everything is working the way we think it is.
.assertTemplateUsed
is the test method that the DjangoTestCase
class provides us. It lets us check what template was used to render a response (NBâit will only work for responses that were retrieved by the test client).
And that test will still pass:
Ran 2 tests in 0.016s OK
Just because Iâm always suspicious of a test I havenât seen fail, letâs deliberately break it:
lists/tests.py
self
.
assertTemplateUsed
(
response
,
'wrong.html'
)
That way weâll also learn what its error messages look like:
AssertionError: False is not true : Template 'wrong.html' was not a template used to render the response. Actual template(s) used: home.html
Thatâs very helpful! Letâs change the assert back to the right thing. While
weâre at it, we can delete our old assertions. And we can also delete the
old test_root_âurl_resolves
test, because thatâs tested implicitly by the
Django Test Client. Weâve combined two long-winded tests into one!
lists/tests.py (ch04l010)
from
django.test
import
TestCase
class
HomePageTest
(
TestCase
):
def
test_uses_home_template
(
self
):
response
=
self
.
client
.
get
(
'/'
)
self
.
assertTemplateUsed
(
response
,
'home.html'
)
The main point, though, is that instead of testing constants weâre testing our implementation. Great!3
On Refactoring
That was an absolutely trivial example of refactoring. But, as Kent Beck puts it in Test-Driven Development: By Example, âAm I recommending that you actually work this way? No. Iâm recommending that you be able to work this wayâ.
In fact, as I was writing this my first instinct was to dive in and change the
test firstâmake it use the assertTemplateUsed
function straight away;
delete the three superfluous assertions, leaving just a check of the contents
against the expected render; and then go ahead and make the code change. But
notice how that actually would have left space for me to break things: I could
have defined the template as containing any arbitrary string, instead of
the string with the right <html>
and <title>
tags.
Tip
When refactoring, work on either the code or the tests, but not both at once.
Thereâs always a tendency to skip ahead a couple of steps, to make a couple of tweaks to the behaviour while youâre refactoring, but pretty soon youâve got changes to half a dozen different files, youâve totally lost track of where you are, and nothing works any more. If you donât want to end up like Refactoring Cat (Figure 4-2), stick to small steps; keep refactoring and functionality changes entirely separate.
Note
Weâll come across âRefactoring Catâ again during this book, as an example of what happens when we get carried away and want to change too many things at once. Think of it as the little cartoon demon counterpart to the Testing Goat, popping up over your other shoulder and giving you bad adviceâ¦
Itâs a good idea to do a commit after any refactoring:
$ git status # see tests.py, views.py, settings.py, + new templates folder $ git add . # will also add the untracked templates folder $ git diff --staged # review the changes we're about to commit $ git commit -m "Refactor home page view to use a template"
A Little More of Our Front Page
In the meantime, our functional test is still failing. Letâs now make an
actual code change to get it passing. Because our HTML is now in a template,
we can feel free to make changes to it, without needing to write any extra unit
tests. We wanted an <h1>
:
lists/templates/home.html
<html>
<head>
<title>
To-Do lists</title>
</head>
<body>
<h1>
Your To-Do list</h1>
</body>
</html>
Letâs see if our functional test likes it a little better:
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: [id="id_new_item"]
OKâ¦
lists/templates/home.html
[...]<h1>
Your To-Do list</h1>
<input
id=
"id_new_item"
/>
</body>
[...]
And now?
AssertionError: '' != 'Enter a to-do item'
We add our placeholder textâ¦
lists/templates/home.html
<input
id=
"id_new_item"
placeholder=
"Enter a to-do item"
/>
Which gives:
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: [id="id_list_table"]
So we can go ahead and put the table onto the page. At this stage itâll just be emptyâ¦
lists/templates/home.html
<input
id=
"id_new_item"
placeholder=
"Enter a to-do item"
/>
<table
id=
"id_list_table"
>
</table>
</body>
Now what does the FT say?
File "functional_tests.py", line 43, in test_can_start_a_list_and_retrieve_it_later any(row.text == '1: Buy peacock feathers' for row in rows) AssertionError: False is not true
Slightly cryptic. We can use the line number to track it down, and it turns out
itâs that any
function I was so smug about earlierâor, more precisely, the
assertTrue
, which doesnât have a very explicit failure message. We can pass
a custom error message as an argument to most assertX
methods in unittest
:
functional_tests.py
self
.
assertTrue
(
any
(
row
.
text
==
'1: Buy peacock feathers'
for
row
in
rows
),
"New to-do item did not appear in table"
)
If you run the FT again, you should see our message:
AssertionError: False is not true : New to-do item did not appear in table
But now, to get this to pass, we will need to actually process the userâs form submission. And thatâs a topic for the next chapter.
For now letâs do a commit:
$ git diff $ git commit -am "Front page HTML now generated from a template"
Thanks to a bit of refactoring, weâve got our view set up to render a template, weâve stopped testing constants, and weâre now well placed to start processing user input.
Recap: The TDD Process
Weâve now seen all the main aspects of the TDD process, in practice:
-
Functional tests
-
Unit tests
-
The unit-test/code cycle
-
Refactoring
Itâs time for a little recap, and perhaps even some flowcharts. Forgive me, years misspent as a management consultant have ruined me. On the plus side, it will feature recursion.
What is the overall TDD process? See Figure 4-3.
We write a test. We run the test and see it fail. We write some minimal code to get it a little further. We rerun the test and repeat until it passes. Then, optionally, we might refactor our code, using our tests to make sure we donât break anything.
But how does this apply when we have functional tests and unit tests? Well, you can think of the functional test as being a high-level view of the cycle, where âwriting the codeâ to get the functional tests to pass actually involves using another, smaller TDD cycle which uses unit tests. See Figure 4-4.
We write a functional test and see it fail. Then, the process of âwriting codeâ to get it to pass is a mini-TDD cycle of its own: we write one or more unit tests, and go into the unit-test/code cycle until the unit tests pass. Then, we go back to our FT to check that it gets a little further, and we can write a bit more of our applicationâusing more unit tests, and so on.
What about refactoring, in the context of functional tests? Well, that means we use the functional test to check that weâve preserved the behaviour of our application, but we can change or add and remove unit tests, and use a unit test cycle to actually change the implementation.
The functional tests are the ultimate judge of whether your application works or not. The unit tests are a tool to help you along the way.
This way of looking at things is sometimes called âDouble-Loop TDDâ. One of my eminent tech reviewers, Emily Bache, wrote a blog post on the topic, which I recommend for a different perspective.
Weâll explore all of the different parts of this workflow in more detail over the coming chapters.
1 You could also just use the string "\n"
, but Keys
also lets you send special keys like Ctrl so I thought Iâd show it.
2 Some people like to use another subfolder named after the app (i.e., lists/templates/lists) and then refer to the template as lists/home.html. This is called âtemplate namespacingâ. I figured it was overcomplicated for this small project, but it may be worth it on larger projects. Thereâs more in the Django tutorial.
3 Are you unable to move on because youâre wondering what those ch04l0xx things are, next to some of the code listings? They refer to specific commits in the bookâs example repo. Itâs all to do with my bookâs own tests. You know, the tests for the tests in the book about testing. They have tests of their own, naturally.
Get Test-Driven Development with Python, 2nd Edition now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.