Network Programmability and Automation, 2nd Edition

Chapter 4. Python

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 6th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at [e-mail].

As a network engineer, there has never been a better time for you to learn to automate and write code. As we articulated in Chapter 1, the network industry is fundamentally changing. It is a fact that networking had not changed much from the late 1990s to about 2010, both architecturally and operationally. In that span of time as a network engineer, you undoubtedly typed in the same CLI commands hundreds, if not thousands, of times to configure and troubleshoot network devices. Why the madness?

It is specifically around the operations of a network that learning to read and write some code starts to make sense. In fact, scripting or writing a few lines of code to gather information on the network, or to make change, isn’t new at all. It’s been done for years. There are engineers who took on this feat—​coding in their language of choice, learning to work with raw text using complex parsing, regular expressions, and querying SNMP MIBs in a script. If you’ve ever attempted this yourself, you know firsthand that it’s possible, but working with regular expressions and parsing text is time-consuming and tedious.

Luckily, things are starting to move in the right direction and the barrier to entry for network automation is more accessible than ever before. We are seeing advances from network vendors, but also in the open source tooling that is available to use for automating the network, both of which we cover in this book. For example, there are now network device APIs, vendor- and community-supported Python libraries, and freely available open source tools that give you and every other network engineer access to a growing ecosystem to jump start your network automation journey. This ultimately means that you have to write less code than you would have in the past, and less code means faster development and fewer bugs.

Before we dive into the basics of Python, there is one more important question that we’ll take a look at because it always comes up in conversation among network engineers: Should network engineers learn to code?

Should Network Engineers Learn to Code?

Unfortunately, you aren’t getting a definitive yes or no from us. Clearly, we have a full chapter on Python and another on Go, and plenty of other examples throughout the book on how to use Python or Go to communicate to network devices using network APIs and extend DevOps platforms like Ansible, Salt, and Puppet, so we definitely think learning the basics of any programming language is valuable. We also think it’ll become an even more valuable skill as the network and IT industries continue to transform at such a rapid pace, and we happen to think Python is a pretty great first choice.

Note

It’s worth pointing out that we do not hold any technology religion to Python (we are covering Go in Chapter 5). However, we feel when it comes to network automation it is a great first choice for several reasons. First, Python is a dynamically typed language that allows you to create and use Python objects (such as variables and functions) where and when needed, meaning they don’t need to be defined before you start using them. This simplifies the getting started process. Second, Python is also super readable. It’s common to see conditional statements like if device in device_list:, and in that statement, you can easily decipher that we are simply checking to see if a device is in a particular list of devices. Another reason is that network vendors and open source projects are building a great set of libraries and tools using Python. This just adds to the benefit of learning to program with Python.

The real question, though, is should every network engineer know how to read and write a basic script? The answer to that question would be a definite yes. Now should every network engineer become a software developer? Absolutely not. Many engineers will gravitate more toward one discipline than the next, and maybe some network engineers do transition to become developers, but all types of engineers, not just network engineers, should not fear trying to read through some Python or Ruby, or even more advanced languages like C, Go or Rust. System administrators have done fairly well already with using scripting as a tool to allow them to do their jobs more efficiently by using bash scripts, Python, Ruby, and PowerShell.

On the other hand, this hasn’t been the case for network administrators (which is a major reason for this book!). As the industry progresses and engineers evolve, it’s very realistic for you, as a network engineer, to be more DevOps oriented, in that you end up somewhere in the middle—​not as a developer, but also not as a traditional CLI-only network engineer. You could end up using open source configuration management and automation tools and then add a little code as necessary (and if needed) to accomplish and automate the workflows and tasks in your specific environment.

Warning

Unless your organization warrants it based on size, scale, compliance, or control, it’s not common or recommended to write custom software for everything and build a home-grown automation platform. It’s not an efficient use of time. What is recommended is that you understand the components involved in programming, software development, and especially fundamentals such as core data types that are common in all tools and languages, as we cover in this chapter focused on Python.

So we know the industry is changing, devices have APIs, and it makes sense to start the journey to learn to write some code. This chapter provides you with the building blocks to go from 0 to 60 to help you start your Python journey.

Throughout the rest of this chapter, we cover the following topics:

• Using the Python interactive interpreter

• Understanding Python data types

• Adding conditional logic to your code

• Understanding containment

• Using loops in Python

• Functions

• Working with files

• Creating Python programs

• Working with Python modules

• Classes

• Error Handling

• Parallelize your Python Programs

Get ready—​we are about to jump in and learn some Python!

Note

This chapter’s sole focus is to provide an introduction to Python foundational concepts for network engineers looking to learn Python to augment their existing skillsets. It is not intended to provide an exhaustive education for full-time developers to write production-quality Python software.

Additionally, please note the concepts covered in this chapter are heavily relevant outside the scope of Python. For example, you must understand concepts like loops and data types—which we’ll explore here—in order to work with tools like Ansible, Salt, Puppet, or Terraform.

Using the Python Interactive Interpreter

The Python interactive interpreter isn’t always known by those just starting out to learn to program or even those who have been developing in other languages, but we think it is a tool that everyone should know and learn before trying to create standalone executable scripts.

The interpreter is a tool that is instrumental to developers of all experience levels. The Python interactive interpreter, also commonly known as the Python shell, is used as a learning platform for beginners, but it’s also used by the most experienced developers to test and get real-time feedback without having to write a full program or script.

The Python shell, or interpreter, is found on nearly all native Linux distributions as well as many of the more modern network operating systems from vendors including, but not limited to, Cisco, HP, Juniper, Cumulus, and Arista.

To enter the Python interactive interpreter, you simply open a Linux terminal window, or SSH to a modern network device, type in the command python3, and hit Enter.

Note

All examples throughout this chapter that denote a Linux terminal command start with $. While you’re at the Python shell, all lines and commands start with >>>. Additionally, all examples shown are from a system running Ubuntu 20.04 LTS and Python 3.8.12.

It’s also important to note that since the first version of this book was published, Python 2.7 has reached End of Life. As a result, this chapter now exclusively uses Python 3 syntax and libraries.

For an explicit usage of Python 3 across the book, we use python3 to run Python scripts and launch the interactive interpreter, disambiguating with python alias, which could still be pointing to Python 2.

After entering the python command and hitting Enter, you are taken directly into the shell. While in the shell, you start writing Python code immediately! There is no text editor, no IDE, and no prerequisites to getting started.

$ python3
Python 3.8.12 (default, Feb 26 2022, 00:05:23)
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.

Although we are jumping into much more detail on Python throughout this chapter, we’ll take a quick look at a few examples right now to see the power of the Python interpreter.

Let’s start by simply creating a variable called hostname and assigning it the value of ROUTER_1.

>>> hostname = 'ROUTER_1'

Notice that you did not need to declare the variable first or define that hostname was going to be of type string. This is a departure from some programming languages such as C and Java, and a reason why Python is called a dynamic language.

Now, you can print the variable hostname.

>>> print(hostname)
ROUTER_1
>>>
>>> hostname
'ROUTER_1'

Once you’ve created the variable you can easily print it using the print command, but while in the shell, you have the ability to also display the value of hostname or any variable by just typing in the name of the variable and pressing Enter. One difference to point out between these two methods is that when you use the print statement, characters such as the end of line (or \n) are interpreted, but are not when you’re not using the print statement.

For example, using print interprets the \n and a new line is printed, but when you’re just typing the variable name into the shell and hitting Enter, the \n is not interpreted and is just displayed to the terminal.

>>> banner = "\n\n   WELCOME TO ROUTER_1   \n\n"
>>>
>>> print(banner)


   WELCOME TO ROUTER_1


>>>
>>> banner
'\n\n   WELCOME TO ROUTER_1   \n\n'

Can you see the difference?

When you are validating or testing, the Python shell is a great tool to use. In the preceding examples, you may have noticed that single quotes and double quotes were both used. Now you may be thinking, could they be used together on the same line? Let’s not speculate about it; let’s use the Python shell to test it out.

>>> hostname = 'ROUTER_1"
  File "<stdin>", line 1
    hostname = 'ROUTER_1"
                        ^
SyntaxError: EOL while scanning string literal

And just like that, we verified that Python supports both single and double quotes, but learned they cannot be used together.

Most examples throughout this chapter continue to use the Python interpreter—​feel free to follow along and test them out as they’re covered.

We’ll continue to use the Python interpreter as we review the different Python data types with a specific focus on networking.

Understanding Python Data Types

This section provides an overview of various Python data types including strings, numbers (integers and floats), booleans, lists, and dictionaries and also touches upon tuples and sets.

The sections on strings, lists, and dictionaries are broken up into two parts. The first is an introduction to the data type and the second covers some of its built-in methods. As you’ll see, methods are natively part of Python, making it extremely easy for developers to manipulate and work with each respective data type.

For example, a method called upper that takes a string and converts it to all uppercase letters can be executed with the statement, "router1".upper(), which returns ROUTER1. We’ll show many more examples of using methods throughout this chapter.

The sections on integers and booleans provide an overview to show you how to use mathematical operators and boolean expressions while writing code in Python.

Finally, we close the section on data types by providing a brief introduction to tuples and sets. They are more advanced data types, but we felt they were still worth covering in an introduction to Python.

Table 4-1 describes and highlights each data type we’re going to cover in this chapter. This should act as a reference throughout the chapter.

Table 4-1. Python data types summary

Data Type	Description	Short Name (Type)	Characters	Example
String	Series of any characters surrounded by quotes	str	""	hostname="nycr01"
Integer	Whole numbers represented without quotes	int	n/a	eos_qty=5
Float	Floating point number (decimals)	float	n/a	cpu_util=52.33
Boolean	Either True or False (no quotes)	bool	n/a	is_switchport=True
List	Ordered sequence of values. Values can be of any data type.	list	[]	vendors=['cisco', 'juniper', 'arista', 'cisco']
Dictionary	Ordered collection of key-value pairs	dict	{}	facts={"vendor":"cisco", "platform":"catalyst", "os":"ios"}
Set	Unordered collection of unique elements	set	set()	set(vendors)=>['cisco', 'juniper', 'arista']
Tuple	Ordered and unchangeable sequence of values	tuple	()	ipaddr=(10.1.1.1, 24)

Let’s get started and take a look at Python strings.

Learning to Use Strings

Strings are a sequence of characters that are enclosed by quotes and are arguably the most well-known data type that exists in all programming languages.

Warning

Strings could look like a simple data type, but they are slightly more complex. A string could represent two different kinds of data - text and bytes. In Python 2, both types of data were represented as a str, but in Python 3, for each one there are separate and incompatible types.

You can go deeper on the differences, but in short, in Python 2 strings are ASCII characters and in Python 3 are Unicode characters (by default UTF-8), so if you receive bytes you will need to encode them to use string methods, using one of the available encode types, such as UTF-8, UTF-16, ASCII or others.

Earlier in the chapter, we looked at a few basic examples for creating variables that were of type str. Let’s examine what else you need to know when starting to use strings.

First, we’ll define two new variables that are both strings: final and ipaddr.

>>> final = 'The IP address of router1 is: '
>>>
>>> ipaddr = '1.1.1.1'

Tip

You can use the built-in function called type to verify the data type of any given object in Python.

>>> type(final)
<class 'str'>

This is how you can easily check the type of one object, which is often helpful in troubleshooting code, especially if it’s code you didn’t write.

Next, let’s look at how to combine, add, or concatenate strings.

>>> final + ipaddr
'The IP address of router1 is: 1.1.1.1'

So far, we created two new variables: final and ipaddr. Each is a string. After they were both created, we concatenated them using the + operator, and finally printed them out. Fairly easy, right?

The same could be done even if final was not a predefined object:

>>> print('The IP address of router1 is: ' + ipaddr)
The IP address of router1 is: 1.1.1.1

Using built-in methods of strings

To view the available built-in methods for strings, you use the built-in dir() function while in the Python shell. You first create any variable that is a string or use the formal data type name of str and pass it as an argument to dir() to view the available methods.

Tip

dir() can be used on any Python object, not just strings, as we’ll show throughout this chapter.

>>> dir(str)
# output has been omitted
['__add__', '__class__', '__contains__', '__delattr__', '__doc__',
'endswith', 'expandtabs', 'find', 'format', 'index', 'isalnum', 'isalpha',
'isdigit', 'islower', 'isspace', 'istitle', 'isupper', 'join', 'lower',
'lstrip', 'replace', 'rstrip', 'split', 'splitlines', 'startswith',
'strip', 'upper']

Note

Across the chapter, in order to keep the output clean and simplify some examples, we will add an omitted comment in some outputs.

In this example, we’ve removed some objects that start and end with double underscores(”__“), also known as dunders. These internal methods are used to implement some magic operations without explicitly calling them.

For instance, as we will see in “Understanding Python Classes”, when an object is created from a class, there is an implicit call to the init() method (the constructor) and it returns an object. By overwriting this dunder, you can customize how your objects will be created.

To reiterate what we said earlier, it’s possible to also pass any string to the dir() function to produce the same output as above. For example, if you defined a variable such as hostname = 'ROUTER', hostname can be passed to dir()—that is, dir(hostname)—producing the same output as dir(str) to determine what methods are available for strings.

Navigate the Code with: Type(), Dir() and Help()

In order to learn how to use a given method that you see in the output of a dir(), you can use the built-in function called help(). In order to use the built-in help feature, you pass in the object (or variable) and the given method. The following example shows how you can use help() and learn how to use the upper method:

>>> help(hostname.upper)
Help on built-in function upper:

upper() method of builtins.str instance
    Return a copy of the string converted to uppercase.
(END)

When you’re finished, enter Q to quit viewing the built-in help.

At this point we have already introduced the three functions that will help you navigate through the code getting understanding of it. Here is our recommended flow:

1. Check your data type using type()

2. Check the available methods for your object using dir()

3. After knowing which method you want to use, learn how to use it using help()

Here is an example of that workflow:

>>> hostname = 'router1'
>>>
>>> type(hostname)
<class 'str'>
>>>
>>> dir(hostname)
>>> # output has been omitted, same as above
>>>
>>> help(hostname.upper)
>>> # output has been omitted, same as above

Let’s take a look at several of the string methods, including count, endswith, startswith, format, isdigit, join, lower, upper, and strip.

As we review each method, there are two key questions that you should be asking yourself. What value is returned from the method? And what action is the method performing on the original object?

Using the upper() and lower() methods

Using the upper() and lower() methods is helpful when you need to compare strings that do not need to be case-sensitive. For example, maybe you need to accept a variable that is the name of an interface such as “Ethernet1/1,” but want to also allow the user to enter “ethernet1/1.” The best way to compare these is to use upper() or lower().

>>> interface = 'Ethernet1/1'
>>>
>>> interface.lower()
'ethernet1/1'
>>>
>>> interface.upper()
'ETHERNET1/1'

You can see that when you’re using a method, the format is to enter the object name, or string in this case, and then append .methodname():

• “.”: to access a method (or an attribute) of an object

• “methodname”: the name of the method

• “()”: to call the method. As we will see soon, it could take arguments

Next, after executing interface.lower(), notice that ethernet1/1 was printed to the terminal. This is telling us that ethernet1/1 was returned when lower() was executed. The same holds true for upper(). When something is returned, you also have the ability to assign it as the value to a new or existing variable.

>>> intf_lower = interface.lower()
>>>
>>> print(intf_lower)
ethernet1/1

In this example, you can see how to use the method, but also assign the data being returned to a variable.

What about the original variable called interface? Let’s see what, if anything, changed with interface.

>>> print(interface)
Ethernet1/1

Since we are exploring the first examples, it still may not be clear what we’re looking for to see if something changed in the original variable interface, but we do know that it still holds the value of Ethernet1/1 and nothing changed. Don’t worry, we’ll see plenty of examples of when the original object is modified throughout this chapter.

Using the startswith() and endswith() methods

As you can probably guess, startswith() is used to verify whether a string starts with a certain sequence of characters, and endswith() is used to verify whether a string ends with a certain sequence of characters.

>>> ipaddr = '10.100.20.5'
>>>
>>> ipaddr.startswith('10')
True
>>>
>>> ipaddr.startswith('100')
False
>>>
>>> ipaddr.endswith('.5')
True

The lower() and upper() methods returned a string, and that string was a modified string with all lowercase or uppercase letters. However, startswith() and endswith() do not return a string, but rather a boolean (bool) object. As you’ll learn later in this chapter, boolean values are True and False. The startswith() method returns True if the sequence of characters being passed in matches the respective starting or ending sequence of the object. Otherwise, it returns False.

Note

Take note that boolean values are either True or False, no quotes are used for booleans, and the first letter must be capitalized. Booleans are covered in more detail later in the chapter.

Using these methods proves to be valuable when you’re looking to verify the start or end of a string. Maybe it’s to verify the first or fourth octet of an IPv4 address, or maybe to verify an interface name, just like we had in the previous example using lower(). Rather than assume a user of a script was going to enter the full name, it’s advantageous to do a check on the first two characters to allow the user to input “ethernet1/1”, “eth1/1” and “et1/1”.

For this check, we’ll show how to combine methods, or use the return value of one method as the base string object for the second method.

>>> interface = 'Eth1/1'
>>>
>>> interface.lower().startswith('et')
True

We verify it is an Ethernet interface by first executing lower(), which returns eth1/1, and then the boolean check is performed to see whether “eth1/1” starts with “et”. And, clearly, it does.

Of course, there are other things that could be invalid beyond the “eth” in an interface string object, but the point is that methods can be easily used together.

Using the strip() method

Some legacy network devices still don’t have application programming interfaces, or APIs. It is very plausible that at some point if you want to write a script, you’ll try it out on an older CLI-based device. If you do this, you’ll be sure to encounter globs of raw text coming back from the device—​this could be the result of any show command from the output of show interfaces to a full show running-config.

When you need to store or simply print something, you may not want any whitespace wrapping the object you want to use or see. In trying to be consistent with previous examples, this may be an IP address.

What if the object you’re working with has the value of " 10.1.50.1 " including the whitespace. The methods startswith() or endswith() do not work because of the spaces. For these situations, strip() is used to remove the whitespace.

>>> ipaddr = '   10.1.50.1   '
>>>
>>> ipaddr.strip()
'10.1.50.1'

Using strip() returned the object without any spaces on both sides. Examples aren’t shown for lstrip() or rstrip(), but they are two other built-in methods for strings that remove whitespace specifically on the left side or right side of a string object.

Using the isdigit() method

There) may be times you’re working with strings, but need to verify the string object is a number. Technically, integers are a different data type (covered in the next section), but numbers can still be values in strings.

Using isdigit() makes it extremely straightforward to see whether the character or string is actually a digit.

>>> ten = '10'
>>>
>>> ten.isdigit()
True
>>>
>>> bogus = '10a'
>>>
>>> bogus.isdigit()
False

Just as with startswith(), isdigit() also returns a boolean. It returns True if the value is an integer, otherwise it returns False.

Using the count() method

Imagine working with a binary number—​maybe it’s to calculate an IP address or subnet mask. While there are some built-in libraries to do binary-to-decimal conversion, what if you just want to count how many 1’s or 0’s are in a given string? You can use count() to do this for you.

>>> octet = '11111000'
>>>
>>> octet.count('1')
5

The example shows how easy it is to use the count() method. This method, however, returns an int (integer) unlike any of the previous examples.

When using count(), you are not limited to sending a single character as a parameter either.

>>> octet.count('111')
1
>>>
>>> test_string = "Don't you wish you started programming a little earlier?"
>>>
>>> test_string.count('you')
2

Using the format() method

We saw earlier how to concatenate strings. Imagine needing to create a sentence, or better yet, a command to send to a network device that is built from several strings or variables. How would you format the string, or CLI command?

Let’s use ping as an example and assume the command that needs to be created is the following:

ping 8.8.8.8 vrf management

Note

In the examples in this chapter, the network CLI commands being used are generic, as no actual device connections are being made. Thus, they map to no specific vendor as they are the “industry standard” examples that work on various vendors including Cisco IOS, Cisco NXOS, Arista EOS, and many others.

If you were writing a script, it’s more than likely the target IP address you want to send ICMP echo requests to and the virtual routing and forwarding (VRF) will both be user input parameters. In this particular example, it means '8.8.8.8' and 'management' are the input arguments (parameters).

One way to build the string is to start with the following:

>>> ipaddr = '8.8.8.8'
>>> vrf = 'management'
>>>
>>> ping = 'ping' + ipaddr + 'vrf' + vrf
>>>
>>> print(ping)
ping8.8.8.8vrfmanagement

You see the spacing is incorrect, so there are two options—​add spaces to your input objects or within the ping object. Let’s look at adding them within ping.

>>> ping = 'ping' + ' ' + ipaddr + ' ' + 'vrf ' + vrf
>>>
>>> print(ping)
ping 8.8.8.8 vrf management

As you can see, this works quite well and is not too complicated, but as the strings or commands get longer, it can get quite messy dealing with all the quotes and spaces. Using the format() method can simplify this.

>>> ping = 'ping {} vrf {}'.format(ipaddr, vrf)
>>>
>>> print(ping)
ping 8.8.8.8 vrf management

The format() method takes a number of arguments, which are inserted between the curly braces ({}) found within the string. Notice how the format() method is being used on a raw string, unlike the previous examples.

Note

It’s possible to use any of the string methods on both variables or raw strings. This is true for any other data type and its built-in methods as well.

The next example shows using the format() method, with a pre-created string object (variable) in contrast to the previous example, when it was used on a raw string.

>>> ping = 'ping {} vrf {}'
>>>
>>> command = ping.format(ipaddr, vrf)
>>>
>>> print(command)
ping 8.8.8.8 vrf management

This scenario is more likely, in that you would have a predefined command in a Python script with users inputting two arguments, and the output is the final command string that gets pushed to a network device.

Note

Another option to concatenate strings is using the % operator. One example for inserting strings (s) is provided here:

>>> hostname = 'r5'
>>> interface = 'Eth1/1'
>>>
>>> test  = 'Device %s has one interface: %s' % (hostname, interface)
>>> print(test)
Device r5 has one interface: Eth1/1

Since Python 3.6, there is a new formatting option for strings available: f-strings, which has an “f” at the beginning of the string and uses “{}” to contain the expressions that will be rendered to strings. This formatting option has gained a lot of popularity within the Python community because it helps to visualize where the variables are used.

>>> f'ping {ipaddr} vrf {vrf}'
'ping 8.8.8.8 vrf management'

Tip

When using f-strings for debugging, you can use “=” after the object name to populate the name of the object rendered.

>>> f'Rendering command with: {ipaddr=} {vrf=}'
"Rendering command with: ipaddr='8.8.8.8' vrf='management'"

Using the join() and split() methods

These are the last methods for strings covered in this chapter. We saved them for last since they include working with another data type called list.

Note

Be aware that lists are formally covered later in the chapter, but we wanted to include a very brief introduction here in order to show the join() and split() methods for string objects.

Lists are exactly what they sound like. They are a list of objects—​each object is called an element, and each element is of the same or different data type. Note that there is no requirement to have all elements in a list be of the same data type.

If you had an environment with five routers, you may have a list of hostnames.

>>> hostnames = ['r1', 'r2', 'r3', 'r4', 'r5']

You can also build a list of commands to send to a network device to make a configuration change. The next example is a list of commands to shut down an Ethernet interface on a switch.

>>> commands = ['config t', 'interface Ethernet1/1', 'shutdown']

It’s quite common to build a list like this, but if you’re using a traditional CLI-based network device, you might not be able to send a list object directly to the device. The device may require strings be sent (or individual commands).

join() is one such method that can take a list and create a string, but insert required characters, if needed, between them.

Remember that \n is the end of line (EOL) character. When sending commands to a device, you may need to insert a \n between commands to allow the device to render a new line for the next command.

If we take commands from the previous example, we can see how to leverage join() to create a single string with a \n inserted between each command.

>>> '\n'.join(commands)
'config t\ninterface Ethernet1/1\nshutdown'

Another practical example is when using an API such as NX-API that exists on Cisco Nexus switches. Cisco gives the option to send a string of commands, but they need to be separated by a semicolon (;).

To do this, you would use the same approach.

>>> ' ; '.join(commands)
'config t ; interface Ethernet1/1 ; shutdown'

In this example, we added a space before and after the semicolon, but it’s the same overall approach.

Tip

In the examples shown, a semicolon and an EOL character were used as the separator, but you should know that you don’t need to use any characters at all. It’s possible to concatenate the elements in the list without inserting any characters, like this: ''.join(list_name).

You learned how to use join() to create a string out of a list, but what if you needed to do the exact opposite and create a list from a string? One option is to use the split() method.

In the next example, we start with the previously generated string, and convert it back to a list.

>>> commands = 'config t ; interface Ethernet1/1 ; shutdown'
>>>
>>> cmds_list = commands.split(' ; ')
>>>
>>> print(cmds_list)
['config t', 'interface Ethernet1/1', 'shutdown']

This shows how simple it is to take a string object and create a list from it. Another common example for networking is to take an IP address (string) and convert it to a list using split(), creating a list of four elements—​one element per octet.

>>> ipaddr = '10.1.20.30'
>>>
>>> ipaddr.split('.')
['10', '1', '20', '30']

That covered the basics of working with Python strings. Let’s move on to the next data type, which is numbers.

Learning to Use Numbers

We don’t spend much time on different types of numbers such as floats (decimal numbers) or imaginary numbers, but we do briefly look at the data type that is denoted as int, better known as an integer. Quite frankly, this is because most people understand numbers and there aren’t built-in methods that make sense to cover at this point. Rather than cover built-in methods for integers, we take a look at using mathematical operators while in the Python shell.

Note

You should also be aware that decimal numbers in Python are referred to as floats. Remember, you can always verify the data type by using the built-in function type():

>>> cpu = 41.3
>>>
>>> type(cpu)
<class 'float'>

Performing mathematical operations

If you need to add numbers, there is nothing fancy needed: just add them.

>>> 5 + 3
8
>>> a = 1
>>> b = 2
>>> a + b
3

There may be a time when a counter is needed as you are looping through a sequence of objects. You may want to say counter = 1, perform some type of operation, and then do counter = counter + 1. While this is perfectly functional and works, it is more idiomatic in Python to perform the operation as counter += 1. This is shown in the next example.

>>> counter = 1
>>> counter = counter + 1
>>> counter
2
>>>
>>> counter = 5
>>> counter += 5
>>>
>>> counter
10

Very similar to addition, there is nothing special for subtraction. We’ll dive right into an example.

>>> 100 - 90
10
>>> count = 50
>>> count - 20
30

When multiplying, yet again, there is no difference. Here is a quick example.

>>> 100 * 50
5000
>>>
>>> print(2 * 25)
50

The nice thing about the multiplication operator (*) is that it’s also possible to use it on strings. You may want to format something and make it nice and pretty.

>>> print('=' * 50)
==================================================

The preceding example is extremely basic, and at the same time extremely powerful. Not knowing this is possible, you may be tempted to print one line a time and print a string with the command print(================), but in reality after learning this and a few other tips covered later in the chapter, pretty-printing text data becomes much simpler.

If you haven’t performed any math by hand in recent years, division may seem like a nightmare. As expected, though, it is no different from the previous three mathematical operations reviewed. Well, sort of.

There is not a difference with how you enter what you want to accomplish. To perform an operation you still use 10 / 2 or 100 / 50, and so on, like so:

>>> 100 / 50
2

These examples are probably what you expected to see.

The difference is what is returned when there is a remainder:

>>> 12 / 10
1

As you know, the number 10 goes into 12 one time. This is what is known as the quotient, so here the quotient is equal to 1. What is not displayed or returned is the remainder. To see the remainder in Python, you must use the %, or modulus operation.

>>> 12 % 10
2

This means to fully calculate the result of a division problem, both the / and % operators are used.

Tip

You can convert (or cast) an integer to a string by using str(10), and also do the opposite, converting a string to an integer by using int('10').

>>> str(10)
'10'
>>> int('10')
10

That was a brief look at how to work with numbers in Python. We’ll now move on to booleans.

Learning to Use Booleans

Boolean objects, otherwise known as objects that are of type bool in Python, are fairly straightforward. Let’s first review the basics of general boolean logic by looking at a truth table (Table 4-2).

Table 4-2. Boolean truth table

A	B	A and B	A or B	Not A
False	False	False	False	True
False	True	False	True	True
True	False	False	True	False
True	True	True	True	False

Notice how all values in the table are either True or False. This is because with boolean logic all values are reduced to either True or False. This actually makes booleans easy to understand.

Since boolean values can be only True or False, all expressions also evaluate to either True or False. You can see in the table that BOTH values, for A and B, need to be True, for “A and B” to evaluate to True. And “A or B” evaluates to True when ANY value (A or B) is True. You can also see that when you take the NOT of a boolean value, it calculates the inverse of that value. This is seen clearly as “NOT False” yields True and “NOT True” yields False.

From a Python perspective, nothing is different. We still only have two boolean values, and they are True and False. To assign one of these values to a variable within Python, you must enter it just as you see it, (with a capitalized first letter, and without quotes).

>>> exists = True
>>>
>>> exists
True
>>>
>>> exists = true
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'true' is not defined

As you can see in this example, it is quite simple. Based on the real-time feedback of the Python interpreter, we can see that using a lowercase t doesn’t work when we’re trying to assign the value of True to a variable.

Here are a few more examples of using boolean expressions while in the Python interpreter.

>>> True and True
True
>>>
>>> True or False
True
>>>
>>> False or False
False

In the next example, these same conditions are evaluated, assigning boolean values to variables.

>>> value1 = True
>>> value2 = False
>>>
>>> value1 and value2
False
>>>
>>> value1 or value2
True

Notice that boolean expressions are also not limited to two objects.

>>> value3 = True
>>> value4 = True
>>>
>>> value1 and value2 and value3 and value4
False
>>>
>>> value1 and value3 and value4
True

When extracting information from a network device, it is quite common to use booleans for a quick check. Is the interface a routed port? Is the management interface configured? Is the device reachable? While there may be a complex operation to answer each of those questions, the result is stored as True or False.

The counter to those questions would be, is the interface a switched port or is the device not reachable? It wouldn’t make sense to have variables or objects for each question, but we could use the not operator, since we know the not operation returns the inverse of a boolean value.

Let’s take a look at using not in an example.

>>> not False
>>> True
>>>
>>> is_layer3 = True
>>> not is_layer3
False

In this example, there is a variable called is_layer3. It is set to True, indicating that an interface is a Layer 3 port. If we take the not of is_layer3, we would then know if it is a Layer 2 port.

We’ll be taking a look at conditionals (if-else statements) later in the chapter, but based on the logic needed, you may need to know if an interface is in fact Layer 3. If so, you would have something like if is_layer3:, but if you needed to perform an action if the interface was Layer 2, then you would use if not is_layer3:.

In addition to using the and and or operands, the equal to == and does not equal to != expressions are used to generate a boolean object. With these expressions, you can do a comparison, or check, to see if two or more objects are (or not) equal to one another.

>>> True == True
True
>>>
>>> True != False
True
>>>
>>> 'network' == 'network'
True
>>>
>>> 'network' == 'no_network'
False

Empty objects

Python offers a simple and readable way to evaluate if an object is empty or not. If the object is not empty, the evaluation will return True, and if empty, will return False. In the following example, you can notice how easy is reading the code if not devices:. It’s almost literal, if there are no devices, it will match the if statement.

>>> devices = []
>>> if not devices:
...     print('devices is empty')
...
devices is empty
>>>
>>> hostname = 'something'
>>>
>>> if hostname:
...    print('hostname is not empty')
...
hostname is not empty

However, even though this evaluation usually matches what we understand as emptiness, the actual logic applied is the boolean evaluation of the object, implemented via the bool dunder method, and could be customized for our own class as we will explore in “Understanding Python Classes”.

After a quick look at working with boolean objects, operands, and expressions, we are ready to cover how to work with Python lists.

Learning to Use Python Lists

You had a brief introduction to lists when we covered the string built-in methods called join() and split(). Lists are now covered in a bit more detail.

Lists are the object type called list, and at their most basic level are an ordered sequence of objects. The examples from earlier in the chapter when we looked at the join() method with strings are provided again next to provide a quick refresher on how to create a list. Those examples were lists of strings, but it’s also possible to have lists of any other data type as well, which we’ll see shortly.

>>> hostnames = ['r1', 'r2', 'r3', 'r4', 'r5']
>>> commands = ['config t', 'interface Ethernet1/1', 'shutdown']

The next example shows a list of objects where each object is a different data type!

>>> new_list = ['router1', False, 5]
>>>
>>> print(new_list)
['router1', False, 5]

Now you understand that lists are an ordered sequence of objects and are enclosed by brackets. One of the most common tasks when you’re working with lists is to access an individual element of the list.

Let’s create a new list of interfaces and show how to print a single element of a list.

>>> interfaces = ['Eth1/1', 'Eth1/2', 'Eth1/3', 'Eth1/4']

The list is created and now three elements of the list are printed one at a time.

>>> print(interfaces[0])
Eth1/1
>>>
>>> print(interfaces[1])
Eth1/2
>>>
>>> print(interfaces[3])
Eth1/4

To access the individual elements within a list, you use the element’s index value enclosed within brackets. It’s important to see that the index begins at 0 and ends at the “length of the list minus 1.” This means in our example, to access the first element you use interfaces[0] and to access the last element you use interfaces[3].

In the example, we can easily see that the length of the list is four, but what if you didn’t know the length of the list?

Luckily Python provides a built-in function called len() to help with this.

>>> len(interfaces)
4

So, you can access the last element of the list subtracting one from its length: list_name[len(list_name) - 1]:

>>> interfaces[len(interfaces) - 1]
'Eth1/4'

Another way, more pythonic, to access the last element in any list is: list_name[-1]. So, with the minus we are indexing the list from the end, instead of the beginning.

>>> interfaces[-1]
'Eth1/4'

Note

Oftentimes, the terms function and method are used interchangeably, but up until now we’ve mainly looked at methods, not functions. The slight difference is that a function is called without referencing a parent object. As you saw, when you use a built-in method of an object, it is called using the syntax object.method(), and when you use functions like len(), you call it directly. That said, it is very common to call a method a function.

The same indexing access used for lists, also works for strings. If there is a variable called router that is assigned the value of 'DEVICE', router[0] returns 'D'.

However, a string is not a list, so you can’t assign a new value to the indexed position. String is an immutable type; it cannot be modified. Notice that if you assign a different value- hostname = 'something else'- you are not modifying the string, you are creating a new one. If we try to modify an element of a string, we will observe an error raised.

>>> hostname[1] = 'A'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'str' object does not support item assignment

A useful technique, to produce a subset of a sequence (such as lists or strings), is slicing. Using “:” before or after the index, you can indicate that you want to retrieve all the elements before or after the index, respectively. This can become pretty powerful when you need to parse through different types of objects.

>>> hostname = 'DEVICE_12345'
>>>
>>> hostname[4:]
'CE_12345'
>>>
>>> hostname[:-2]
'DEVICE_123'

Using built-in methods of Python lists

To view the available built-in methods for lists, the dir() function is used just like we showed previously when working with string objects. You can create any variable that is a list or use the formal data type name of list and pass it as an argument to dir(). We’ll use the interfaces list for this.

>>> dir(interfaces)
# omitted dunders
['append', 'clear', 'copy', 'count', 'extend', 'index',
'insert', 'pop', 'remove', 'reverse', 'sort']

Let’s take a look at a few of these built-in methods.

Using the append() method

The great thing about these method names, as you’ll continue to see, is that they are human-readable, and for the most part, intuitive. The append() method is used to append an element to an existing list.

This is shown in the next example, but let’s start with creating an empty list. You do so by assigning empty brackets to an object.

>>> vendors = []

Let’s append, or add vendors to this list.

>>> vendors.append('arista')
>>>
>>> print(vendors)
['arista']
>>>
>>> vendors.append('cisco')
>>>
>>> print(vendors)
['arista', 'cisco']

You can see that using append() adds the element to the last position in the list. In contrast to many of the methods reviewed for strings, this method is not returning anything, but modifying the original variable, or object.

Using the insert() method

Rather than just append an element to a list, you may need to insert an element at a specific location. This is done with the insert() method.

To use insert(), you need to pass it two arguments. The first argument is the position, or index, where the new element gets stored, and the second argument is the actual object getting inserted into the list.

In the next example, we’ll look at building a list of commands.

>>> commands = ['interface Eth1/1', 'ip address 1.1.1.1/32']

Note

As a reminder, the commands in these examples are generic and do not map back to a specific vendor or platform.

Let’s now assume we need to add two more commands to the previous commands list. The command that needs to be added as the first element is config t and the one that needs to be added just before the IP address is no switchport.

>>> commands = ['interface Eth1/1', 'ip address 1.1.1.1/32']
>>>
>>> commands.insert(0, 'config t')
>>>
>>> print(commands)
['config t', 'interface Eth1/1', 'ip address 1.1.1.1/32']
>>>
>>> commands.insert(2, 'no switchport')
>>>
>>> print(commands)
['config t', 'interface Eth1/1', 'no switchport', 'ip address 1.1.1.1/32']

Tip

Using indexes bigger than the actual length of the list will not raise an error. It will simply insert the objects at the end of the list.

>>> commands.insert(9999, "shutdown")
>>> commands
['config t', 'interface Eth1/1', 'no switchport', 'ip address 1.1.1.1/32',
 'shutdown']

Using the count() method

If you are doing an inventory of types of devices throughout the network, you may build a list that has more than one of the same object within a list. To expand on the example from earlier, you may have a list that looks like this:

>>> vendors = ['cisco', 'cisco', 'juniper', 'arista', 'cisco', 'hp', 'cumulus',
'arista', 'cisco']

You can count how many instances of a given object are found by using the count() method. In our example, this can help determine how many Cisco or Arista devices there are in the environment.

>>> vendors.count('cisco')
4
>>>
>>> vendors.count('arista')
2

Take note that count() returns an int, or integer, and does not modify the existing object like insert(), append(), and a few others that are reviewed in the upcoming examples.

Using the pop() and index() methods

Most of the methods thus far have either modified the original object or returned something. pop() does both.

>>> hostnames = ['r1', 'r2', 'r3', 'r4', 'r5']

The preceding example has a list of hostnames. Let’s pop (remove) r5 because that device was just decommissioned from the network.

>>> hostnames.pop()
'r5'
>>>
>>> print(hostnames)
['r1', 'r2', 'r3', 'r4']

As you can see, the element being popped is returned and the original list is modified as well.

You should have also noticed that no element or index value was passed in, so you can see by default, pop() pops the last element in the list.

What if you need to pop "r2"? It turns out that in order to pop an element that is not the last element, you need to pass in an index value of the element that you wish to pop. But how do you find the index value of a given element? This is where the index() method comes into play.

To find the index value of a certain element, you use the index() method.

>>> hostnames.index('r2')
1

Here you see that the index of the value "r2" is 1.

So, to pop "r2", we would perform the following:

>>> hostnames.pop(1)
'r2'
>>>
>>> print(hostnames)
['r1', 'r3', 'r4']

It could have also been done in a single step:

>>> hostnames.pop(hostnames.index('r2'))

Using the sort() method

The last built-in method that we’ll take a look at for lists is sort(). As you may have guessed, sort() is used to sort a list.

In the next example, we have a list of IP addresses in non-sequential order, and sort() is used to update the original object. Notice that nothing is returned.

>>> available_ips = ['10.1.1.1', '10.1.1.9', '10.1.1.8', '10.1.1.7', '10.1.1.4']
>>>
>>> available_ips.sort()
>>>
>>> available_ips
['10.1.1.1', '10.1.1.4', '10.1.1.7', '10.1.1.8', '10.1.1.9']

Note

Be aware that the sort from the previous example sorted IP addresses as strings, instead of IP addresses. So, if in available_ips we had the 2.2.2.2 IP, even being a smaller IP address, it will be sorted the last, because, as characters, '10' < '2'.

A useful practice when working with methods is to check the available customization options. Usually, a method behavior has already implemented some tuning options to cover the common use cases. help() will show you all the details.

>>> help(list.sort)
Help on method_descriptor:

sort(self, /, *, key=None, reverse=False)
    Sort the list in ascending order and return None.
    ...

    The reverse flag can be set to sort in descending order.
(END)

The sort() method supports a named boolean argument, reverse, that can sort the list in descending order.

>>> available_ips.sort(reverse=True)
>>>
>>> available_ips
['10.1.1.9', '10.1.1.8', '10.1.1.7', '10.1.1.4', '10.1.1.1']

In nearly all examples we covered with lists, the elements of the list were the same type of object; that is, they were all strings: commands, IP addresses, vendors, or hostnames. However, Python allows you to create lists that store different types or objects.

A prime example of storing different objects arises when storing information about a particular device. Maybe you want to store the hostname, vendor, and OS. A list to store these device attributes would look something like this:

>>> device = ['router1', 'juniper', '12.2']

Since elements of a list are indexed by an integer, you need to keep track of which index is mapped to which particular attribute. While it may not seem hard for this example, what if there were 10, 20, or 100 attributes that needed to be accessed? Even if there were mappings available, it could get extremely difficult since lists are ordered. Replacing or updating any element in a list would need to be done very carefully.

Wouldn’t it be nice if you could reference the individual elements of a list by name and not worry so much about the order of elements? So, rather than access the hostname using device[0], you could access it like device['hostname'].

As luck would have it, this is exactly where Python dictionaries come into action, and they are the next data type we cover in this chapter.

Learning to Use Python Dictionaries

We’ve now reviewed some of the most common data types, including strings, integers, booleans, and lists, which exist across all programming languages. In this section, we take a look at the dictionary, which is a Python-specific data type. In other languages, they are known as associative arrays, maps, or hash maps.

Dictionaries are ordered lists by insertion and their values are accessed by names, otherwise known as keys, instead of by index (integer). Dictionaries are simply a collection of key-value pairs called items.

Note

In earlier versions of Python, dictionaries were an unordered data structure. In other words, you were not able to warrant the order of the items. However, as of Python 3.7, dictionaries are guaranteed to preserve the order in which key/value pairs are inserted.

We finished the previous section on lists using this example:

>>> device = ['router1', 'juniper', '12.2']

If we build on this example and convert the list device to a dictionary, it would look like this:

>>> device = {'hostname': 'router1', 'vendor': 'juniper', 'os': '12.1'}

The notation for a dictionary is a curly brace ({), then key, colon, and value, for each key-value pair separated by a comma (,), and then it closes with another curly brace (}).

Once the dict object is created, you access the desired value by using dict[key].

>>> print(device['hostname'])
router1
>>>
>>> print(device['os'])
12.1
>>>
>>> print(device['vendor'])
juniper

It’s worth noting that it’s possible to create the same dictionary from the previous example a few different ways. These are shown in the next two code blocks.

>>> device = {}
>>> device['hostname'] = 'router1'
>>> device['vendor'] = 'juniper'
>>> device['os'] = '12.1'
>>>
>>> print(device)
{'hostname': 'router1', 'vendor': 'juniper', 'os': '12.1'}

>>> device = dict(hostname='router1', vendor='juniper', os='12.1')
>>>
>>> print(device)
{'hostname': 'router1', 'vendor': 'juniper', 'os': '12.1'}

Using built-in methods of Python dictionaries

Python dictionaries have a few built-in methods worth covering, so as usual, we’ll dive right into them.

Just as with the other data types, we first look at all available methods minus those that start and end with underscores.

>>> dir(dict)
['clear', 'copy', 'fromkeys', 'get', 'items', 'keys', 'pop', 'popitem', 'setdefault',
'update', 'values']

Using the get() method

We saw earlier how to access a key-value pair of a dictionary using the notation of dict[key]. That is a very popular approach, but with one caveat. If the key does not exist, it raises a KeyError since the key does not exist (similarly, a list would raise an IndexError when accessing a nonexistent position)

>>> device
{'hostname': 'router1', 'vendor': 'juniper', 'os': '12.1'}
>>>
>>> print(device['model'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'model'

Using the get() method provides another approach that is arguably safer, unless you want to raise an exception. We will explore more about exceptions in “Embrace Failure with try/except”.

Let’s first look at an example using get() when the key exists.

>>> device.get('hostname')
'router1'

And now an example for when a key doesn’t exist:

>>> device.get('model')

As you can see from the preceding example, absolutely nothing is returned when the key isn’t in the dictionary, but it gets better than that. get() also allows the user to define a value to return when the key does not exist! Let’s take a look.

>>> device.get('model', False)
False
>>>
>>> device.get('model', 'DOES NOT EXIST')
'DOES NOT EXIST'
>>>
>>> device.get('hostname', 'DOES NOT EXIST')
'router1'

Pretty simple, right? You can see that the value to the right of the key is only returned if the key does not exist within the dictionary.

Using the keys() and values() methods

Dictionaries are a list of key-value pairs. Using the built-in methods called keys() and values(), you have the ability to access each objects individually.

In Python3, these methods do not return a simple list, but a dictionary view objects that can be iterated to yield the data, as we will see in “Using Loops in Python”. Notice that the order of the items is preserved.

>>> device.keys()
dict_keys(['hostname', 'vendor', 'os'])
>>>
>>> device.values()
dict_values(['router1', 'juniper', '12.1'])

However, these view objects can’t be access directly.

>>> device.keys()[0]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'dict_keys' object is not subscriptable

To access a specific object, we need to convert it to a list with list() before:

>>> list(device.keys())[0]
'hostname'

Using the pop() method

We first saw a built-in method called pop() earlier in the chapter when we were reviewing lists. It just so happens dictionaries also have a pop() method, and it’s used very similarly. Instead of passing the method an index value as we did with lists, we pass it a key.

>>> device
{'hostname': 'router1', 'vendor': 'juniper', 'os': '12.1'}
>>>
>>> device.pop('vendor')
'juniper'
>>>
>>> device
{'hostname': 'router1', 'os': '12.1'}

You can see from the example that pop() modifies the original object and returns the value that is being popped.

Using the update() method

There may come a time when you are extracting device information such as hostname, vendor, and OS and have it stored in a Python dictionary. And down the road you need to add or update it with another dictionary that has other attributes about a device.

The following shows two different dictionaries.

>>> device = {'hostname': 'router1', 'vendor': 'juniper', 'os': '12.1'}
>>>
>>> oper = dict(cpu='5%', memory='10%')
>>>
>>> oper
{'cpu': '5%', 'memory': '10%'}

The update() method can now be used to update one of the dictionaries, basically adding one dictionary to the other. Let’s add oper to device.

>>> device.update(oper)
>>>
>>> print(device)
{'hostname': 'router1', 'vendor': 'juniper', 'os': '12.1', 'cpu': '5%',
'memory': '10%'}

In case there is a match with some dictionary key, the update() method will update the old value with the new value from the new dictionary. In the next example, the vendor key from device is changed from the original juniper to arista because the data from new_vendor is updated on the reference dictionary.

>>> new_vendor = {'vendor': 'arista'}
>>>
>>> device.update(new_vendor)
>>>
>>> print(device)
{'hostname': 'router1', 'vendor': 'arista', 'os': '12.1', 'cpu': '5%',
'memory': '10%'}

Notice how nothing was returned with update(). Only the object being updated, or device in this case, was modified.

Using the items() method

When working with dictionaries, you’ll see items() used a lot, so it is extremely important to understand—​not to discount the other methods, of course!

We saw how to access individual values using get() and how to get a list of all the keys and values using the keys() and values() methods, respectively.

What about accessing a particular key-value pair of a given item at the same time, or iterating over all items? If you need to iterate (or loop) through a dictionary and simultaneously access keys and values, items() is a great tool for your tool belt.

Note

There is a formal introduction to loops later in this chapter, but because items() is commonly used with a for loop, we are showing an example with a for loop here. The important takeaway until loops are formally covered is that when using the for loop with items(), you can access a key and value of a given item at the same time.

The most basic example is looping through a dictionary with a for loop and printing the key and value for each item. Again, loops are covered later in the chapter, but this is meant just to give a basic introduction to items().

>>> for key, value in device.items():
...     print(f'{key}: {value}')
...
hostname: router1
vendor: arista
os: 12.1
cpu: 5%
memory: 10%

It’s worth pointing out that in the for loop, key and value are user defined and could have been anything, as you can see in the example that follows.

>>> for my_attribute, my_value, in device.items():
...     print(f'{my_attribute}: {my_value}')
...
hostname: router1
vendor: arista
os: 12.1
cpu: 5%
memory: 10%

We’ve now covered the major data types in Python. You should have a good understanding of how to work with strings, numbers, booleans, lists, and dictionaries. We’ll now provide a short introduction into two more data types, namely sets and tuples, that are a bit more advanced than the previous data types covered.

Learning About Python Sets and Tuples

The next two data types don’t necessarily need to be covered in an introduction to Python, but as we said at the beginning of the chapter, we wanted to include a quick summary of them for completeness. These data types are set and tuple.

If you understand lists, you’ll understand sets. Sets are a list of elements, but there can only be one of a given element in a set, and additionally elements cannot be indexed (or accessed by an index value like a list).

You can see that a set looks like a list, but is surrounded by set():

>>> vendors = set(['arista', 'cisco', 'arista', 'cisco', 'juniper', 'cisco'])

The preceding example shows a set being created with multiple elements that are the same. We used a similar example when we wanted to use the count() method for lists when we wanted to count how many of a given vendor exists. But what if you want to only know how many, and which, vendors exist in an environment? You can use a set.

>>> vendors = set(['arista', 'cisco', 'arista', 'cisco', 'juniper', 'cisco'])
>>>
>>> vendors
set(['cisco', 'juniper', 'arista'])
>>>
>>> len(vendors)
3

Notice how vendors only contains three elements.

The next example shows what happens when you try to access an element within a set. In order to access elements in a set, you must iterate through them, using a for loop as an example.

>>> vendors[0]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'set' object is not subscriptable

It is left as an exercise for the reader to explore the built-in methods for sets.

The tuple is an interesting data type and also best understood when compared to a list. It is like a list, but cannot be modified. We saw that lists are mutable, meaning that it is possible to update, extend, and modify them. Tuples, on the other hand, are immutable, and it is not possible to modify them once they’re created. Also, like lists, it’s possible to access individual elements of tuples.

>>> description = tuple(['ROUTER1', 'PORTLAND'])
>>>
>>> description
('ROUTER1', 'PORTLAND')
>>>
>>> print(description[0])
ROUTER1

And once the variable object description is created, there is no way to modify it. You cannot modify any of the elements or add new elements. This could help if you need to create an object and want to ensure no other function or user can modify it. The next example shows that you cannot modify a tuple and that a tuple has no methods such as update() or append().

>>> description[1] = 'trying to modify one'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
>>>
>>> dir(tuple)
# Omitted dunders
['count', 'index']

To help compare and contrast lists, tuples, and sets, we have put this high-level summary together:

Lists

are mutable, they can be modified, individual elements they can be accessed directly, and can have duplicate values.

Sets

are mutable, they can be modified, individual elements cannot be accessed directly, and they cannot have duplicate values.

Tuples

are immutable, they cannot be updated or modified once created, individual elements can be accessed directly, and they can have duplicate values.

We have learned about different Python types, each one with its own behavior and methods. So, in our programs we could like to validate the variable’s type before using it. isinstance() is a built-in function to identify it, not only for built-in types but also for custom class types.

>>> hostname = ''
>>> devices = []

>>> if isinstance(devices, list):
...     print('devices is a list')
...
devices is a list
>>>
>>> if isinstance(hostname, str):
...     print('hostname is a string')
...
hostname is a string

This concludes the section on data types. You should now have a good understanding of the data types covered, including strings, numbers, booleans, lists, dictionaries, sets, and tuples.

We’ll now shift gears a bit and jump into using conditionals (if then logic) in Python.

Adding Conditional Logic to Your Code

By now you should have a solid understanding of working with different types of objects. The beauty of programming comes into play when you start to use those objects by applying logic within your code, such as executing a task or creating an object when a particular condition is true (or not true!).

Conditionals are a key part of applying logic within your code, and understanding conditionals starts with understanding the if statement.

Let’s start with a basic example that checks the value of a string.

>>> hostname = 'NYC'
>>>
>>> if hostname == 'NYC':
...     print('The hostname is NYC')
...
The hostname is NYC

Even if you did not understand Python before starting this chapter, odds are you knew what was being done in the previous example. This is part of the value of working in Python—​it tries to be as human-readable as possible.

There are two things to take note of with regard to syntax when you’re working with an if statement. First, all if statements end with a colon (:). Second, the code that gets executed if your condition is true is part of an indented block of code—​this indentation should be four spaces, but technically does not matter. All that technically matters is that you are consistent.

Tip

Generally speaking, it is good practice to use a four-space indent when writing Python code. This is widely accepted by the Python community as the norm for writing idiomatic Python code. This makes code sharing and collaboration much easier.

The next example shows a full indented code block.

>>> if hostname == 'NYC':
...     print('This hostname is NYC')
...     print(len(hostname))
...     print('The End.')
...
This hostname is NYC
3
The End.

Now that you understand how to construct a basic if statement, let’s add to it.

What if you needed to do a check to see if the hostname was “NJ” in addition to “NYC”? To accomplish this, we introduce the else if statement, or elif.

>>> hostname = 'NJ'
>>>
>>> if hostname == 'NYC':
...     print('This hostname is NYC')
... elif hostname == 'NJ':
...     print('This hostname is NJ')
...
This hostname is NJ

It is very similar to the if statement in that it still needs to end with a colon and the associated code block to be executed must be indented. You should also be able to see that the elif statement must be aligned to the if statement.

What if NYC and NJ are the only valid hostnames, but now you need to execute a block of code if some other hostname is being used? This is where we use the else statement.

>>> hostname = 'DEN_CO'
>>>
>>> if hostname == 'NYC':
...     print('This hostname is NYC')
... elif hostname == 'NJ':
...     print('This hostname is NJ')
... else:
...     print('UNKNOWN HOSTNAME')
...
UNKNOWN HOSTNAME

Using else isn’t any different than if and elif. It needs a colon (:) and an indented code block underneath it to execute.

When Python executes conditional statements, the conditional block is exited as soon as there is a match. For example, if hostname was equal to 'NYC', there would be a match on the first line of the conditional block, the print statement print('This hostname is NYC') would be executed, and then the block would be exited (no other elif or else would be executed).

The following is an example of an error that is produced when there is an error with indentation. The example has extra spaces in front of elif that should not be there.

>>> if hostname == 'NYC':
...     print('This hostname is NYC')
...   elif hostname == 'NJ':
  File "<stdin>", line 3
    elif hostname == 'NJ':
                         ^
IndentationError: unindent does not match any outer indentation level

And the following is an example of an error produced with a missing colon.

>>> if hostname == 'NYC'
  File "<stdin>", line 1
    if hostname == 'NYC'
                       ^
SyntaxError: invalid syntax

The point is, even if you have a typo in your code when you’re just getting started, don’t worry; you’ll see pretty intuitive error messages.

You will continue to see conditionals in upcoming examples, including the next one, which introduces the concept of containment.

Understanding Containment

When we say containment, we are referring to the ability to check whether some object contains a specific element or object. Specifically, we’ll look at the usage of in building on what we just learned with conditionals.

Although this section only covers in, it should not be underestimated how powerful this feature of Python is.

If we use the variable called vendors that has been used in previous examples, how would you check to see if a particular vendor exists? One option is to loop through the entire list and compare the vendor you are looking for with each object. That’s definitely possible, but why not just use in?

Using containment is not only readable, but also simplifies the process for checking to see if an object has what you are looking for.

>>> vendors = ['arista', 'juniper', 'big_switch', 'cisco']
>>>
>>> 'arista' in vendors
True

You can see that the syntax is quite straightforward and a bool is returned. It’s worth mentioning that this syntax is another one of those expressions that is considered writing idiomatic Python code.

This can now be taken a step a further and added into a conditional statement.

>>> if 'arista' in vendors:
...     print('Arista is deployed.')
...
'Arista is deployed.'

The next example checks to see if part of a string is in another string compared to checking to see if an element is in a list. The examples show a basic boolean expression and then show using the expression in a conditional statement.

>>> version = "CSR1000V (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version 16.3.1"
>>>
>>> "16.3.1" in version
True
>>>
>>> if "16.3.1" in version:
...   print("Version is 16.3.1!!")
...
Version is 16.3.1!!

Note

The in operator is implemented by the contains dunder method. So, any object that implements this method can be used with the in operator.

As we previously stated, containment when combined with conditionals is a simple yet powerful way to check to see if an object or value exists within another object. In fact, when you’re just starting out, it is quite common to build really long and complex conditional statements, but what you really need is a more efficient way to evaluate the elements of a given object.

One such way is to use loops while working with objects such as lists and dictionaries. Using loops simplifies the process of working with these types of objects. This will become much clearer soon, as our next section formally introduces loops.

Using Loops in Python

We’ve finally made it to loops. As objects continue to grow, especially those that are much larger than our examples thus far, loops are absolutely required. Start to think about lists of devices, IP addresses, VLANs, and interfaces. We’ll need efficient ways to search data or perform the same operation on each element in a set of data (as examples). This is where loops begin to show their value.

We cover two main types of loops—​the for loop and while loop.

From the perspective of a network engineer who is looking at automating network devices and general infrastructure, you can get away with almost always using a for loop. Of course, it depends on exactly what you are doing, but generally speaking, for loops in Python are pretty awesome, so we’ll save them for last.

Understanding the while Loop

The general premise behind a while loop is that some set of code is executed while some condition is true. In the example that follows, the variable counter is set to 1 and then for as long as, or while, it is less than 5, the variable is printed, and then increased by 1.

The syntax required is similar to what we used when creating if-elif-else statements. The while statement is completed with a colon (:) and the code to be executed is also indented four spaces.

>>> counter = 1
>>>
>>> while counter < 5:
...     print(counter)
...     counter += 1
...
1
2
3
4

From an introduction perspective, this is all we are going to cover on the while loop, as we’ll be using the for loop in the majority of examples going forward.

Understanding the for Loop

for loops in Python are awesome because when you use them you are usually looping, or iterating, over a set of objects, like those found in a list, string, or dictionary. for loops in other programming languages require an index and increment value to always be specified, which is not the case in Python.

Let’s start by reviewing what is sometimes called a for-in or for-each loop, which is the more common type of for loop in Python.

As in the previous sections, we start by reviewing a few basic examples.

The first is to print each object within a list. You can see in the following example that the syntax is simple, and again, much like what we learned when using conditionals and the while loop. The first statement or beginning of the for loop needs to end with a colon (:) and the code to be executed must be indented.

>>> vendors = ['arista', 'juniper', 'cisco']
>>>
>>> for vendor in vendors:
...     print(f'VENDOR: {vendor}')
...
VENDOR:  arista
VENDOR:  juniper
VENDOR:  cisco

As mentioned earlier, this type of for loop is often called a for-in or for-each loop because you are iterating over each element in a given object.

In the example, the name of the object vendor is totally arbitrary and up to the user to define, and for each iteration, vendor is equal to that specific element. For example, in this example vendor equals arista during the first iteration, juniper in the second iteration, and so on.

To show that vendor can be named anything, let’s rename it to be network_vendor.

>>> for network_vendor in vendors:
...     print(f'VENDOR: {network_vendor}')
...
VENDOR:  arista
VENDOR:  juniper
VENDOR:  cisco

Let’s now combine a few of the things learned so far with containment, conditionals, and loops.

The next example defines a new list of vendors. One of them is a great company, but just not cut out to be a network vendor! Then it defines approved_vendors, which is basically the proper, or approved, vendors for a given customer. This example loops through the vendors to ensure they are all approved, and if not, prints a statement saying so to the terminal.

>>> vendors = ['arista', 'juniper', 'cisco', 'oreilly']
>>>
>>> approved_vendors = ['arista', 'juniper', 'cisco']
>>>
>>> for vendor in vendors:
...     if vendor not in approved_vendors:
...         print(f'NETWORK VENDOR NOT APPROVED: {vendor}')
...
NETWORK VENDOR NOT APPROVED:  oreilly

You can see that not can be used in conjunction with in, making it very powerful and easy to read what is happening.

Example 4-1 shows a more challenging example where we loop through a dictionary, while extracting data from another dictionary, and even get to use some built-in methods you learned earlier in this chapter.

Example 4-1. For Loop with items()

>>> COMMANDS = {
...     'description': 'description {}',
...     'speed': 'speed {}',
...     'duplex': 'duplex {}',
... }
>>> CONFIG_PARAMS = {
...     'description': 'auto description by Python',
...     'speed': '10000',
...     'duplex': 'auto'
... }
>>> commands_list = []
>>>
>>> for feature, value in CONFIG_PARAMS.items():
...     command = COMMANDS.get(feature).format(value)
...     commands_list.append(command)
...
>>> commands_list.insert(0, 'interface Eth1/1')
>>>
>>> print(commands_list)
['interface Eth1/1', 'description auto description by Python',
'speed 10000', 'duplex auto']

In Example 4-1, we start building a dictionary that stores CLI commands to configure certain features on a network device:

>>> print(COMMANDS)
{'duplex': 'duplex {}', 'speed': 'speed {}', 'description': 'description {}'}

We see that we have a dictionary that has three items (key-value pairs). Each item’s key is a network feature to configure, and each item’s value is the start of a command string that’ll configure that respective feature. These features include speed, duplex, and description. The values of the dictionary each have curly braces ({}) because we’ll be using the format() method of strings to insert variables.

After the COMMANDS dictionary is created, we create a second dictionary called CONFIG_PARAMS that will be used to dictate which commands will be executed and which value will be used for each command string defined in COMMANDS.

>>> CONFIG_PARAMS
{'description': 'auto description by Python', 'speed': '10000', 'duplex': 'auto'}

Then we use a for loop to iterate through CONFIG_PARAMS() using the items built-in method for dictionaries. As we iterate through, we’ll use the key from CONFIG_PARAMS and use that to get the proper value, or command string, from COMMANDS. This is possible because they were prebuilt using the same key structure. The command string is returned with curly braces, but as soon as it’s returned, we use the format() method to insert the proper value, which happens to be the value in CONFIG_PARAMS.

>>> commands_list = []
>>>
>>> for feature, value in CONFIG_PARAMS.items():
...     command = COMMANDS.get(feature).format(value)
...     commands_list.append(command)
...

Now we’ll walk through this in even more detail. Please take your time and even test this out yourself while on the Python interactive interpreter.

In the first line commands_list is creating an empty list []. This is required in order to append() to this list later on.

We then use the items() built-in method as we loop through CONFIG_PARAMS. This was covered very briefly earlier in the chapter, but items() is giving you, the network developer, access to both the key and value of a given key-value pair at the same time. This example iterates over three key-value pairs, namely description/auto description by Python, speed/10000, and duplex/auto.

During each iteration—​that is, for each key-value pair that is being referred to as the variables feature and value—a command is being pulled from the COMMANDS dictionary. If you recall, the get() method is used to get the value of a key-value pair when you specify the key. In the example, this key is the feature object. The value being returned is description {} for description, speed {} for speed, and duplex {} for duplex. As you can see, all of these objects being returned are strings, so then we are able to use the format() method to insert the value from CONFIG_PARAMS because we also saw earlier that multiple methods can be used together on the same line. Once the value is inserted, the command is appended to commands_list.

>>> commands_list.insert(0, 'interface Eth1/1')

Finally, after the commands are built, we insert() Eth1/1. This could have also been done first, with commands_list = ['interface Eth1/1'].

If you understand this example, you are at a really good point already with getting a grasp on Python!

You’ve now seen some of the most common types of for loops that allow you to iterate over lists and dictionaries. We’ll now take a look at another way to construct and use a for loop.

Using the enumerate() function

Occasionally, you may need to keep track of an index value as you loop through an object. We show this fairly briefly, since most examples that are reviewed are like the previous examples already covered.

enumerate() is used to enumerate the list and give an index value, and is often handy to determine the exact position of a given element.

The next example shows how to use enumerate() within a for loop. You’ll notice that the beginning part of the for loop looks like the dictionary examples, only unlike items(), which returns a key and value, enumerate() returns an index, starting at 0, and the object from the list that you are enumerating.

The example prints both the index and value to help you understand what it is doing:

>>> vendors = ['arista', 'juniper', 'cisco']
>>>
>>> for index, each in enumerate(vendors):
...     print(f'{index} {each}')
...
0 arista
1 juniper
2 cisco

Maybe you don’t need to print all indices and values out. Maybe you only need the index for a given vendor. This is shown in the next example.

>>> for index, each in enumerate(vendors):
...     if each == 'arista':
...         print(f'arista index is: {index}')
...
arista index is:  0

Flow control within loops

Loops can become a trap. In the enumerate example, we have started and completed a full iteration over the objects in vendors, even though we were only interested in the ones matching arista. In that case, this was not a big issue. But, what if vendors had thousands of elements? Should we continue iterating over the list when we have already found the item we were looking for?

Luckily, in Python, there is a break statement to stop a loop iteration at any point, for example when the purpose of the loop is fulfilled. In the following variation, we add a print() for each iteration and a break to stop the loop once we find “arista”, so we can see how the loop is stopped when a condition is met.

>>> vendors = ['arista', 'juniper', 'cisco']
>>>
>>> for index, each in enumerate(vendors):
...     print(index)
...     if each == 'arista':
...         print(f'arista index is: {index}')
...         break
...
0
arista index is: 0

Another useful statement is continue, to jump to the next iteration of the loop and skip any pending logic for the present iteration. In the next example we use continue to achieve the same goal of the example, printing the arista index.

>>> for index, each in enumerate(vendors):
...     print(index)
...     if each != 'arista':
...         continue
...     print(f'arista index is: {index}')
...
0
arista index is: 0
1
2

Obviously, in this case we are not breaking the loop (we are not using break) and we print all the indexes, but not the strings. We can observe that using continue makes the code easy to read by adding a condition at the beginning of the iteration to understand if it makes sense to continue with the iteration, or to move to the next one.

We’ve covered quite a bit of Python so far, from data types to conditionals to loops. However, we still haven’t covered how to efficiently reuse code through the use of functions. This is what we cover next.

Using Python Functions

Because you are reading this book, you probably at some point have heard of functions, but if not, don’t worry—​we have you covered! Functions are all about eliminating redundant and duplicate code and easily allowing for the reuse of code.

On a daily basis network engineers are configuring VLANs over and over again. And they are likely proud of how fast they can enter the same CLI commands into a network device or switch over and over. Writing a script with functions eliminates writing the same code over and over.

Let’s assume you need to create a few VLANs across a set of switches. Based on a device from Cisco or Arista, the commands required may look something this:

vlan 10
  name USERS
vlan 20
  name VOICE
vlan 30
  name WLAN

Imagine you need to configure 10, 20, or 50 devices with the same VLANs! It is very likely you would type in those six commands for as many devices as you have in your environment.

This is actually a perfect opportunity to create a function and write a small script. Since we haven’t covered scripts yet, we’ll still be working on the Python shell.

For our first example, we’ll start with a basic print() function and then come right back to the VLAN example.

>>> def print_vendor(net_vendor):
...     print(net_vendor)
...
>>>
>>> vendors = ['arista', 'juniper', 'cisco']
>>>
>>> for vendor in vendors:
...     print_vendor(vendor)
...
arista
juniper
cisco

In the preceding example, print_vendor() is a function that is created and defined using def. If you want to pass variables (parameters) into your function, you enclose them within parentheses next to the function name. This example is receiving one parameter and is referenced as vendor while in the function called print_vendor(). Like conditionals and loops, function declarations also end with a colon (:). Within the function, there is an indented code block that has a single statement—​it simply prints the parameter being received.

Once the function is created, it is ready to be immediately used, even while in the Python interpreter.

For this first example, we looped through vendors. During each iteration of the loop, we passed the object, which is a string of the vendor’s name, to print_vendor().

Notice how the variables have different names based on where they are being used, meaning that we are passing vendor, but it’s received and referenced as net_vendor from within the function. There is no requirement to have the variables use the same name while within the function, although it’ll work just fine if you choose to do it that way.

Since we now have an understanding of how to create a basic function, let’s return to the VLAN example.

We will create two functions to help automate VLAN provisioning.

The first function, called get_commands() (shown in Example 4-2), obtains the required commands to send to a network device. It accepts two parameters, one that is the VLAN ID using the parameter vlan and one that is the VLAN NAME using the parameter name.

Example 4-2. get_commands() Function

>>> def get_commands(vlan, name):
...     commands = []
...     commands.append(f'vlan {vlan}')
...     commands.append(f'name {name}')
...     return commands
...

In Example 4-3, the second function, called push_commands(), pushes the actual commands that were gathered from get_commands to a given list of devices. This function also accepts two parameters: device, which is the device to send the commands to, and commands, which is the list of commands to send. In reality, the push isn’t happening in this function, but rather it is printing commands to the terminal to simulate the command execution.

Example 4-3. push_commands() Function

>>> def push_commands(device, commands):
...     print(f'Connecting to device: {device}')
...     for cmd in commands:
...         print(f'Sending command: {cmd}')
...

In order to use these functions, we need two things: a list of devices to configure and the list of VLANs to send.

The list of devices to be configured is as follows:

>>> devices = ['switch1', 'switch2', 'switch3']

In order to create a single object to represent the VLANs, we have created a list of dictionaries. Each dictionary has two key-value pairs, one pair for the VLAN ID and one for the VLAN name.

>>> vlans = [{'id': '10', 'name': 'USERS'}, {'id': '20', 'name': 'VOICE'},
{'id': '30', 'name': 'WLAN'}]

If you recall, there is more than one way to create a dictionary. Any of those options could have been used here.

The next section of code shows one way to use these functions. The following code loops through the vlans list. Remember that each element in vlans is a dictionary. For each element, or dictionary, the id and name are obtained by way of the get() method. There are two print statements, and then the first function, get_commands(), is called—id and name are parameters that get sent to the function, and then a list of commands is returned and assigned to commands.

Once we have the commands for a given VLAN, they are executed on each device by looping through devices, as implemented in Example 4-4. In this process push_commands() is called for each device for each VLAN.

Example 4-4. Using Defined Functions

>>> for vlan in vlans:
...     vid = vlan.get('id')
...     name = vlan.get('name')
...     print(f'CONFIGURING VLAN: {vid}')
...     commands = get_commands(vid, name)
...     for device in devices:
...         push_commands(device, commands)
...         print()
...

And this is the output generated:

CONFIGURING VLAN: 10
Connecting to device: switch1
Sending command: vlan 10
Sending command: name USERS

Connecting to device: switch2
Sending command: vlan 10
Sending command: name USERS

Connecting to device: switch3
Sending command: vlan 10
Sending command: name USERS

CONFIGURING VLAN: 20
Connecting to device: switch1
Sending command: vlan 20
Sending command: name VOICE

Connecting to device: switch2
Sending command: vlan 20
Sending command: name VOICE

Connecting to device: switch3
Sending command: vlan 20
Sending command: name VOICE

CONFIGURING VLAN: 30
Connecting to device: switch1
Sending command: vlan 30
Sending command: name WLAN

Connecting to device: switch2
Sending command: vlan 30
Sending command: name WLAN

Connecting to device: switch3
Sending command: vlan 30
Sending command: name WLAN

Tip

Remember, not all functions require parameters, and not all functions return a value.

You should now have a basic understanding of creating and using functions, understanding how they are called and defined with and without parameters, and how it’s possible to call functions from within loops.

Next, we cover how to read and write data from files in Python.

Working with Files

This section is focused on showing you how to read and write data from files. Our focus is on the basics and to show enough that you’ll be able to easily pick up a complete Python book from O’Reilly to continue learning about working with files.

Reading from a File

For our example, we have a configuration snippet located in the same directory from where we entered the Python interpreter.

The filename is called vlans.cfg and it looks like this:

vlan 10
  name USERS
vlan 20
  name VOICE
vlan 30
  name WLAN
vlan 40
  name APP
vlan 50
  name WEB
vlan 60
  name DB

With just two lines in Python, we can open and read the file.

>>> vlans_file = open('vlans.cfg', 'r')
>>>
>>> vlans_file.read()
'vlan 10\n  name USERS\nvlan 20\n  name VOICE\nvlan 30\n
name WLAN\nvlan 40\n  name APP\nvlan 50\n  name WEB\nvlan 60\n
  name DB'
>>>
>>> vlans_file.close()

This example read in the full file as a complete str object by using the read() method for file objects.

The next example reads the file and stores each line as an element in a list by using the readlines() method for file objects.

>>> vlans_file = open('vlans.cfg', 'r')
>>>
>>> vlans_file.readlines()
['vlan 10\n', '  name USERS\n', 'vlan 20\n', '  name VOICE\n', 'vlan 30\n',
'  name WLAN\n', 'vlan 40\n', '  name APP\n', 'vlan 50\n', '  name WEB\n',
'vlan 60\n', '  name DB']
>>>
>>> vlans_file.close()

Let’s reopen the file, save the contents as a string, but then manipulate it, to store the VLANs as a dictionary similar to how we used the vlans object in the example from the section on functions.

>>> vlans_file = open('vlans.cfg', 'r')
>>>
>>> vlans_text = vlans_file.read()
>>>
>>> vlans_list = vlans_text.splitlines()
>>>
>>> vlans_list
['vlan 10', '  name USERS', 'vlan 20', '  name VOICE', 'vlan 30',
'  name WLAN', 'vlan 40', '  name APP', 'vlan 50', '  name WEB',
'vlan 60', '  name DB']

In this example, the file is read and the contents of the file are stored as a string in vlans_text. A built-in method for strings called splitlines() is used to create a list where each element in the list is each line within the file. This new list is called vlans_list and has a length equal to the number of commands that were in the file.

>>> vlans = []
>>> for item in vlans_list:
...     if 'vlan' in item:
...         temp = {}
...         id = item.strip().strip('vlan').strip()
...         temp['id'] = id
...     elif 'name' in item:
...         name = item.strip().strip('name').strip()
...         temp['name'] = name
...         vlans.append(temp)
...
>>>
>>> vlans
[{'id': '10', 'name': 'USERS'}, {'id': '20', 'name': 'VOICE'},
{'id': '30', 'name': 'WLAN'}, {'id': '40', 'name': 'APP'},
{'id': '50', 'name': 'WEB'}, {'id': '60', 'name': 'DB'}]
>>>
>>> vlans_file.close()

Once the list is created, it is iterated over within a for loop. The variable item is used to represent each element in the list as it’s being iterated over. In the first iteration, item is 'vlan 10'; in the second iteration, item is ' name users'; and so on. Within the for loop, a list of dictionaries is ultimately created where each element in the list is a dictionary with two key-value pairs: id and name. We accomplish this by using a temporary dictionary called temp, adding both key-value pairs to it, and then appending it to the final list only after appending the VLAN name. Per the following note, temp is reinitialized only when it finds the next VLAN.

Notice how strip() is being used. You can use strip() to strip not only whitespace, but also particular substrings within a string object. Additionally, we chained multiple methods together in a single Python statement. For example, with the value ' name WEB', when strip() is first used, it returns 'name WEB'. Then, we used strip('name'), which returns ' WEB', and then finally strip() to remove any whitespace that still remains to produce the final name of 'WEB'.

Note

The previous example is not the only way to perform an operation for reading in VLANs. That example assumed a VLAN ID and name for every VLAN, which is usually not the case, but is done this way for conveying certain concepts. It initialized temp only when “VLAN” was found, and only appended temp after the “name” was added (this would not work if a name did not exist for every VLAN and is a good use case for using Python error handling using try/except statements, as we cover in “Embrace Failure with try/except”).

Writing to a File

The next example shows how to write data to a file.

The vlans object that was created in the previous example is used here too.

>>> vlans
[{'id': '10', 'name': 'USERS'}, {'id': '20', 'name': 'VOICE'},
{'id': '30', 'name': 'WLAN'}, {'id': '40', 'name': 'APP'},
{'id': '50', 'name': 'WEB'}, {'id': '60', 'name': 'DB'}]

A few more VLANs are created before we try to write the VLANs to a new file.

>>> add_vlan = {'id': '70', 'name': 'MISC'}
>>> vlans.append(add_vlan)
>>>
>>> add_vlan = {'id': '80', 'name': 'HQ'}
>>> vlans.append(add_vlan)
>>>
>>> print(vlans)
[{'id': '10', 'name': 'USERS'}, {'id': '20', 'name': 'VOICE'},
{'id': '30', 'name': 'WLAN'}, {'id': '40', 'name': 'APP'},
{'id': '50', 'name': 'WEB'}, {'id': '60', 'name': 'DB'},
{'id': '70', 'name': 'MISC'}, {'id': '80', 'name': 'HQ'}]

There are now eight VLANS in the vlans list. Let’s write them to a new file, but keep the formatting the way it should be with proper spacing.

The first step is to open the new file. If the file doesn’t exist, which it doesn’t in our case, it’ll be created. You can see this in the first line of code that follows.

>>> write_file = open('vlans_new.cfg', 'w')

Once it is open, we’ll use the get() method again to extract the required VLAN values from each dictionary and then use the file method called write() to write the data to the file. Finally, the file is closed.

>>> for vlan in vlans:
...     id = vlan.get('id')
...     name = vlan.get('name')
...     write_file.write(f'vlan {id}\n')
...     write_file.write(f'  name {name}\n')
...
>>>
>>> write_file.close()

The previous code created the vlans_new.cfg file and generated the following contents in the file:

$ cat vlans_new.cfg
vlan 10
  name USERS
vlan 20
  name VOICE
vlan 30
  name WLAN
vlan 40
  name APP
vlan 50
  name WEB
vlan 60
  name DB
vlan 70
  name MISC
vlan 80
  name HQ

As you start to use file objects more, you may see some interesting things happen. For example, you may forget to close a file, and wonder why there is no data in the file that you know should have data!

Note

By default, what you are writing with the write() method is held in a buffer and only written to the file when the file is closed. This setting is configurable.

It’s also possible to use the with statement, a context manager, to help manage this process. In this example we see how we use it and can skip closing the because the context manager takes care of if automatically.

>>> with open('vlans_new.cfg', 'w') as write_file:
...     write_file.write('vlan 10\n')
...     write_file.write('  name TEST_VLAN\n')
...

Note

When you open a file with open('vlans.cfg', 'r'), you notice that two parameters are passed. The first is the name of the file including the relative or absolute path of the file. The second is the mode, which is an optional argument, but if not included, is the equivalent of read-only, which is the r mode. Other modes include w, which opens a file only for writing (if you’re using the name of a file that already exists, the contents are erased), a opens a file for appending, and r+ opens a file for reading and writing.

Since Python 3, file’s content is handled as bytes and encoded to text with an implicit UTF-8 encoding. If we want to change the encoding type, we can use the named argument encoding=ascii.

>>> vlans_file = open('vlans.cfg', 'r', encoding='ascii')

Everything in this chapter thus far has been using the dynamic Python interpreter. This showed how powerful the interpreter is for writing and testing new methods, functions, or particular sections of your code. No matter how great the interpreter is, however, we still need to be able to write programs and scripts that can run as a standalone entity. This is exactly what we cover next.

Creating Python Programs

Let’s take a look at how to build on what we’ve been doing on the Python shell and learn how to create and run a standalone Python script, or program. This section shows how to easily take what you’ve learned so far and create a script within just a few minutes.

Note

If you’re following along, feel free to use any text editor you are comfortable with, including, but not limited to vi, vim, Sublime Text, Notepad++, or even a full-blown Integrated Development Environment (IDE), such as PyCharm or VSCode.

Let’s look at a few examples.

Creating a Basic Python Script

The first step is to create a new Python file that ends with the .py extension. From the Linux terminal, create a new file by typing touch net_script.py and open it in your text editor. As expected, the file is completely empty.

The first script we’ll write simply prints text to the terminal. Add the following five lines of text to net_script.py in order to create a basic Python script.

#!/usr/bin/env python3

if __name__ == "__main__":
    print('^' * 30)
    print('HELLO NETWORK AUTOMATION!!!!!')
    print('^' * 30)

Now that the script is created, let’s execute it.

To execute a Python script from the Linux terminal, you use the python3 command. All you need to do is append the script name to the command as shown here.

$ python3 net_script.py
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
HELLO NETWORK AUTOMATION!!!!!
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

And that’s it! If you were following along, you just created a Python script. You might have noticed that everything under the if __name__ == "__main__": statement is the same as if you were on the Python interpreter.

Now we’ll take a look at the two unique statements that are optional, but recommended, when you are writing Python scripts. The first one is called the shebang. You may also recall that we first introduced the shebang in Chapter 3.

Comments in Python

Python code is usually easy to read, even without adding extra comments. However, there are situations when adding comments in your programs could save a lot of time for new developers jumping into the code (reviewing, fixing or extending), or even when you come back to this code after a while, and don’t fully remember the context around it.

This book isn’t about clean code best practices, but an accepted rule of thumb is that comments should be used to explain or clarify your intent when it is not obvious enough.

In Python you can use the # (known as a hashtag, number sign, or pound sign) for inline comments.

...
commands = []
# The order of the commands is relevant, do not change it
commands.append(f'vlan {vlan}')
# Due to the naming convention, the name of a vlan includes the vlan ID
commands.append(f'name {name}_{vlan}')
...

Python linters used to perform checks on the code can also act upon the text that comes after comments starting with #.

Now we’ll take a look at a special comment type, optional but recommended, when you are writing Python scripts, the shebang. You may also recall that we first introduced the shebang in Chapter 3.

Understanding the Shebang

The shebang is the first line in the script: #!/usr/bin/env python3. This is a special and unique line for Python programs.

It is the only line of code that uses the # as the first character other than comments. We will cover comments later in the chapter, but note for now that # is widely used for commenting in Python. The shebang happens to be the exception and also needs to be the first line in a Python program, when used.

The shebang instructs the system which Python interpreter to use to execute the program. Of course, this also assumes file permissions are correct for your program file (i.e., that the file is executable). If the shebang is not included, you must use the python keyword to execute the script, which we have in all of our examples anyway.

For example, if we had the following script in hello.py:

if __name__ == "__main__":
    print('Hello Network Automation!')

We could execute using the statement $ python3 hello.py, assuming the file was saved as hello.py. But we could not execute it using the statement $ ./hello.py. In order for the statement $ ./hello.py to be executed, we need to add the shebang to the program file because that’s how the system knows how to execute the script. Also, remember to add execute permissions to the file with chmod +x hello.py!

#!python3

if __name__ == "__main__":
    print('Hello Network Automation!')

The shebang as we have it, python3, defaults to using Python 3.8.12 on the system we’re using to write this book. We can check the exact Python version with which python3.

$ which python3
/usr/bin/python3

If your machine has different installations of Python, you can refer directly to the desired one in your shebang. For instance, using #!/usr/bin/python3.9 will use the Python 3.9 executable. However, it has the drawback of being inflexible (for instance this will cause problems in virtualenvs, covered in “Isolate your Dependencies with Virtualenv”).

A better approach is to use /usr/bin/env python3 which allows the external environment to influence which specific installation your script uses.

Now, that you understand the shebang, we can continue to discover another optional statement that will open the door to start creating our first Python scripts.

Migrating Code from the Python Interpreter to a Python Script

A common component of a Python script is the if __name__ == "__main__": statement. Based on the quotes, or lack thereof, you can see that __name__ is a variable and "__main__" is a string. When a Python file is executed as a standalone script, the variable name __name__ is automatically set to "__main__". Thus, whenever you do python3 <script>.py, everything underneath the if __name__ == "__main__": statement is executed.

At this point, you are probably thinking, when wouldn’t __name__ be equal to "__main__"? That is discussed in “Working with Python Modules”, but the short answer is: when you are importing particular objects from Python files, but not necessarily using those files as a standalone program.

This next example is the same example from “Using Python Functions”. The reason for this is to show you firsthand how easy it is to migrate from using the Python interpreter to writing a standalone Python script.

In Example 4-5, named push.py, you have a complete example.

Example 4-5. Script to Render Push Commands

#!/usr/bin/env python3

def get_commands(vlan, name):
    commands = []
    commands.append(f'vlan {vlan}')
    commands.append(f'name {name}')
    return commands

def push_commands(device, commands):
    print(f'Connecting to device: {device}')
    for cmd in commands:
        print(f'Sending command: {cmd}')

if __name__ == "__main__":

    devices = ['switch1', 'switch2', 'switch3']

    vlans = [
        {'id': '10', 'name': 'USERS'},
        {'id': '20', 'name': 'VOICE'},
        {'id': '30', 'name': 'WLAN'}
    ]

    for vlan in vlans:
        vid = vlan.get('id')
        name = vlan.get('name')
        print(f'CONFIGURING VLAN: {vid}')
        commands = get_commands(vid, name)
        for device in devices:
            push_commands(device, commands)
            print()

You can execute this script with the commands python3 push.py or ./push.py. The output you see is exactly the same output you saw when it was executed on the Python interpreter in Example 4-4.

Tip

Python offers the -i flag to be used when executing a script, but instead of exiting the script, it enters the interpreter, giving you access to all the objects built in the script—​this is great for testing.

Let’s see what happens when we run the script with the -i flag set.

$ python3 -i push.py
>>>
>>> print(devices)
['switch1', 'switch2', 'switch3']
>>>
>>> print(commands)
['vlan 30', 'name WLAN']

Notice how it executed, but then it dropped you right into the Python shell and you have access to those objects, at the end of the execution. Pretty cool, right?

If you were creating several scripts that performed various configuration changes on the network, we can intelligently assume that the function called push_commands() would be needed in almost all scripts. One option is to copy and paste the function in all the scripts. Clearly, that would not be optimal because if you needed to fix a bug in that function, you would need to make that change in all of the scripts.

Just like functions allow us to reuse code within a single script, there is a way to reuse and share code between scripts/programs. We do so by creating a Python module, which is what we’ll cover next as we continue to build on the previous example.

Working with Python Modules

We are going to continue to leverage the push.py file we just created in the previous section to better articulate how to work with a Python module. You can think of a module as a type of Python file that holds information, (e.g., Python objects), that can be used by other Python programs, but is not a standalone script or program itself.

For this example, we are going to enter back into the Python interpreter while in the same directory where the push.py file exists.

Let’s assume you need to generate a new list of commands to send to a new list of devices. You remember that you have this function called push_commands() in another file that already has the logic to push a list of commands to a given device. Rather than re-create the same function in your new program (or in the interpreter), you reuse the push_commands() function from within push.py. Let’s see how this is done.

While at the Python shell, we will type in import push and hit Enter. This imports all the objects within the push.py file.

>>> import push

Take a look at the imported objects by using dir(push).

>>> dir(push)
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__',
'__package__', '__spec__', 'get_commands', 'push_commands']

Just as we saw with the standard Python data types, push also has methods that start and end with underscores (dunders), but you should also notice the two objects called get_commands and push_commands, which are the functions from the push.py file!

If you recall, push_commands() requires two parameters. The first is a device and the second is a list of commands. Let’s now use push_commands() from the interpreter.

>>> device = 'router1'
>>> commands = ['interface Eth1/1', 'shutdown']
>>>
>>> push.push_commands(device, commands)
Connecting to device: router1
Sending command: interface Eth1/1
Sending command: shutdown

If you are importing multiple modules and there is a chance of overlap between function names, the method shown using import push is definitely a good option because it makes really easy to know where (in which module) the function exists. On the other hand, there are other options for importing objects.

One other option is to use from import. For our example, it would look like this: from push import push_commands. Notice in the following code, you can directly use push_commands() without referencing push.

>>> from push import push_commands
>>>
>>> push_commands(device, commands)
Connecting to device: router1
Sending command: interface Eth1/1
Sending command: shutdown

Tip

It’s recommended to make import statements as specific as possible and only import what’s used in your code. You should not use wildcard imports, such as from push import *. Statements like this load all objects from the module, potentially overloading and causing namespace conflicts with objects you’ve defined. And it also complicates troubleshooting, as it makes it difficult to decipher where an object was defined or came from.

Another option is to rename the object as you are importing it, using from import as. If you happen to not like the name of the object or think it is too long, you can rename it on import. It looks like this for our example:

>>> from push import push_commands as pc

Notice how easy it is to rename the object and make it something shorter and or more intuitive.

Let’s use it in an example.

>>> from push import push_commands as pc
>>>
>>> pc(device, commands)
Connecting to device: router1
Sending command: interface Eth1/1
Sending command: shutdown

Reusing functions is great, but it implies that the consumer of the code may not know the concrete purpose, or usage, of the imported function. To make reusability easier, the next step is to learn how to document your functions.

Documenting Functions

When you create functions that will be used by others, it’s convenient to document how your function should be used. In Python, you have docstrings that are added to functions, methods, and classes to describe them using triple quotes (""").

def get_commands(vlan, name):
    """Get commands to configure a VLAN.
    """

    commands = []
    commands.append(f'vlan {vlan}')
    commands.append(f'name {name}')
    return commands

You learned how to import a module, namely push.py. Let’s import it again now to see what happens when we use help on get_commands() since we now have a docstring configured.

>>> import push
>>>
>>> help(push.get_commands)

Help on function get_commands in module push:

get_commands(vlan, name)
    Get commands to configure a VLAN.
(END)

You see the whole function’s docstring when you use help. Additionally, you see information about the parameters and what data is returned if properly documented.

We’ve now added Args and Returns values to the docstring.

def get_commands(vlan, name):
    """Get commands to configure a VLAN.

    Args:
        vlan (int): vlan id
        name (str): name of the vlan

    Returns:
        List of commands is returned.
    """

    commands = []
    commands.append(f'vlan {vlan}')
    commands.append(f'name {name}')
    return commands

These are now displayed when the help() function is used to provide users of this function much more context on how to use it.

>>> import push
>>>
>>> help(push.get_commands)

Help on function get_commands in module push:

get_commands(vlan, name)
    Get commands to configure a VLAN.

    Args:
        vlan (int): vlan id
        name (str): name of the vlan

    Returns:
        List of commands is returned.
(END)

Tip

Some code editors provide contextual help to automatically render the docstring information when you are typing a function name.

By now you should understand not only how to create a script, but also how to create a Python module with functions (and other objects) and how to use those reusable objects in other scripts and programs.

Passing Arguments into a Python Script

In the last two sections, we looked at writing Python scripts and using Python modules. We have already mentioned the sys module that’s part of the Python standard library (i.e., comes with Python by default), but now we will understand how it allows us to easily pass in arguments from the command line into a Python script. Specifically we’re going to use an attribute (or variable) within the module called argv.

Let’s take a look at a basic script called send_command.py that only has a single print statement.

#!/usr/bin/env python3

import sys

if __name__ == "__main__":
    print(sys.argv)

Warning

sys.args is only available when running a Python script. It doesn’t exist in the Python Interpreter.

Now we’ll execute the script, passing in a few arguments simulating data we’d need to log in to a device and issue a show command.

ntc@ntc:~$ python3 send_command.py username password 10.1.10.10 "show version"
['send_command.py', 'username', 'password', '10.1.10.10', 'show version']

You should see that sys.argv is a list. In fact, it’s simply a list of strings of what we passed in from the Linux command line. This is a standard Python list that has elements matching the arguments passed in. You can also infer what really happened: Python did a split (str.split(" ")) on send-command.py username password 10.1.10.10 "show version" which created the list of five elements!

Finally, note that when you’re using sys.argv the first element is always the script name.

If you’d like, you can assign the value of sys.argv to an arbitrary variable to simplify working with the parameters passed in. In either case, you can extract values using the appropriate index values as shown:

#!/usr/bin/env python3

import sys

if __name__ == "__main__":
    args = sys.argv
    print(f"Username:  {args[1]}")
    print(f"Password:  {args[2]}")
    print(f"Device IP: {args[3]}")
    print(f"Command:   {args[4]}")

And if we execute this script:

ntc@ntc:~$ python3 send_command.py username password 10.1.10.10 "show version"
Username:  username
Password:  password
Device IP: 10.1.10.10
Command:   show version

Warning

When using sys.argv, you still need to account for error handling (at a minimum, check the length of the list). Additionally, the user of the script must know the precise order of the elements that need to be passed in.

In our example, if we call the script with less than the 4 expected arguments, we will hit and IndexError because we are accessing an index in a list that doesn’t exist.

ntc@ntc:~$ python3 send_command.py
Traceback (most recent call last):
  File "/your/path/examples/ch06-python/send_command.py", line 8, in <module>
    print(f"Username:  {args[1]}")
IndexError: list index out of range

For more advanced argument handling, you should look at the Python module called argparse that offers a very user-intuitive way of passing in arguments with “flags” and a built-in help menu. This is out of the scope of the book.

You can continue to build on this to perform more meaningful network tasks as you continue reading this book. For example, after reading [Link to Come], you’ll be able to pass parameters like this into a script that actually connects to a device using an API issuing a show command (or equivalent).

Using pip and Installing Python Packages

As you get started with Python, it’s likely you’re going to need to install other software written in Python. For example, you may want to test automating network devices with netmiko, a popular SSH client for Python, that we cover in [Link to Come]. It’s most common to distribute Python software, including netmiko, using the Python Package Index (PyPI), pronounced “pie-pie.” You can also browse and search the PyPI repository directly at https://pypi.python.org/pypi.

For any Python software hosted on PyPI such as netmiko, you can use the program called pip3 to install it on your machine directly from PyPI. pip3 is an installer that by default goes to PyPI, downloads the software, and installs it on your machine.

Note

pip3 is an updated version of original pip from Python 2, that installs packages for Python 3+.

Even though pip as a soft link could be also pointing to pip3, to avoid ambiguity, we use pip3 consistently in this book.

Using pip3 to install netmiko can be done with a single line on your Linux machine:

ntc@ntc:~$ sudo pip3 install netmiko
# output has been omitted

This will install netmiko in a system path (it’ll vary based on the OS being used).

By default, this will install the latest and greatest (stable release) of a given Python package. However, you may want to ensure you install a specific version—​this is helpful to ensure you don’t automatically install the next release without testing. This is referred to as pinning. You can pin your install to a specific release. In the next example, we show how you can pin the installation of netmiko to version 3.4.0.

ntc@ntc:~$ sudo pip3 install netmiko==3.4.0
# output has been omitted

You can use pip3 to upgrade versions of software too. For example, you may have installed version 3.4.0 of netmiko and when you’re ready, you can upgrade to the latest release by using the --upgrade or -U flag on the command line.

ntc@ntc:~$ sudo pip3 install netmiko --upgrade
# output has been omitted

It’s also common to need to install Python packages from source. That simply means getting the latest release—for example, from GitHub, a version control system that we cover in [Link to Come]. Maybe the software package on GitHub has a bug fix you need that is not yet published to PyPI. In this case, it’s quite common to perform a git clone—something that we also show you how to do in [Link to Come].

When you perform a clone of a Python project from GitHub, there is a good chance you’ll see two files in the root of the project: requirements.txt and setup.py. These can be used to install the Python package from source. The requirements file lists the requirements that are needed for the application to run. For example, here is the current requirements.txt for netmiko:

paramiko>=2.0.0
scp>=0.10.0
pyyaml
pyserial
textfsm

You can see that netmiko has five dependencies, commonly referred to as deps. You can also install these deps directly from PyPI using a single statement, again using the pip3 installer.

ntc@ntc:~$ sudo pip3 install -r requirements.txt
# output has been omitted

To completely install netmiko (from source) including the requirements, you can also use execute the setup.py file that you’d see in the same directory after performing the Git clone.

ntc@ntc:~$ sudo python3 setup.py install
# output has been omitted

By default, installing the software with setup.py will also install directly into a system path. Should you want to contribute back to a given project on GitHub, and actively develop on the project, you can also install the application directly from where the files exist (directory where you cloned the project).

ntc@ntc:~$ sudo python3 setup.py develop
# output has been omitted

This makes it such that the files in your local directory are the ones running netmiko, for our example. Otherwise, if you use the install option when running setup.py, you’ll need to modify the files in your system path to effect the local netmiko install (for troubleshooting as another example).

Tip

There is an even easier way to install a Python project as above but directly from GitHub, without cloning the repository locally. This is specially useful in case you don’t actually need to develop code on it, just use a specific version, from a branch or a commit (we will cover these concepts in detail in [Link to Come]). To target a specific branch, you can append @develop after the repository url. develop being the name of the branch, but you could point to a specific commit or tag. This is particularly useful when the library is still not released to PyPI or when there is some feature in a branch not released that you want to use.

ntc@ntc:~$ sudo pip3 install \
    git+https://github.com/ktbyers/netmiko.git@develop
# output has been omitted

Isolate your Dependencies with Virtualenv

By default, pip3 installs the packages into your global Python environment. This works well for single purpose applications (such as a container running one process) but your local development environment is going to become a nightmare sooner than later.

For example, when you work on your first Python project, you install a specific version of an external library. Then, a new project comes up, and one of its dependencies depends on exactly the same library you already had, but on a different version. Because pip3 installs the libraries globally, you have a problem.

But don’t worry, Python offers you a solution by creating virtual environments, called virtualenvs. It is simply an isolated Python environment that lives inside a folder, with all the dependencies of a specific project. Notice that this folder could be placed wherever you prefer, one common option is to put it directly in each project folder with name .venv.

To create a new Python virtualenv, you use python -m venv, followed by the folder name. This command will create a brand-new Python environment in this folder, where the packages will be installed.

ntc@ntc:~$ python3 -m venv .venv
ntc@ntc:~$ ls .venv
bin  include  lib  lib64  pyvenv.cfg

But, so far, you have only created the virtualenv, and you are still not using it because it’s not active_. Next step is to activate it. Run source on bin/activate within the virtualenv, and you will see how the path changes to (.venv)(the name of your virtualenv), indicating that now the Python environment is not the global one but the one that has been activated. Now, if you check your Python binary path it points to the one within the .venv folder.

ntc@ntc:~$ source .venv/bin/activate
(.venv) ntc@ntc:~$
(.venv) ntc@ntc:~$ which python
/home/ntc/.venv/bin/python

Do you remember installing netmiko before? If you check the installed packages within the virtualenv you won’t find it! Similarly, anything you install while the virtualenv is activated is only kept on this virtualenv and won’t affect the global environment or other virtualenvs.

(.venv) ntc@ntc:~$ pip3 list
Package    Version
---------- -------
pip        21.2.4
setuptools 57.5.0
wheel      0.37.1

Finally, you can go back to the global mode simply deactivating the virtualenv.

(.venv) ntc@ntc:~$ deactivate
ntc@ntc:~$ which python
/usr/bin/python

Tip

In Python there are several packages (Pipenv, Poetry, and others) that can help you with dependency managing, installation and/or packaging. We strongly recommend you give them a try, they are really helpful!

As a best practice, keep every Python project in a different virtual environment, it will make your life much easier because you will avoid the pain of inconsitent packages versions, inherited from each project direct package dependencies.

Now that we have learned about installing and importing others’ code, and also explored the Python built-in data types in “Understanding Python Data Types”, the next stop is to understand how to define our own data types: classes.

Understanding Python Classes

A class, in a programming language, contains data (attributes) and procedures (methods) that are related to its data. They are one of the key abstractions that enable Object-Oriented Programming (OOP) in Python (similarly in other programming languages). OOP is about modeling real-world things and how they interact together, and used correctly can make your code more readable, maintainable, and even more testable.

But, before going into defining your classes, you will need to understand how you could use others’ defined classes.

Using Classes

To get access to classes from other modules you use the same import statement you used to import functions in “Working with Python Modules”. In the next example, we import the class Device from our package vendors.cisco.device.

>>> from vendors.cisco.device import Device

Classes are just abstract representations of an object. However, to create a specific instance of an object, we must instantiate this class. This looks very similar to the way we call functions, and you can think of this syntax as calling a function which returns an instance of the class.

Then, you can instantiate an instance of the Device class, and get a Device object. Similar to functions, we pass some arguments, which are used to construct the actual instance. When we instantiate a class, we are actually calling the __init__ method, known as the constructor. The following snippet describes how we create three objects from Device, and save them in different variables.

>>> switch1 = Device(ip='10.1.1.1', username='cisco', password='cisco')
>>> switch2 = Device(ip='10.1.1.2', username='cisco', password='cisco')
>>> switch3 = Device(ip='10.1.1.3', username='cisco', password='cisco')

Notice that each variable is a separate instance of Device.

>>> type(switch1)
<class 'vendors.cisco.device.Device'>

Once the class object is initialized, you can start using its methods. This is just like using the built-in methods for the data types we learned about earlier in the chapter. The syntax is class_object.method.

>>> switch1.show('show version')

Tip

As a reminder, using method objects of a class is just like using the methods of the different data types such as strings, lists, and dictionaries.

If we executed show for switch2 and switch3, we would get the proper return data back from each device as expected, since each object is a different instance of Device, and the show() method will use the connection data from each instance.

Here is a brief example that shows the creation of two Device objects, and then, uses those objects to get the output of show hostname on each device. We can notice that the response from the show() method is stored in response variable, and then we extract the information accessing the data in the specific index “1”.

>>> switch1 = Device(ip='nycsw01', username='cisco', password='cisco')
>>> switch2 = Device(ip='nycsw02', username='cisco', password='cisco')
>>>
>>> for switch in [switch1, switch2]:
...     response = switch.show('show hostname')
...     print(response[1])
...
# output has been omitted

Now that we understand how to use classes, let’s create our first class!

Building your own classes

In the previous section we introduced the two basic elements we need to define when building our own class:

• A constructor method (init) to initialize the instance data (attributes)

• The methods that the objects of this class will expose

Taking as reference the example from “Using Classes”, in Example 4-6, we actually implement the Device class, starting by the init method.

Example 4-6. Building a Class

>>> class Device:
...     def __init__(self, ip, username, password):
...         self.host_ip = ip
...         self.user = username
...         self.pswd = password
...
>>> router = Device(ip='192.0.2.1', username='abc', password='123')
>>>
>>> router.host_ip
'192.0.2.1'
>>>
>>> router.__dict__
{'host_ip': '192.0.2.1', 'user': 'abc', 'pswd': '123'}

With the class statement we start the class definition (by convention, starting with capital letter, but that’s not mandatory).

And right after it, we define the __init__ constructor method (with def). Remember that this method is a dunder which will be called, magically, when the class is instantiated via Device(), returning an actual instance object. This new created object is referenced, within the method definition, by self. As we build more class methods, we will use self to represent the instance of the class. However, self is a convention, not an actual Python keyword. It is simply the first argument a class method takes. In Example 4-6, the __init__ method only stores input arguments into object attributes, but this method could contain a more complex logic.

After the class definition, we instantiate it and assign to the router variable, and then we access the data from one of its attributes with router.host_ip, similar to how we would call the methods that will expose later, but without the parenthesis.

Last, we expose a hidden attribute, __dict__, that returns a default conversion of all attributes as a dictionary. This could look like magic now because we have not explicitly defined it, but you will understand when we explain inheritance in Example 4-8.

Defining class methods is pretty similar to defining functions, but with the context reference to self that points to the instance object created when the class is instantiated. In Example 4-7, we define the show() method, in the Device class. This method will return a string, combining an internal class attribute, self.host_ip, and a method argument, command. Class metods have access to the class instance object, via the first argument (usually named self).

Example 4-7. Adding a Method to a Class

>>> class Device:
...     def __init__(self, ip, username, password):
...         # omitted for brevity
...
...     def show(self, command):
...         return f'{command} output from {self.host_ip}'
...
>>> router = Device(ip='192.0.2.1', username='abc', password='123')
>>>
>>> router.show('show version')
'show version output from 192.0.2.1'

Sometimes, you don’t need to completely create a brand-new class only to add a new attribute or a new method. In these cases it’s wiser to extend a class via inheritance. We will not go deep on this topic, but in the Example 4-8 you will understand how simple is to extend a class for your convenience while keeping the definition (attributes and methods) from the parent class.

Example 4-8. Extending a Class

>>> class Router(Device):
...     def disable_routing(self):
...         return f'Routing is disabled for {self.host_ip}'
...
>>>
>>> router = Router(ip='192.0.2.1', username='abc', password='123')
>>>
>>> router.show('show version')
'show version output from 192.0.2.1'
>>>
>>> router.disable_routing()
'Routing is disabled for 192.0.2.1'

In Example 4-8, first, we define the new class class Router like we did in Example 4-6, but taking an argument: Device. At this point, Router class is a copy of the original Device class, with a different name. An easy way to understand it is observing that initialization of the router object (router = Router(...)) is exactly the same as Example 4-7, even though we have not defined a new __init__ method.

After inheriting from a parent class, you can add new methods, as we do with def disable_routing(self), or you can add other class attributes. The same way you add a new method, you can overwrite a previously existing one by redefining it in the child class.

Moreover, when an object is created from a class which inherits from another class, the object belongs to all the classes. In our example, router is, at the same time, a Device and a Router, as we can check with the isinstance() function.

>>> isinstance(router, Router)
True
>>> isinstance(router, Device)
True

We have barely touched the surface of Python classes, giving you the basics to start using object-oriented programming in your programs. Next, you will learn how to make your code execution more robust.

Embrace Failure with try/except

When you write code, you try to build it to provide a seamless execution, without flow execution crashes. However, as the code grows in complexity, warranting an execution without errors becomes a big challenge due to the multiple cases to cover.

To facilitate errors handling, every coding language provides a pattern/mechanism. In Python, error handling is built around exceptions (we’ve already seen some of them, such as KeyError, in previous code examples) and the try/expect block.

Functions and methods communicate that something special has happened via raising an exception. Python has a raise statement to enable our code to signal to the caller of the function that a specific event has happened. Usually, this event is something wrong and must be handled by the caller, but its meaning is not necessary negative.

From the caller perspective, you can use the try/except block to handle exceptions. The try block is where the code is executed and exceptions may be raised, and the except captures specific exceptions, in order to handle them appropriately.

When exploring the dictionaries access, we saw that a KeyError exception is raised when we access a key that doesn’t exist. Now we try to not break the execution using try/except.

>>> device = {'hostname': 'router1', 'vendor': 'juniper', 'os': '12.1'}
>>>
>>> device["random_key"]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'random_key'

Here, we observe that when we access the “random_key” without the try/except block, a Traceback is printed. A Traceback is a report containing where and how your code failed and broke the execution, and is useful to diagnose the issue and debug your code. To better understand their content, we should read them bottom-up.

1. KeyError: Exception name raised

2. 'random_key': The error message, in this case the key that we were accessing

3. File "<stdin>", line 1, in <module>: Calls involved, pointing out to the file and the line of code. In this case, because we were using the interactive mode, stdin is showed.

Now, using the try/except block, we make our code ready to handle the KeyError exception gracefully, simply printing a message and resuming the code execution.

>>> try:
...     device["random_key"]
... except KeyError:
...     print('The key is not present')
...
The key is not present

Tip

Python has a built-in class Exception which is the parent of all the other exceptions. Indeed, the KeyError exception inherits from Exception.

Thus, you can use the Exception as a wildcard to catch all Python exceptions, and protect your code execution against any unexpected failure. However, use this with caution because it could mask real programming errors.

Exception is yet another class, with some special dunders that make it special. This means that all we learned about classes still apply here, so you can create your own exceptions from the base Exception class, overwriting methods or adding extra attributes, and catch them in try/except blocks. When you build a Python library, the exception classes have a relevant role too, besides the other functions and classes you expose.

Before closing the Python chapter, we want to explain a slightly more advanced concept to introduce you to a new programming paradigm, parallel code execution.

Parallelize your Python Programs

By default, Python programs follow a serial execution pattern. Starting from an entry point, our code is executed one task after the other, so the second task doesn’t start until the previous task is completed. This is not a problem when you have a small program or when the time spent in each task is not significant. But, what if one task takes too long, simply waiting for an external process to complete? Should our precious CPU stay empty until we get the response? Or, having multiple CPU cores, why not distribute processing load across them, instead of leaving them in the bench?

The solution to this challenge is performing multiple operations at the “same time”. Figure 4-1 illustrates the different approach between running your code/tasks in serial or running them in parallel.

Figure 4-1. Serial vs Parallel execution

The most common network automation use-cases, interacting with APIs or remote network devices, are I/O-bound operations (entering a blocking state when waiting for the input/output data). To parallelize these operations, you can use Threads. A Thread is a separate flow of execution that runs at the same time.

Python supports different implementation options for parallelization. We don’t pretend to cover all of them in this book. We use the ThreadPoolExecutor utility to exemplify the idea.

Note

For CPU-bound operations, where it’s necessary to spread the load over multiple processes (instead of multiple threads on the same process), you can use the ProcessPoolExecutor, similarly to how we use ThreadPoolExecutor here.

First, we simulate a long-standing I/O task adding a blocking call,time.sleep(5), in the get_commands function, the same function we defined in Example 4-2.

>>> import time
>>>
>>> def get_commands(vlan, name):
...     time.sleep(5)
...     commands = []
...     commands.append(f'vlan {vlan}')
...     commands.append(f'name {name}')
...     return commands
...

As expected, in Example 4-9, if we execute the get_commands() function in a three iteration loop, the total execution time is 15 seconds, because each function is called when the previous ends. We wrap the loop into a run_task() function, where we print the time spent substracting the initial and final time, using the time.time() function, that returns the time in seconds since the epoch.

Example 4-9. Serial Execution

>>> def run_task(vlans):
...     start_time = time.time()
...     for vlan in vlans:
...         result = get_commands(vlan=vlan['id'], name=vlan['name'])
...         print(result)
...     print(f'Time spent: {time.time() - start_time} seconds')
...
>>> vlans = [{'id': '10', 'name': 'USERS'}, {'id': '20', 'name': 'VOICE'},
{'id': '30', 'name': 'WLAN'}]
>>>
>>> run_task(vlans)
['vlan 10', 'name USERS']
['vlan 20', 'name VOICE']
['vlan 30', 'name WLAN']
Time spent: 15.008316278457642 seconds

Now, in Example 4-10, we introduce the ThreadPoolExecutor class, imported from concurrent.futures, to implement concurrency running the same get_commands() function. We use it as a context manager with the executor variable, automatically cleaning up threads upon completion.

Example 4-10. Parallel Execution

>>> import concurrent.futures
>>>
>>> with concurrent.futures.ThreadPoolExecutor() as executor:
...     start_time = time.time()
...     tasks = []
...     for vlan in vlans:
...         tasks.append(
...             executor.submit(
...                 get_commands, vlan=vlan['id'], name=vlan['name']
...             )
...         )
...     for task in concurrent.futures.as_completed(tasks):
...         print(task.result())
...     print(f'Time spent: {time.time() - start_time} seconds')
...
['vlan 10', 'name USERS']
['vlan 20', 'name VOICE']
['vlan 30', 'name WLAN']
Time spent: 5.001068830490112 seconds

As expected, now, the time spent is 5 seconds, instead of 15 seconds, because the three tasks were executed concurrently. Imagine what this means when you have to execute a 30 seconds operation for hundreds of devices. We hope this sounds appealing to you.

The code in Example 4-10 can look a bit overwhelming, but we can summarize it in two main blocks, one for each for loop. The first one appends future tasks jobs (using the executor), and the second, iterates over these tasks, getting the results when completed.

This example helps to illustrate the potential of using parallelization in Python. However, getting into the details of this code, or exploring other options, such as AsyncIO, would go out of the scope of this book.

However, this paradigm introduces some collateral effects that must be taken into account when building your code:

• When running code in parallel, by default, you don’t control the ending order of the tasks. If your tasks execution has some dependencies, some kind of coordination (for instance, using semaphores) would be needed.

• Depending on the parallelization approach, your functions would need to be rewritten, slightly. For instance, to adopt asynchronous behavior with the async/await pattern.

• Multiple tasks accessing the same memory object could cause data inconsistency. For instance, two tasks updating the value the same variable at the exact same time. This can be solved using a locking mechanism, preventing this concurrent access.

• Increasing the number of parallel tasks, running concurrent connections, could overload the target endpoint if it’s not ready to handle so many sessions. A good approach would be to check the concurrency impact in the target endpoint, progressively, so as to not completely knock down it.

In spite of the previous considerations, running code in parallel, with the appropriate design, can save a lot of time in some use cases.

With this last superpower, we end the Python chapter, where you have hopefully gained the basic knowledge to start building your Python applications.

Summary

This chapter provided a grassroots introduction to Python for network engineers. We covered foundational concepts such as working with data types, conditionals, loops, functions, and files, and even how to create a Python module that allows you to reuse the same code in different Python programs/scripts. We also got into more advanced topics such as classes and error handling, ending up with a quick glance about parallel execution. All together should help you as a reference as you continue on with your Python and network automation journey.

In Chapter 6 we introduce you to different data formats such as YAML, JSON, and XML, and in that process, we also build on what was covered in this chapter. For example, you’ll take what you learned and start to use Python modules to simplify the process of working with these data types, and also see the direct correlation between YAML, JSON, and Python dictionaries.

But, Python is not the only coding language used in network automation. Next, in Chapter 5, you will learn about another popular coding language for automation, and you will start noticing what are their commonalities and differences.

Now, you are ready to start coding your first Python program, and as a bonus point, we give you a simple trick to learn more about Python design philosophy. Try import this and see what happens!