Chapter 4. Junos PyEZ

This chapter looks at Junos PyEZ, another automation tool that allows remote procedure calls to be invoked on Junos devices. PyEZ is a Python library that enables administration and automation of Junos devices. It is an open source project maintained and supported by Juniper Networks with contributions from the user community. The Junos PyEZ project is hosted on GitHub at https://github.com/Juniper/py-junos-eznc.

The PyEZ APIs provide a “mini-framework” that can be used to solve both simple and complex automation tasks. PyEZ can be used from the interactive Python shell to quickly perform simple tasks on one or more Junos devices, or incorporated in full-blown Python scripts of varying complexity to automate the management and administration of an entire network of Junos devices. The first several sections of this chapter demonstrate entering commands at the interactive Python shell indicated by the >>> prompt. “A PyEZ Example” will demonstrate a full Python script utilizing the PyEZ library.

PyEZ provides an abstraction layer built on top of the NETCONF protocol covered in Chapter 2. It does not require direct NETCONF interaction, but utilizes the vendor-agnostic ncclient¹ library for its NETCONF transport. Because the PyEZ library utilizes NETCONF for its remote procedure calls, it can be used with all currently supported Junos software versions and Junos platforms.

Like the Junos RESTful API covered in Chapter 3, the PyEZ library supports invoking individual Junos RPCs and retrieving the resulting responses. However, unlike the Junos RESTful API service, PyEZ also offers optional features for further simplifying common automation tasks. One example of these features happens automatically upon initial connection to a Junos device with the PyEZ library. By default, the PyEZ library gathers basic information about the device and stores this information in a Python dictionary. This dictionary can be easily accessed with the facts attribute, covered in “Facts Gathering”.

Other abstraction features of the PyEZ library involve dealing with RPC responses. Rather than returning RPC responses as XML or JSON strings, PyEZ utilizes the lxml library to directly return XML-specific Python data structures. It may also be combined with the jxmlease library, introduced in “Using Structured Data in Python”, to simplify parsing these XML-specific data structures into a native Python data structure. Tables and views are another tool for mapping RPC responses into native Python data structures. PyEZ provides several predefined tables and views for common RPCs and also allows users to define their own to extract information from any Junos RPC.

For configuration, PyEZ supports changes in text, XML, or set formats. In addition, it includes an engine for combining user-supplied values with templates to dynamically produce device, customer, or feature-specific configuration changes.

Installation

Running a Python script that utilizes the Junos PyEZ library requires that Junos PyEZ be installed on the automation host executing the script. Junos PyEZ is dependent upon Python and several system libraries whose installation is operating system–specific. These dependencies are also subject to change with new PyEZ releases. Therefore, this book does not attempt to cover the procedure for installing the prerequisite system software. Instead, refer to the “Installing PyEZ” section of the release-specific “Junos PyEZ Developer Guide” found on Juniper’s Junos PyEZ landing page for information on installing the required system software on common operating systems.

Note

At the time of this writing, PyEZ does not support Python 3.x. Ensure Python 2.7 (or a later 2.x release) is installed on your system, and the system is properly configured to use Python 2.x for any scripts that utilize the Junos PyEZ library.

Once the appropriate system libraries are installed on the host system, PyPI, the Python Package Index, can be used to install Junos PyEZ and its dependent Python libraries. Simply execute the command pip install junos-eznc at a root shell prompt to install the latest stable release of Junos PyEZ. You can also use the command pip install git+https://github.com/Juniper/py-junos-eznc.git² to install the latest development version of Junos PyEZ directly from the GitHub repository.

Warning

PyPI automatically installs prerequisite Python modules when you install Junos PyEZ with the pip install command. One of those prerequisite Python modules is lxml. As part of its installation, the lxml module insists on downloading and compiling the libxml2 C library from source code, even if your system already has a functional libxml2 library installed. This requirement means that your host must have all of the necessary tools installed to compile libxml2. These include a C compiler, a make program, and a zlib compression library including header files (usually found in a zlib-dev package). If the host is missing any of these required tools, the Junos PyEZ installation may fail.

Device Connectivity

As explained in the chapter introduction, PyEZ uses NETCONF to communicate with a remote Junos device. Therefore, PyEZ requires NETCONF to be enabled on the target device. All currently supported releases of the Junos software support the NETCONF protocol over an SSH transport, but this NETCONF-over-SSH service is not enabled by default. The minimal configuration necessary to enable the NETCONF-over-SSH service is:

set system services netconf ssh

The NETCONF-over-SSH service listens on TCP port 830, by default, and can operate over both IPv4 and IPv6. The following Junos CLI output demonstrates configuring the NETCONF-over-SSH service and verifying that the service is indeed listening on TCP port 830:

user@r0> configure 
Entering configuration mode

[edit]
user@r0# set system services netconf ssh 

[edit]
user@r0# commit and-quit 
commit complete
Exiting configuration mode

user@r0> show system connections inet | match 830 
tcp4       0      0  *.830                    *.*                       LISTEN

user@r0> show system connections inet6 | match 830   
tcp6       0      0  *.830                    *.*                       LISTEN

Note

When the SSH service is enabled with the set system services ssh configuration, it is also possible to reach the NETCONF-over-SSH service on TCP port 22. However, the set system services netconf ssh configuration is preferred because the PyEZ library attempts to connect to the NETCONF-over-SSH port (TCP port 830) by default.

Once the NETCONF-over-SSH service has been configured, the device is ready to use with Junos PyEZ.

Creating a Device Instance

The PyEZ library provides a jnpr.junos.Device class to represent a Junos device being accessed by the PyEZ library. The first step in using the library is to instantiate an instance of this class with the parameters specific to the Junos device:

user@h0$ python 
Python 2.7.9 (default, Mar  1 2015, 12:57:24) 
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from jnpr.junos import Device 
>>> r0 = Device(host='r0',user='user',password='user123')

: First invoke the Python interactive shell with the python command.
Note
The exact command to invoke the Python interactive shell is specific to the operating system and Python installation of the automation host. Use the command appropriate for your specific environment.
: Before the jnpr.junos.Device class can be used, it must first be imported. This line imports the jnpr.junos Python package and copies the Device name into the local namespace, allowing you to simply reference Device(). An alternative syntax is import jnpr.junos. Again, this imports the jnpr.junos Python package, but it does not copy the Device name into the local namespace. Using this syntax requires you to reference the class as an attribute of jnpr.junos using the syntax jnpr.junos.Device().
: Calling the Device class object with the Device() syntax creates a new instance object. This instance represents a specific NETCONF-over-SSH session to a specific Junos device. In this case, the instance object is assigned to the variable named r0. There’s nothing special about the name r0, and any valid Python variable name could be used in its place. The parameters to the Device() call set the initial values of the instance’s attributes. In this example, the host, user, and password parameters have been set to appropriate values for the Junos device with a hostname of r0.

While it is typical to specify the host, user, and password parameters, the only mandatory argument to the Device() call is the host parameter. The host information can also be specified as the first (unnamed) argument to the Device() call. Table 4-1 details the Device() parameters and their default values.

Table 4-1. Parameters to the jnpr.junos.Device class
Parameter	Description	Default value
`host`	A hostname, domain name, or IPv4 or IPv6 address on which the Junos device is running the NETCONF-over-SSH service. If a hostname or domain name is used, it must resolve to an IPv4 or IPv6 address. This parameter can alternatively be specified as the first unnamed argument to the `Device()` call.	`None` (Must be specified by the caller.)
`port`	The TCP port on which the NETCONF-over-SSH service is reachable. If `set system` `services` `ssh` is configured on the Junos device, you can reach the NETCONF service by specifying the argument `port = 22` to the `Device()` call.	`830`
`user`	The username used to log in to the Junos device. As discussed in “Authentication and Authorization”, RPC execution is controlled by the authorization configuration of this user account on the Junos device.	The value of the `$USER` environment variable for the account running the Python script on the automation host. This is usually set to the username of the user executing the Python script. The default behavior can be useful if the username on the automation host and the Junos device are the same. At the Python interactive shell, you can confirm the value of the `$USER` environment variable with: >>> `import os` >>> `print(os.environ['USER'])` user
`password`	The password used to authenticate the user on the Junos device. If SSH keys are being used, this value is used as the passphrase to unlock the SSH private key. Otherwise, this value is used for password authentication.	`None` (A password is not needed for an SSH key with an empty passphrase.)
`gather_facts`	A Boolean that indicates whether or not basic information is gathered from the device upon initial connection with the `open()` instance method. See “Facts Gathering” for details.	`True` (Facts are gathered.)
`auto_probe`	This setting attempts to check if the TCP port specified by `port` is reachable by first attempting a simple TCP connection to that port. Only after this test connection succeeds is the real NETCONF-over-SSH connection attempted. The `auto_probe` value is an integer defining the number of seconds to attempt to make this test TCP connection to `port` before timing out. If the value is `0`, auto probing is disabled and no test TCP connection is attempted. (In this case, the real NETCONF-over-SSH connection is attempted immediately.)	`0` (Autoprobing is disabled. This value is inherited from the `Device` class’s `auto_probe` attribute upon instantiation. The `auto_probe` value can be changed for all device instances by setting `Device.auto_probe = value` before instantiating any device instances.)
`ssh_config`	The path, on the automation host, to an SSH client configuration file used for the SSH connection. The SSH client configuration file can be used to control many aspects of the SSH connection. For Unix-like automation hosts, use `man ssh_config` for details on the available settings.	`~/.ssh/config` (The `~` is expanded to the user’s home directory. If no SSH configuration file is found, the system-wide defaults are used.)
`ssh_private_key_file`	The path, on the automation host, to an SSH private key file used with SSH key authentication.	None (If specified, the SSH key files configured in the `ssh_config` file are used.)
`normalize`	A Boolean to indicate whether or not the XML responses from this device should have whitespace normalized. See “Response Normalization” for more information.	`False` (Whitespace is not normalized.)

Parallel Execution

The ncclient library used by PyEZ is synchronous and blocking. This behavior means when an RPC function is invoked on a Device instance, execution of your script is paused until the resulting RPC response, or an exception, is received from the device and returned in the function’s response. It is possible to execute RPCs on multiple Junos devices in parallel by having your script create multiple threads or fork multiple processes. Each process can instantiate a Device instance to a different Junos device and execute RPCs on that Junos device. In fact, it is even possible to have parallel processes, each with different Device instances to the same Junos device. Each Device instance would have its own NETCONF connection to the Junos device and each connection would handle RPC execution independently.

Making the Connection

Creating an instance of the Device class does not initiate a NETCONF connection to the instance. While the instance has been initialized with all the necessary information to make a NETCONF connection, the actual connection is only made when the open() instance method is invoked. A NETCONF connection to the r0 device instance created in the previous example is initiated with:

>>> r0.open()
Device(r0)

The open() method returns the device instance, as shown by the Device(r0) output³ in our example. This return value is unneeded in this example, because it points to the same object as the variable r0. However, returning the device instance has a purpose—this enables an alternative syntax that chains the Device() and open() invocations on a single line. Here’s that alternative syntax:

>>> r0 = Device(host='r0',user='user',password='user123').open()
>>>

Regardless of whether the open() call is invoked on an existing instance variable or chained with the Device() instantiation, the NETCONF connection is opened using the information stored in the device instance attributes. These attributes include the auto_probe, host, port, user, and password parameters, as well as the SSH configuration information. All of these attributes are set when the instance is instantiated by passing arguments to the Device() call, or using the default values detailed in Table 4-1. However, it is possible to override the auto_probe, gather_facts, and normalize settings by specifying parameters to the open() call. If any of these parameters are supplied as arguments to the open() call, they override the device instance’s attributes.

Authentication and Authorization

Unlike the RESTful API service, PyEZ does not require each API call to be authenticated. Instead, authentication happens only when the NETCONF-over-SSH session is initiated by the open() call. The authentication of the NETCONF-over-SSH service can use either SSH public key or password authentication methods.

The SSH authentication methods used, and the order in which they are tried, is dependent upon the client’s SSH configuration. A typical client SSH configuration attempts to use public key authentication first, and then tries password authentication. In either case, the Junos device uses the standard Junos authentication system specified at the [edit system] hierarchy level of the Junos configuration to permit or deny the authentication. In other words, NETCONF-over-SSH sessions are authenticated exactly the same as standard SSH connections to the CLI.

When the public key authentication method is used, the SSH public key must be configured on the Junos device at the [edit system login user username authentication] level of the configuration hierarchy. Depending on the type of the SSH public key, the key is specified using the ssh-dsa, ssh-ecdsa, ssh-ed25519, or ssh-rsa configuration statement. In addition to having the public key configured on the Junos device, the corresponding SSH private key must exist on the automation host. In order to be used by PyEZ, this SSH private key must be in the default location, or in the path specified by the ssh_private_key_file device instance attribute. If the private SSH key requires a passphrase, the open() method attempts to use the instance’s password attribute as the passphrase. If the private SSH key’s passphrase is empty, then the instance’s password attribute does not need to be set.

Here’s an example Junos configuration snippet with a public key configured for the user user:

user@r0> show configuration system login user user 
uid 2001;
class super-user;
authentication {
    ssh-dsa "ssh-dss AAAAB3Nza ...output trimmed... SJCS9boQ== user@h0";
}

The corresponding private SSH key is in the default location, ~/.ssh/id_dsa, on the automation host:

user@h0$ cat ~/.ssh/id_dsa
-----BEGIN DSA PRIVATE KEY-----
MIIBugIBAAKBgQCqBuyGycDhwXmEDb3hXcEfSpD5gaomT91ojlcsSPVtoj773KqZ
...ouput trimmed...
PO+bL6L74rIKIi3cfFk=
-----END DSA PRIVATE KEY-----

This private key has an empty passphrase,⁴ allowing a NETCONF connection to r0 without specifying a password:

user@h0$ python
Python 2.7.9 (default, Mar  1 2015, 12:57:24) 
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from jnpr.junos import Device
>>> r0 = Device('r0')
>>> r0.open()
Device(r0)
>>>

In this example, the user parameter was also omitted from the Device() call. This works because the $USER environment variable is user, which matches the remote username on r0.

When password authentication is used, the Junos device can use RADIUS, TACACS+, or a local password database to verify the password. The exact authentication order is determined by the [edit system authentication-order] configuration hierarchy. Again, this is exactly the same as authenticating an SSH connection to the CLI. The configured authentication-order may try multiple password databases before ultimately permitting or denying the authentication attempt.

Authentication occurs when the NETCONF-over-SSH session is established, but authorization occurs for each remote procedure call. NETCONF authorization uses the exact same mechanism and configuration as CLI authorization. Authorization to execute a specific RPC is determined by mapping a user, or potentially a template user in the case of RADIUS/TACACS+ authentication, to a login class. In turn, the login class specifies a set of permissions that determine if an RPC call is permitted or denied. If needed, you can refer back to “Authentication and Authorization” for a refresher on the relationship between the Junos configuration and RPC authorization.

Connection Exceptions

The Junos PyEZ library defines several exceptions that may be raised when the open() call fails. These exceptions are subclasses of the more general jnpr.junos.exception.ConnectError class, although a generic jnpr.junos.exception.ConnectError exception may also be raised for other unrecognized connection errors. Table 4-2 provides a list and descriptions of these exceptions and of the situation in which each exception is raised.

Table 4-2. Possible exceptions raised by the open() method
Exception	Description
`jnpr.junos.exception.ProbeError`	Raised if `auto_probe` is nonzero and the probe action fails. This generally indicates the `host` device is not reachable on TCP port `port`. This could indicate a name resolution issue, an IP reachability issue, or device misconfiguration. While this exception does not provide a very specific indication of the problem, it is an indication that the NETCONF connection cannot succeed.
`jnpr.junos.exception.ConnectUnknownHostError`	Raised if the hostname specified in the `host` attribute cannot be resolved to an IP address. This generally indicates a problem with the Domain Name System (DNS) resolution process.
`jnpr.junos.exception.ConnectTimeoutError`	Raised if there is an IP reachability problem connecting to `port` on `host`. This indicates no response was received from the Junos device. Possible causes include an incorrect IP address, firewall filtering, or general routing issues between the automation host and the Junos device.
`jnpr.junos.exception.ConnectRefusedError`	Raised if the Junos device rejects the TCP connection to `port`. This might indicate the NETCONF-over-SSH service is not configured on `port`, or it might indicate the maximum number of simultaneous connections has been reached.
`jnpr.junos.exception.ConnectAuthError`	Raised if the `user` or `password` is incorrect and the Junos device rejects the authentication attempt.
`jnpr.junos.exception.ConnectNotMasterError`	Raised if the connection is made to a non-master routing engine on a multi-RE device.
`jnpr.junos.exception.ConnectError`	A generic exception raised if the `ncclient` library raises an unrecognized exception during the connection process.

In order to catch and handle these exceptions gracefully, wrap the open() method in a try/except block. Here’s a simple example that prints a message when one of these exceptions is encountered. In the output, a jnpr.junos.exception.ConnectAuthError is intentionally raised by specifying an incorrect password:

>>> from jnpr.junos import Device
>>> import jnpr.junos.exception
>>> r0 = Device(host='r0',user='user',password='badpass')
>>> try:
...     r0.open()
... except jnpr.junos.exception.ConnectError as err:
...     print('Error: ' + repr(err))
... 
Error: ConnectAuthError(r0)
>>>

The jnpr.junos.exception module must be imported by the import statement before the specific jnpr.junos.exception.ConnectError exception is referenced in the except statement.

Note

Because all of these exceptions are subclasses of the jnpr.junos.exception.ConnectError class, specifying the single exception catches all of the possible exceptions raised by open().

Facts Gathering

By default, the PyEZ library gathers basic information about the Junos device during the open() call. PyEZ refers to this basic information as facts, and the information is accessible via the device instance’s facts dictionary attribute. This example uses the pprint module to “pretty print” the facts dictionary gathered during the r0.open() call:

>>> r0.open()
Device(r0)
>>> from pprint import pprint
>>> pprint(r0.facts)
{'2RE': False,
 'HOME': '/var/home/user',
 'RE0': {'last_reboot_reason': 'Router rebooted after a normal shutdown.',
         'mastership_state': 'master',
         'model': 'RE-VMX',
         'status': 'OK',
         'up_time': '6 days, 7 hours, 36 minutes, 44 seconds'},
 'domain': 'example.com',
 'fqdn': 'r0.example.com',
 'hostname': 'r0',
 'ifd_style': 'CLASSIC',
 'master': 'RE0',
 'model': 'MX960',
 'personality': 'MX',
 'serialnumber': 'VMX5868',
 'switch_style': 'BRIDGE_DOMAIN',
 'vc_capable': False,
 'version': '15.1R1.9',
 'version_RE0': '15.1R1.9',
 'version_info': junos.version_info(major=(15, 1), type=R, minor=1, build=9)}
>>>

These facts can be easily tested in a script to implement logic based on the facts. This code shows a simple example based on the model key in the facts dictionary. The model key indicates the model of the Junos device:

if r0.facts['model'] == 'MX480':
    # Handle the MX480 case
    ... MX480 code ...
else:
    # Handle the case of other models
    ... Other models code ...

The example follows one code path if the Junos device is an MX480 and another code path for all other models of Junos device.

While facts gathering is enabled by default, it can be disabled by setting the optional gather_facts parameter to False in the Device() or open() calls. You might want to disable facts gathering to speed the initial connection or if there is an unexpected problem gathering facts on a device.

If facts gathering is disabled during the initial NETCONF connection, it can still be initiated later by invoking the facts_refresh() instance method. As the name implies, the facts_refresh() method can also be used to refresh an existing facts dictionary with the latest information from the Junos device.

Closing the Connection

The close() method cleanly shuts down the NETCONF session that was initiated by the successful execution of the open() method. The close() method should be invoked when you have finished making RPC calls to the device instance:

>>> r0.close()

Calling the close() method does not destroy the device instance; it just closes its NETCONF session. Calling an RPC after an instance has been closed causes a jnpr.junos.exception.ConnectClosedError exception to be raised, as shown in this example:

>>> r0.close()
>>> version_info = r0.rpc.get_software_information()
Traceback (most recent call last):
...ouput trimmed...
jnpr.junos.exception.ConnectClosedError: ConnectClosedError(r0)
>>> r0.open()
Device(r0)
>>> version_info = r0.rpc.get_software_information()
>>> r0.close()

The example also shows how an instance that has previously been closed can be reopened by invoking the open() instance method again. The RPC is executed, and the instance is again closed.

RPC Execution

This section begins with one of PyEZ’s low-level capabilities. PyEZ allows the user to invoke Junos XML RPCs using simple Python statements. While PyEZ offers higher-level abstractions, like tables and views (covered in “Operational Tables and Views”), it still simplifies the process of invoking Junos XML RPCs and parsing
the corresponding XML responses. PyEZ does not require formatting the RPC
as XML, and it does not require directly interacting with NETCONF.

RPC on Demand

Once a device instance has been created and the open() method has been invoked to initiate the NETCONF session, the device instance’s rpc property can be used to execute an RPC. Each Junos XML RPC can be invoked as a method of the rpc property. The general format of these method calls is:

device_instance_variable.rpc.rpc_method_name()

For example, the get-route-summary-information RPC is invoked on the already opened r0 device instance with:

>>> route_summary_info = r0.rpc.get_route_summary_information()

Note

The XML RPC name is get-route-summary-information, while the method name is get_route_summary_information. The method name is derived from the XML RPC name by simply substituting underscores for hyphens. This substitution is required because Python’s naming rules don’t allow method names to include dashes.

The response from the get-route-summary-information RPC is returned by the r0.rpc.get_route_summary_information() method and stored in the route_summary_info variable. For now, don’t worry about the content of the response. RPC response content will be covered in detail in “RPC Responses”.

PyEZ refers to this concept as “RPC on Demand” because the PyEZ library does not contain a method for each Junos XML RPC. Having an actual method for each Junos XML RPC would require thousands of methods. In addition, to avoid these methods being perpetually out of sync with the device’s capabilities, PyEZ would have to be tightly coupled to the Junos platform and version. With RPC on Demand, there is no tight coupling; new features are added to each platform with each Junos release and the existing version of PyEZ can instantly access those RPCs.

Instead, RPC on Demand is implemented using the concept of metaprogramming. Each RPC method is generated, and executed, dynamically at the time it is invoked. PyEZ users aren’t required to understand the details of how this metaprogramming is implemented, but it is helpful to understand the general concept.

Because these RPC methods are generated “on demand,” an RPC method can be invoked for every Junos XML RPC on any Junos platform. If a Junos device supports a particular XML RPC, that XML RPC can always be invoked from PyEZ using RPC on Demand.

The corresponding aspect of this design paradigm is that the PyEZ library cannot know in advance if an XML RPC is valid. Any RPC method name will be first converted to its corresponding XML RPC name by substituting hyphens for underscores, then wrapped in proper XML elements and sent over the NETCONF session to the device. Only after the NETCONF response is received from the device can an error be discovered and an exception raised. “RPC Exceptions” details this case and other possible exceptions.

Warning

PyEZ also provides a cli() device instance method, but this method is primarily intended for debugging and should be avoided in PyEZ scripts. By default, this method returns a text string containing the normal CLI output:

>>> response = r0.cli('show system uptime')
/usr/local/lib/python2.7/dist-packages/jnpr/junos/devi...
  warnings.warn("CLI command is for debug use only!", ...)
>>> print response

Current time: 2015-07-13 14:02:00 PDT
Time Source:  NTP CLOCK 
System booted: 2015-07-13 07:42:46 PDT (06:19:14 ago)
Protocols started: 2015-07-13 07:42:46 PDT (06:19:14 ago)
Last configured: 2015-07-13 08:07:26 PDT (05:54:34 ago) by root
 2:02PM  up 6:19, 1 users, load averages: 0.59, 0.41, 0.33

>>>

The cli() method is prone to all of the “gotchas” of traditional “screen scraping” network automation. Neither the command nor the response is in a structured data format. Both are subject to errors in parsing or even changes between Junos versions. For this reason, you should avoid using this method in any production automation effort.

RPC Discovery with PyEZ

“Discovering Operational RPC Syntax” explained how to discover the RPC for a given CLI command by using the | display xml rpc pipe command. This technique can easily be used while developing a PyEZ script by opening a separate terminal window to the device’s CLI. However, PyEZ also offers an RPC discovery method that can be used from within Python itself: the display_xml_rpc() instance method. You can fetch the RPC for the show system users CLI command with:

>>> r0.display_xml_rpc('show system users')
<Element get-system-users-information at 0x7fa22b1294d0>

Notice this command returned an object, not a string. By default, all PyEZ RPC responses are returned as an lxml.etree.Element object. “RPC Responses” provides much more detail on using these responses. For now, it’s sufficient to know that an lxml.etree.Element object represents an XML element. In this case, it’s the first XML element inside the response’s <rpc-reply> tag. You can view the actual RPC name by invoking the tag property on this response object. This example imports the etree module from the lxml library and fetches the XML tag by appending the tag property to the previous example:

>>> from lxml import etree
>>> r0.display_xml_rpc('show system users').tag
'get-system-users-information'

You can take this one step further by using the built in replace() string method to substitute hyphens with underscores:

>>> r0.display_xml_rpc('show system users').tag.replace('-','_')
'get_system_users_information'

This recipe can be used to discover the RPC name for any Junos CLI command, but does not provide details on an RPC’s parameters. Passing the response from the display_xml_rpc() method to the etree.dump() function displays the full XML response (including RPC parameters):

>>> from lxml import etree
>>> etree.dump(
...      r0.display_xml_rpc(
...        'show route protocol isis 10.0.15.0/24 active-path'
...      )
... )
<get-route-information>
    <destination>10.0.15.0/24</destination>
    <active-path/>
    <protocol>isis</protocol>
</get-route-information>

RPC Parameters

As you saw with the Junos RESTful API service in “Adding Parameters to RPCs”, some Junos XML RPCs support optional parameters that limit or alter the XML response. When describing the RPC in XML format, these parameters appear as nested XML tags within the RPC name’s XML tag. As an example, here’s the equivalent XML RPC for the show route protocol isis 10.0.15.0/24 active-path CLI command:

user@r0> show route protocol isis 10.0.15.0/24 active-path | display xml rpc
<rpc-reply xmlns:junos="http://xml.juniper.net/junos/15.1R1/junos">
    <rpc>
        <get-route-information>
                <destination>10.0.15.0/24</destination>
                <active-path/>
                <protocol>isis</protocol>
        </get-route-information>
    </rpc>
    <cli>
        <banner></banner>
    </cli>
</rpc-reply>

In the preceding output, you can see three XML elements are nested within the RPC’s <get-route-information> XML element. The <destination> and <protocol> tags have values in their content, while the <active-path/> tag is an empty XML element. The PyEZ RPC mechanism allows the user to specify these parameters without having to format them in XML. The parameters are simply specified as keyword arguments to the RPC methods introduced in “RPC on Demand”. An equivalent PyEZ method invocation for the show route protocol isis 10.0.15.0/24 active-path CLI command is:

>>> isis_route = r0.rpc.get_route_information(protocol='isis',
...                                           destination='10.0.15.0/24',
...                                           active_path=True)

The XML tags become keyword arguments, and any XML content becomes the value of the keyword. Just like in the RPC method names, underscores are substituted for any hyphens in the XML tags. So, the <active-path/> tag becomes the active_path keyword. If the RPC parameter does not require a value, as is the case with <active-path/>, the keyword’s value should be set to True.

RPC Timeout

By default, an RPC method call will time out if the complete response is not received from the Junos device within 30 seconds. While this default is reasonable for most RPC calls, there are times when you may need to override the default. One example of an RPC that might take longer than 30 seconds to execute is get-flow-session-information on an SRX-Series firewall that has many thousands of active security flows. Another example is get-route-information on a router carrying the full Internet routing table (500K+ routes and growing).

When you encounter one of these situations, your first question should always be: “Do I really need all of this information?” Maybe you only need the routes from a particular peer AS, or the security flows with a TCP destination port of 443. In those cases, have the RPC filter the output directly on the device by passing appropriate parameters to the RPC as discussed in the previous section.

There are, however, other times where the situation is unavoidable. The best example may be installing a Junos software package with the request-package-add RPC. In this case, you need to override the 30-second timeout with a longer timeout. In other situations, you might want to shorten the default timeout to have your script more quickly detect an unresponsive device. Regardless of whether you need to increase or decrease the 30-second timeout, there are two ways of achieving this objective.

The first method for changing the RPC timeout is to simply set the device instance’s timeout property. Setting the timeout property affects all RPCs invoked on this device handle. Here is an example using the existing r0 instance variable:

>>> r0.timeout
30
>>> r0.timeout = 10
>>> r0.timeout
10

The preceding example confirms the default 30-second timeout by displaying the value of the r0.timeout attribute. The timeout is then set to 10 seconds, and the final line confirms the value is now 10 seconds.

The second method for modifying the RPC timeout only changes the timeout value for a single RPC. Future RPCs will continue to use the default value, or the value set by assigning the device instance’s timeout attribute. This second method is accomplished by passing a dev_timeout keyword parameter to the RPC’s method. Here’s an example of this method:

>>> summary_info = r0.rpc.get_route_summary_information()
>>> bgp_routes = r0.rpc.get_route_information(dev_timeout = 180,
...                                           protocol='bgp')
>>> isis_routes = r0.rpc.get_route_information(protocol='isis')

In this example, the get-route-summary-information RPC is executed with the default 30-second timeout. BGP routes are then gathered with a specific 180-second timeout. Finally, ISIS routes are gathered with the default 30-second timeout.

RPC Exceptions

In addition to the connection-related exceptions discussed in “Connection Exceptions”, the Junos PyEZ library defines several exceptions that may be raised when an RPC on Demand method fails. Table 4-3 provides a list and descriptions of these exceptions and of the situation in which each exception is raised.

Table 4-3. Possible exceptions raised by RPC on Demand methods
Exception	Description
`jnpr.junos.exception.ConnectClosedError`	Raised if the underlying NETCONF session closed unexpectedly before the RPC method was invoked. This might happen because of an RE switchover on the target device, a network reachability problem between the automation host and the target device, or a failure of the target device itself, or because you forgot to call the `open()` method.
`jnpr.junos.exception.RpcTimeoutError`	Raised if the underlying NETCONF session is connected, and the RPC was successfully sent to the device, but a response is not received from the device within the RPC timeout period (as discussed in the previous section).
`jnpr.junos.exception.PermissionError`	Raised if authorization, as discussed in “Authentication and Authorization”, does not permit the RPC being executed.
`jnpr.junos.exception.RpcError`	Raised if there are `<xnm:error>` or `<xnm:warning>` elements present in the RPC response. This exception is also raised if there are NETCONF `<rpc-error>` elements present in the response or if the `ncclient` library raises an unrecognized exception.

Exactly how each of these exceptions should be handled can be very specific to your particular automation requirements. For example, if the response from a particular RPC is required for further processing, a jnpr.junos.exception.PermissionError exception would indicate an error that prevents further processing (at least for that specific device instance). In this case, printing the error and exiting the script (or continuing to the next device instance in the loop) might be appropriate. However, there might be other cases where the user provides input that selects the RPC to be executed. In this case, you might want to gracefully handle the jnpr.junos.exception.PermissionError exception by printing the error and asking the user to specify a different RPC.

A common exception handling requirement is attempting to reopen the NETCONF connection when a jnpr.junos.exception.ConnectClosedError exception is received. Because this exception may indicate a transient condition, it may be possible to gracefully recover from the condition. However, this exception may also indicate a more persistent problem. Attempting to reopen the NETCONF connection with no constraints could lead to an infinite loop.

The following Python code snippet illustrates an algorithm for attempting to reopen the NETCONF connection, and reexecute the failed RPC, a limited number of times. Not only does it illustrate a common requirement, but it provides a framework for additional more specific exception handling. By simply adding another except block, additional graceful error handling could be added for other exceptions:

import jnpr.junos.exception 
from time import sleep

MAX_ATTEMPTS = 3 
WAIT_BEFORE_RECONNECT = 10

# Assumes r0 already exists and is a connected device instance

for attempt in range(MAX_ATTEMPTS): 
    try: 
        routes = r0.rpc.get_route_information()
    except jnpr.junos.exception.ConnectClosedError: 
        sleep(WAIT_BEFORE_RECONNECT) 
        try: r0.open()
        except jnpr.junos.exception.ConnectError: pass
    else: 
        # Success. No exception was raised.
        # break will skip the for loop's else.
        break
else: 
    # Max attempts exceeded. All attempts have failed.
    # Re-raise most recent exception from last attempt.
    raise

# ... continue with the rest of script if RPC succeeded ...

Importing the PyEZ exception module is required before specific PyEZ exceptions can be caught by an except statement. Import the sleep() function from the time module into the local namespace.

MAX_ATTEMPTS is used as a constant to indicate the maximum number of times an RPC should be retried. For this example, an RPC will be retried a maximum of three times. WAIT_BEFORE_RECONNECT is used as a constant to indicate the number of seconds to wait after an RPC failure before attempting to reopen the NETCONF connection. This allows up to 30 seconds (WAIT_BEFORE_RECONNECT * MAX_ATTEMPTS) for a transient condition to be resolved.

The for loop will attempt to execute the RPC up to MAX_ATTEMPTS times.

The try statement marks a block of statements that will be executed until an exception occurs. The RPC on Demand feature executes the get-route-information RPC inside this try block. The result of the RPC is stored in the routes variable. This RPC may succeed without an exception, or it may raise any of the exceptions listed in Table 4-3.

If an exception is raised by the statement in the try block, the exception is checked to see if it matches a jnpr.junos.exception.ConnectClosedError exception. If the exception matches, the statements in this except block are run. If the exception doesn’t match this except block, Python’s default exception handler will stop the program execution and print the error. Additional exception statements could be added to handle other possible exceptions. Each except block will be tested in order until a match is found, or the list of exceptions is exhausted.

Immediately attempting to reconnect a closed NETCONF connection is likely to fail. Instead, waiting some period of time provides an opportunity for a transient condition (RE switchover, network reconvergence, etc.) to pass. The sleep statement suspends execution of the script for WAIT_BEFORE_RECONNECT seconds. Execution then proceeds to the next line. The open() method is wrapped in a try block to catch any exceptions that will occur if the call fails because the underlying network condition still exists. The except: pass statement will catch and ignore all jnpr.junos.exception.ConnectError exceptions raised by r0.open().

Note

All exceptions raised by r0.open() are listed in Table 4-2. These exceptions are all subclasses of jnpr.junos.exception.ConnectError. Therefore, catching this one exception class catches all exceptions raised by r0.open().

If the NETCONF connection is still not open, this condition will be caught as another jnpr.junos.exception.ConnectClosedError exception during the next attempt to execute the RPC.

The else block of a try/except/else compound statement is executed only if no exceptions were raised by the try block. In other words, the else block is executed only if r0.rpc.get_route_information() executed successfully. The break statement will exit the for loop when the RPC succeeds. Exiting a for loop with a break statement skips the else clause of the loop.

The else clause of a Python for loop is executed only if the loop exits normally. In this code, “exiting normally” means all attempts have been exhausted (attempt > MAX_ATTEMPTS) and the RPC execution has failed (raised an exception) on each attempt. If all attempts have failed, the most recent exception is raised.

This example assumes the r0 variable has already been assigned a device instance, as described in “Creating a Device Instance”, and already has a NETCONF connection opened, as described in “Making the Connection”. The example also purposely treats any exception raised by the RPC other than jnpr.junos.exception.ConnectClosedError as a fatal error. Finally, if the jnpr.junos.exception.ConnectClosedError exception persists for more than MAX_ATTEMPTS, the most recent exception will be propagated to Python’s default error handling mechanism. The most recent exception will likely be one of the exceptions from Table 4-2, raised by the final attempt to reopen the connection.

RPC Responses

Now that you’ve seen how to invoke Junos XML RPCs using simple Python statements, this section covers what to do with the resulting responses. PyEZ offers multiple ways to parse a response into Python data structures, and also offers an optional mechanism for removing extraneous whitespace from the response. This section covers each of the features for controlling RPC responses.

lxml Elements

The default response to a NETCONF XML RPC is a string representing an XML document. As you’ve seen, this RPC response is the same as the output displayed by the CLI when the | display xml modifier is appended to the equivalent CLI command. For example:

user@r0> show system users | display xml
<rpc-reply xmlns:junos="http://xml.juniper.net/junos/15.1R1/junos">
    <system-users-information xmlns="http://xml.juniper.net/junos/15.1R1/junos">
        <uptime-information>
            <date-time junos:seconds="1436915514">4:11PM</date-time>
            <up-time junos:seconds="116940">1 day,  8:29</up-time>
            <active-user-count junos:format="4 users">4</active-user-count>
            <load-average-1>0.56</load-average-1>
            <load-average-5>0.43</load-average-5>
            <load-average-15>0.36</load-average-15>
            <user-table>
                <user-entry>
                    <user>root</user>
                    <tty>u0</tty>
                    <from>-</from>
                    <login-time junos:seconds="1436897214">11:06AM</login-time>
                    <idle-time junos:seconds="60">1</idle-time>
                    <command>cli</command>
                </user-entry>
                <user-entry>
                    <user>foo</user>
                    <tty>pts/0</tty>
                    <from>172.29.104.149</from>
                    <login-time junos:seconds="1436884614">7:36AM</login-time>
                    <idle-time junos:seconds="30900">8:35</idle-time>
                    <command>-cli (cli)</command>
                </user-entry>
                <user-entry>
                    <user>bar</user>
                    <tty>pts/1</tty>
                    <from>172.29.104.149</from>
                    <login-time junos:seconds="1436884614">7:36AM</login-time>
                    <idle-time junos:seconds="30900">8:35</idle-time>
                    <command>-cli (cli)</command>
                </user-entry>
                <user-entry>
                    <user>user</user>
                    <tty>pts/2</tty>
                    <from>172.29.104.149</from>
                    <login-time junos:seconds="1436884614">7:36AM</login-time>
                    <idle-time junos:seconds="0">-</idle-time>
                    <command>-cli (cli)</command>
                </user-entry>
            </user-table>
        </uptime-information>
    </system-users-information>
    <cli>
        <banner></banner>
    </cli>
</rpc-reply>

Rather than returning this XML document string directly to the user, as you saw with the RESTful API service in “Formatting HTTP Responses”, the PyEZ library uses the lxml library to parse the XML document and return the already parsed response. The response is an lxml.etree.Element object rooted at the first child element of the <rpc-reply> element. In the case of the show system users command, or the equivalent get-system-users-information RPC, the first child element of the <rpc-reply> element is the <system-users-information> element. This is demonstrated by displaying the tag attribute of the get-system-users-information RPC response:

>>> response = r0.rpc.get_system_users_information(normalize=True)
>>> response.tag
'system-users-information'

Note

This example passed the argument normalize=True to the r0.rpc.get_system_users_information() method. Response normalization is covered in detail in “Response Normalization”. For the purposes of these examples, simply make sure you also include the normalize argument. Failing to do so will cause some of the following examples to return different results or fail completely.

Each lxml.etree.Element object has links to parent, child, and sibling lxml.etree.Element objects, which form a tree representing the parsed XML response. For debugging purposes, the lxml.etree.dump() function can be used to dump the XML text of the response (albeit without the pretty formatting of the Junos CLI):

>>> from lxml import etree
>>> etree.dump(response)
<system-users-information>
<uptime-information>
...ouput trimmed...
</uptime-information>
</system-users-information>

>>>

While lxml.etree.Element objects may seem more complicated than a data structure composed of native Python lists and dicts, lxml.etree.Element objects do offer a robust set of APIs for selecting and extracting portions of the RPC response. Many of these APIs use an XPath expression to match a subtree, or subtrees, from the response. You were introduced to XPath expressions in “Accessing XML data with XPath”. The following sidebar augments the previous introduction with specific information on the subset of XPath expressions available within lxml. For a more detailed study of XPath in general, check out the W3Schools XPath Tutorial.

Using XPath with lxml

Several lxml methods support a subset of the full XPath expression syntax. Table 4-4 details the most common XPath syntax supported by the lxml find(), findall(), and findtext() methods. In addition, lxml offers an xpath() method, which supports the full XPath syntax. The xpath() method is outside the scope of this book. Refer to the lxml library documentation for details on xpath() and other more advanced XPath usage with lxml.

Table 4-4. XPath syntax supported by the lxml library
Syntax	Description
`.`	Selects the current node. While XPath expressions are relative by default, prefixing a path with `./` gives a more explicit indication that the XPath is relative to the current node.
`..`	Selects the parent node. Allows traversing up a subtree to select a sibling subtree.
`//`	Selects all subtree elements. Allows finding an element regardless of its position in the XML.
`tag`	Selects all child elements with tag `tag`.
`*`	Selects all child elements.
`[tag]`	Selects all elements that have an immediate child with tag `tag`.
`[tag='value']`	Selects all elements that have an immediate child with tag `tag` and value `value`.
`[@attribute]`	Selects all elements that have an attribute named `attribute`.
`[@attribute='value']`	Selects all elements that have an attribute named `attribute` and value `value`.
`[position]`	Selects the element in `position`. This is essentially an index into a list of elements, but unlike with a Python list, the first element is at position 1, not position 0. The expression `last()` can be used to select the element in the last position. Positions can also be expressed relative to the last position. For example, the expression `last()-1` selects the element in the penultimate position.

Now that you’re armed with a basic understanding of XPath, let’s look at some examples of how the lxml APIs can be used to select information from an RPC response. These examples use the same get-system-users-information RPC response variable from previous examples. It may help to analyze each of these statements and their results using the show system users | display xml output at the beginning of this section.⁵

The text content of the first XML element matching an XPath expression can be retrieved with the findtext() method:

>>> response.findtext("uptime-information/up-time")
'1 day, 8:29'

The argument to the findtext() method is an XPath relative to the response element. Because response represents the <system-users-information> element, the uptime-information/up-time XPath matches the <up-time> tag in the response.

The <up-time> element also contains a seconds attribute that provides the system’s uptime in an easier to parse number of seconds since the system booted. The value of this attribute can be accessed by chaining the find() method and the attrib attribute:

>>> response.find("uptime-information/up-time").attrib['seconds']
'116940'

While the findtext() method returns a string, the find() method returns an lxml.etree.Element object. The XML attributes of that lxml.etree.Element object can then be accessed using the attrib dictionary. The attrib dictionary is keyed using the XML attribute name.

In the response variable, there is one <user-entry> XML element for each user currently logged into the device. Each <user-entry> element contains a <user> element with the user’s username. The findall() method returns a list of lxml.etree.Element objects matching an XPath. In this example, findall() is used to select the <user> element within every <user-entry> element:

>>> users = response.findall("uptime-information/user-table/user-entry/user")
>>> for user in users:
...     print user.text
... 
root
foo
bar
user
>>>

The result of this example is a list of usernames for all users currently logged into the Junos device.

The next example combines the findtext() method with an XPath expression that selects the first matching XML element that has a specific matching child element:

>>> response.findtext("uptime-information/user-table/user-entry[tty='u0']/user")
'root'

The result of this example is the username of the user currently logged into the Junos device’s console. (On this device, the console has a tty name of u0.) The XPath selects the <user> element from the first <user-entry> element that also has a <tty> child element with the value u0.

The next example retrieves the number of seconds that the user bar’s session has been idle. This is done by combining the find() method, an XPath with the [tag='text'] predicate, and the attrib attribute dictionary:

>>> XPATH = "uptime-information/user-table/user-entry[user='bar']/idle-time"
>>> response.find(XPATH).attrib['seconds']
'30900'

Note

In the preceding example, the XPATH variable is used as a constant simply to avoid line wrap in the printed book.

Because the preceding example used the find() method, it will only return information for the bar user’s first CLI session. If the bar user has multiple CLI sessions open, you could retrieve a list of the idle times by replacing find() with findall().

One significant difference between lxml.etree.Element objects, with their corresponding methods, and native Python data structures is how nonexistent data is handled. In order to demonstrate this, we’ll create an equivalent multilevel Python dictionary to store the uptime information:

>>> from pprint import pprint
>>> example_dict = { 'uptime-information' : { 'up-time' : '1 day, 8:29' }}
>>> pprint(example_dict)
{'uptime-information': {'up-time': '1 day, 8:29'}}

Accessing the information in this dictionary is arguably easier than using the findtext() method:

>>> example_dict['uptime-information']['up-time']
'1 day, 8:29'

However, consider what happens when you try to access data that doesn’t exist in the response. The native Python dictionary raises a KeyError exception:

>>> example_dict['foo']['bar']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'foo'

If there’s a possibility that a piece of information may be missing from the response, then the dictionary access needs to be wrapped in a try/except block that gracefully handles the resulting KeyError exception. Conversely, if the requested information is missing from an lxml.etree.Element object, the find() and findtext() methods simply return None rather than raising an exception:

>>> print response.find("foo/bar")
None
>>> print response.findtext("foo/bar")
None

The findall() method exhibits a similar behavior. It returns an empty list when the XPath expression fails to match any XML elements:

>>> print response.findall("foo/bar")
[]

It’s also important to remember the findall() method always returns a list of lxml.etree.Element objects. It does not return an lxml.etree.Element object directly. This behavior remains true even when there’s only one object in the list:

>>> user_entries = response.findall("uptime-information/user-table/user-entry") 
>>> type(user_entries)
<type 'list'>
>>> len(user_entries)
4

When multiple users are logged in, this list contains one object for each user. However, the next example demonstrates the response when only one user is logged in:

>>> new_response = r0.rpc.get_system_users_information(normalize=True)
>>> new_users = new_response.findall("uptime-information/user-table/user-entry") 
>>> type(new_users)
<type 'list'>
>>> len(new_users)
1

In this single-user case, notice that the response is still a list; it’s just a list with a single item. This behavior allows the program to loop over the list of user entries without having to create different code paths for the no users, one user, and multiple users cases.

There is one final situation that is important to understand. Attempting to access a nonexistent XML attribute still raises a KeyError. That’s because an lxml.etree.Element object’s attrib attribute is a Python dictionary keyed on the XML attribute name:

>>> response.find("uptime-information/up-time").attrib['foo']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "lxml.etree.pyx", line 2366, in lxml.etree._Attrib.__getitem__ (src/lx...
KeyError: 'foo'

A related but slightly different situation is attempting to access an XML attribute on a nonexistent element. In this example, the find() method returns None because there is no XML element that matches the XPath expression.

One way of handling access to the attrib dictionary is wrapping the access in a try/except block that catches both the AttributeError and the KeyError:

>>> try: foo_attrib = response.find("uptime-information/bar").attrib['foo']
... except (AttributeError, KeyError): foo_attrib = None
... 
>>> print foo_attrib
None
>>> try: foo_attrib = response.find("uptime-information/up-time").attrib['foo']
... except (AttributeError, KeyError): foo_attrib = None
... 
>>> print foo_attrib
None

While this hasn’t been an exhaustive set of examples, you’ve now seen the basics of accessing PyEZ RPC responses in the default lxml.etree.Element object format. These examples have demonstrated some of the most common ways to access the response information, but the lxml library does offer many more tools. Full documentation on lxml’s API is available at the lxml website. The lxml API is also mostly compatible with the well-known ElementTree API. Documentation on the ElementTree API is part of the Python Standard Library documentation at the ElementTree XML API.

The next section begins to explore the other optional formats that PyEZ can use to return RPC responses.

Response Normalization

Response normalization is a PyEZ feature that actually alters the XML content returned from an RPC method. There are some Junos RPCs that return XML data where the values of certain XML elements are wrapped in newline or other whitespace characters. An example of this extra whitespace can be seen with the get-system-users-information RPC we used in the previous section:

>>> response = r0.rpc.get_system_users_information()
>>> response.findtext("uptime-information/up-time")
'\n4 days, 17 mins\n'

Notice the text for the <up-time> element has a newline character before and after the value string. Response normalization is designed to address this situation. When response normalization is enabled, all whitespace characters at the beginning and end of each XML element’s value are removed. Response normalization is disabled by default (except when using tables and views, as discussed in “Operational Tables and Views”). It can be enabled by adding the normalize=True argument to an RPC method:

>>> response = r0.rpc.get_system_users_information(normalize=True)
>>> response.findtext("uptime-information/up-time")
'4 days, 17 mins'

Notice the newline characters at the beginning and end of the value have been removed, but the whitespace within the value is maintained.

You may have noticed the normalize=True argument was added to the r0.rpc.get_system_users_information() method invocation in the previous section. Why was the argument used in those examples? The additional whitespace characters present with some RPC responses make some XPath expressions more difficult and less intuitive. As an example, consider the XPath expression used to find the user logged into the Junos device’s console (on this device the console has a tty name of u0). Without response normalization, the previous XPath example fails to return a matching XML element:

>>> response = r0.rpc.get_system_users_information()
>>> response.findtext("uptime-information/user-table/user-entry[tty='u0']/user")
>>>

The empty response is because the value in the [tag='value'] portion of the XPath expression must match exactly. Explicitly displaying the value of the <tty> tag for the desired user entry (which just happens to be in position 7 in this specific example) reveals that the value has leading and trailing newline characters:

>>> response.findtext("uptime-information/user-table/user-entry[7]/tty")
'\nu0\n'
>>>

You could modify the XPath expression to look for this specific value, as shown in this example:

>>> response.findtext(
      ...    "uptime-information/user-table/user-entry[tty='\nu0\n']/user"
      ... )
'\nroot\n'
>>>

However, it can be somewhat unpredictable if additional whitespace is present for the value of any given XML element in any given RPC response. Instead, it’s easier to simply use response normalization to allow the expected value to be matched without having to worry about leading or trailing whitespace:

>>> response = r0.rpc.get_system_users_information(normalize=True)
>>> response.findtext("uptime-information/user-table/user-entry[tty='u0']/user")
'root'
>>>

Response normalization removes leading and trailing whitespace from the values of all XML elements in the response. So, not only does this simplify the XPath expression, but it avoids the need to do additional processing to remove whitespace from the username value being accessed.

One final note on response normalization: it can be enabled on a per-RPC basis, as shown so far, or it can be enabled for all RPCs on a device instance or in a NETCONF session by specifying the normalize=True argument to the Device() or open() calls, respectively. Here’s an example of enabling it for the device instance:

>>> r0 = Device(host='r0',user='user',password='user123',normalize=True)
>>> r0.open()
Device(r0)
>>> response = r0.rpc.get_system_users_information()
>>> response.findtext("uptime-information/up-time")
'4 days, 2:03'

If response normalization is enabled for the device instance, as just shown, it’s still possible to override the behavior on a per-RPC basis by specifying a normalize=False argument when invoking the RPC method.

jxmlease

In addition to parsing XML documents into lxml.etree.Element objects, you can also use the jxmlease library to parse RPC responses into jxmlease objects. You may find you prefer jxmlease, described in “Using Structured Data in Python”, over using XPath and the lxml library. This format offers an excellent balance between functionality and ease of use. The values of XML elements can be accessed using the same tools as native Python dictionaries and lists. You saw an example of using jxmlease in the example script in Chapter 3. In that example, jxmlease was used to parse the XML string returned from the Junos RESTful API service. However, jxmlease can also be used to directly parse an lxml.etree.Element object. This parsing is done by passing an lxml.etree.Element object to an instance of the jxmlease.EtreeParser class. Here’s an example of using this technique to return the output of the get-system-users-information RPC as an jxmlease.XMLDictNode object:

>>> import jxmlease
>>> parser = jxmlease.EtreeParser()
>>> response = parser(r0.rpc.get_system_users_information())
>>> response.prettyprint(depth=3)
{'system-users-information': {'uptime-information': {'active-user-count': u'6',
                                                     'date-time': u'12:39PM',
                                                     'load-average-1': u'0.29',
                                                     'load-average-15': u'0.41',
                                                     'load-average-5': u'0.43',
                                                     'up-time': u'3 days, 4:57',
                                                     'user-table': {...}}}}
>>> type(response)
<class 'jxmlease.XMLDictNode'>

The response is actually a jxmlease.XMLDictNode object, but behaves much like an ordered dictionary.

You can access any level of the jxmlease.XMLDictNode object by specifying a chain of dictionary keys. The tag of each XML element is used as a dictionary key, and begins with the tag of the RPC response’s root element. As an example, this statement accesses the system uptime:

>>> print response['system-users-information']['uptime-information']['up-time']
3 days, 4:19

Notice that the keys begin with ['system-users-information'] and the equivalent of response normalization is applied to jxmlease.XMLDictNode objects by default.

For XML elements that have attributes, you can use the object’s get_xml_attr() method to retrieve the attribute’s value:

>>> ut = response['system-users-information']['uptime-information']['up-time']
>>> ut.get_xml_attr('seconds')
'274740'

get_xml_attr() also allows a default value to be returned if the XML attribute name does not exist:

>>> ut.get_xml_attr('foo',0)
0

The PyEZ example script in “A PyEZ Example” further demonstrates using jxmlease with PyEZ RPC responses.

JSON

Junos began supporting the JSON output format in release 14.2 or later. When using PyEZ to invoke an RPC on a Junos device running release 14.2 and later, PyEZ can request this JSON output. You request JSON output from an RPC by passing an argument that is a dictionary with a single 'format' key with a value of 'json'. This argument causes the Junos device to return the RPC response as a JSON string. However, the JSON string is not returned directly to the user. Instead, PyEZ invokes json.loads() to parse the JSON string into a native Python data structure composed of dictionaries and lists. Here’s an example:

>>> response = r0.rpc.get_system_users_information({'format': 'json'})
>>> type(response)
<type 'dict'>
>>> from pprint import pprint
>>> pprint(response, depth=3)
{u'system-users-information': [{u'attributes': {...},
                                u'uptime-information': [...]}]}

This behavior of automatically parsing the JSON response into a Python data structure is analogous to the behavior with the default XML format. When using the default XML format, the device returns a string containing an XML document and PyEZ parses the XML document into a Python data structure. The resulting response can be accessed with all of Python’s normal tools for handling dictionaries and lists. See “JSON data” for more details on the JSON format, including its limitations.

Operational Tables and Views

In addition to the “RPC on Demand” feature, PyEZ offers another method for invoking an operational RPC and mapping the response into a Python data structure. This “tables and views” feature provides precise control for mapping portions of the RPC response into a Python data structure. In addition, it allows this mapping to be stored for e asy reuse. In fact, PyEZ comes prepackaged with a set of example tables and views that you can use or modify to suit your particular needs.

The operational data of a Junos device is the set of state representing the current running conditions of the device. Operational data is read-only information that is separate from the device’s configuration data. You typically view operational data using CLI show commands or using operational XML RPCs. You can think of this operational data as similar to a database. Databases are organized into a collection of tables, and in a similar way, PyEZ conceptually organizes a Junos device’s operational data into a collection of tables.

In PyEZ, a “table” represents the information that is returned by a particular XML RPC. A PyEZ table is further divided into a list of “items.” These items are all of the XML nodes in the RPC output that match a specific XPath expression.

Similar to how a database view selects and presents a subset of fields from a database table, a PyEZ “view” selects and maps a set of fields (XML nodes) from each PyEZ table item into a native Python data structure. Each PyEZ table has at least one view, the default view, for mapping an item’s fields into a native Python data structure. Additional views may be defined, but only the default view is required.

Multiple views are used to select different information from the table items. Multiple views are similar in concept to the terse, brief, detail, and extensive flags to various CLI commands. Just like each of these CLI flags outputs different information from the same CLI command, different views present different information from the same table.

Prepackaged Operational Tables and Views

Let’s begin by using some of the existing tables and views included with PyEZ. Tables and views are defined in YAML files, which have a .yml filename extension. The contents of these YAML files will be covered in detail in the next session when explaining how to create your own tables and views.

The prepackaged tables and views included with PyEZ are located in the op sub directory of the jnpr.junos module’s installation directory, as shown by this directory listing:

user@h0$ pwd
/usr/local/lib/python2.7/dist-packages/jnpr/junos
user@h0$ ls op/*.yml
op/arp.yml                     op/fpc.yml           op/lacp.yml  op/phyport.yml
op/bfd.yml                     op/idpattacks.yml    op/ldp.yml   op/routes.yml
op/ccc.yml                     op/intopticdiag.yml  op/lldp.yml  op/teddb.yml
op/ethernetswitchingtable.yml  op/isis.yml          op/nd.yml    op/vlan.yml
op/ethport.yml                 op/l2circuit.yml     op/ospf.yml  op/xcvr.yml

The op sub directory is relative to the location where the jnpr.junos module of the PyEZ library is installed. On the example automation host this directory is /usr/local/lib/python2.7/dist-packages/jnpr/junos, but the location is installation-specific and may be different on your machine. Determine the directory location on your machine by displaying the directory of the jnpr.junos.__file__ attribute. Use this recipe:

>>> import jnpr.junos
>>> import os.path
>>> os.path.dirname(jnpr.junos.__file__)
'/usr/local/lib/python2.7/dist-packages/jnpr/junos'

In order to use the prepackaged tables and views, you will need to know the names of the available tables. You can determine the table names by searching the .yml files for the string Table:, as shown in this example:

user@h0$ pwd
/usr/local/lib/python2.7/dist-packages/jnpr/junos
user@h0$ grep Table: op/*.yml
op/arp.yml:ArpTable:
op/bfd.yml:BfdSessionTable:
op/bfd.yml:_BfdSessionClientTable:
op/ccc.yml:CCCTable:
op/ethernetswitchingtable.yml:EthernetSwitchingTable:
op/ethernetswitchingtable.yml:_MacTableEntriesTable:
op/ethernetswitchingtable.yml:_MacTableInterfacesTable:
op/ethport.yml:EthPortTable:
op/fpc.yml:FpcHwTable:
op/fpc.yml:FpcMiReHwTable:
op/fpc.yml:FpcInfoTable:
op/fpc.yml:FpcMiReInfoTable:
op/idpattacks.yml:IDPAttackTable:
op/intopticdiag.yml:PhyPortDiagTable:
op/isis.yml:IsisAdjacencyTable:
op/isis.yml:_IsisAdjacencyLogTable:
op/l2circuit.yml:L2CircuitConnectionTable:
op/lacp.yml:LacpPortTable:
op/lacp.yml:_LacpPortStateTable:
op/lacp.yml:_LacpPortProtoTable:
op/ldp.yml:LdpNeighborTable:
op/ldp.yml:_LdpNeighborHelloFlagsTable:
op/ldp.yml:_LdpNeighborTypesTable:
op/lldp.yml:LLDPNeighborTable:
op/nd.yml:NdTable:
op/ospf.yml:OspfNeighborTable:
op/phyport.yml:PhyPortTable:
op/phyport.yml:PhyPortStatsTable:
op/phyport.yml:PhyPortErrorTable:
op/routes.yml:RouteTable:
op/routes.yml:RouteSummaryTable:
op/routes.yml:_rspTable:
op/teddb.yml:TedTable:
op/teddb.yml:_linkTable:
op/teddb.yml:TedSummaryTable:
op/vlan.yml:VlanTable:
op/xcvr.yml:XcvrTable:

The first step in using one of these tables is to import the appropriate Python class. (The Python class name is the same as the table name defined in the YAML file.) Let’s use ArpTable from the first line of the preceding output as an example. In order to import the ArpTable class, enter the following from statement:

>>> from jnpr.junos.op.arp import ArpTable

Just like RPC on Demand methods, tables and views operate on a PyEZ device instance with an open NETCONF session. As before, we use the Device() and open() calls to create and open the r0 device instance variable:

>>> from jnpr.junos import Device
>>> r0 = Device(host='r0',user='user',password='user123')
>>> r0.open()
Device(r0)
>>>

Create an empty table instance by passing the device instance variable (r0) as an argument to the class constructor (ArpTable()):

>>> arp_table = ArpTable(r0)

The arp_table instance variable can now be populated with all table items by invoking the get() instance method:

>>> arp_table.get()
ArpTable:r0: 9 items

In the preceding output, the get() method executes an RPC and uses the results to populate the arp_table instance with nine items.

Note

In the case of ArpTable, each item represents an <arp-table-entry> node from the get-arp-table-information RPC’s XML output.

An alternative way of creating and populating the table is to bind the table as an attribute of the device instance variable. The get() method is then invoked on this bound attribute:

>>> from jnpr.junos.op.arp import ArpTable
>>> r0.bind(arp_table=ArpTable)
>>> r0.arp_table.get()
ArpTable:r0: 9 items

In the preceding output, we again see the get() method executes an RPC and populates the r0.arp_table attribute with nine items. Storing the arp_table as an attribute of the device instance variable is a convenient notation when you’re maintaining a table for multiple devices. For example, you can easily store and access separate ArpTable tables for the devices r0, r1, and r2. You can then access each table as an attribute of the respective device instance variable.

It is also possible to retrieve specific table items by passing arguments to the get() method. When the get() method is invoked, it executes a corresponding XML RPC defined in the table’s YAML file. The get() method can be passed the same parameters as if you were invoking the RPC using RPC on Demand. In the case of ArpTable, the get-arp-table-information RPC is executed. The get-arp-table-information RPC supports an <interface> argument to limit the response to ARP entries on a specific interface. Here’s an example of retrieving only the ARP entries on the ge-0/0/0.0 logical interface:

>>> arp_table.get(interface='ge-0/0/0.0')
ArpTable:r0: 1 items
>>> pprint(arp_table.items())
[('00:05:86:18:ec:02',
  [('interface_name', 'ge-0/0/0.0'),
   ('ip_address', '10.0.4.2'),
   ('mac_address', '00:05:86:18:ec:02')])]

Invoking a table’s get() method always updates the table’s items by executing an XML RPC against the Junos device and receiving the RPC’s response. When you invoke the get() method, the new response overwrites all previous data in the table. Remember to invoke the get() method, with the appropriate arguments, any time you need to refresh the data stored in the table.

Note

Normalization of table and view data is enabled by default. Refer back to “Response Normalization” for more information on normalization. If you need to disable normalization, pass the normalize=False argument to the get() method:

>>> arp_table.get(normalize=False)
ArpTable:r0: 9 items

When you create an instance of a table, it returns a jnpr.junos.factory.OpTable object. Each jnpr.junos.factory.OpTable object operates similarly to a Python OrderedDict of view objects. The following example refreshes the arp_table instance with all ARP table entries and then shows arp_table’s type is jnpr.junos.factory.OpTable.ArpTable, which is a subclass of jnpr.junos.factory.OpTable. Like with a dictionary, the keys and values of arp_table are accessed using the items() method:

>>> arp_table.get()
ArpTable:r0: 9 items
>>> type(arp_table)
<class 'jnpr.junos.factory.OpTable.ArpTable'>
>>> from pprint import pprint
>>> pprint(arp_table.items())
[('00:05:86:48:49:00',
  [('interface_name', 'ge-0/0/4.0'),
   ('ip_address', '10.0.1.2'),
   ('mac_address', '00:05:86:48:49:00')]),
 ('00:05:86:78:2a:02',
  [('interface_name', 'ge-0/0/1.0'),
   ('ip_address', '10.0.2.2'),
   ('mac_address', '00:05:86:78:2a:02')]),
 ('00:05:86:68:0b:02',
  [('interface_name', 'ge-0/0/2.0'),
   ('ip_address', '10.0.3.2'),
   ('mac_address', '00:05:86:68:0b:02')]),
 ('00:05:86:18:ec:02',
  [('interface_name', 'ge-0/0/0.0'),
   ('ip_address', '10.0.4.2'),
   ('mac_address', '00:05:86:18:ec:02')]),
 ('00:05:86:08:cd:00',
  [('interface_name', 'ge-0/0/3.0'),
   ('ip_address', '10.0.5.2'),
   ('mac_address', '00:05:86:08:cd:00')]),
 ('10:0e:7e:b1:f4:00',
  [('interface_name', 'fxp0.0'),
   ('ip_address', '10.102.191.252'),
   ('mac_address', '10:0e:7e:b1:f4:00')]),
 ('10:0e:7e:b1:b0:80',
  [('interface_name', 'fxp0.0'),
   ('ip_address', '10.102.191.253'),
   ('mac_address', '10:0e:7e:b1:b0:80')]),
 ('00:00:5e:00:01:c9',
  [('interface_name', 'fxp0.0'),
   ('ip_address', '10.102.191.254'),
   ('mac_address', '00:00:5e:00:01:c9')]),
 ('56:68:a6:6a:47:b2',
  [('interface_name', 'em1.0'),
   ('ip_address', '128.0.0.16'),
   ('mac_address', '56:68:a6:6a:47:b2')])]
>>>

Note

The previous example used the first method of creating and populating the table, the arp_table instance variable. If the table has been bound to the device instance variable, using the second method, you would access it with r0.arp_table rather than arp_table:

>>> type(r0.arp_table)
<class 'jnpr.junos.factory.OpTable.ArpTable'>

Because the table operates similarly to an OrderedDict, the individual items (which are jnpr.junos.factory.View.ArpView objects) can be accessed by either position or key:

>>> type(arp_table[0])
<class 'jnpr.junos.factory.View.ArpView'>
>>> pprint(arp_table[0].items())
[('interface_name', 'ge-0/0/4.0'),
 ('ip_address', '10.0.1.2'),
 ('mac_address', '00:05:86:48:49:00')]
>>> pprint(arp_table['00:05:86:48:49:00'].items())
[('interface_name', 'ge-0/0/4.0'),
 ('ip_address', '10.0.1.2'),
 ('mac_address', '00:05:86:48:49:00')]
>>>

Individual values within a view item can be accessed with two-level references using either an index or a key value for the outer reference:

>>> arp_table['00:05:86:48:49:00']['ip_address']
'10.0.1.2'
>>> arp_table[0]['ip_address']
'10.0.1.2'
>>> arp_table['00:05:86:48:49:00']['interface_name']
'ge-0/0/4.0'
>>> arp_table[0]['interface_name']
'ge-0/0/4.0'

If you assign a view object (a subclass of jnpr.junos.factory.View) to a variable, you can also access each of the view object’s fields as Python properties, or as a dictionary:

>>> arp_item = arp_table[0]
>>> arp_item.ip_address
'10.0.4.2'
>>> arp_item.interface_name
'ge-0/0/0.0'
>>> arp_item.mac_address
'00:05:86:18:ec:02'
>>>
>>> arp_item.keys()
['interface_name', 'ip_address', 'mac_address']
>>> arp_item.items()
[('interface_name', 'ge-0/0/0.0'), 
 ('ip_address', '10.0.4.2'), 
 ('mac_address', '00:05:86:18:ec:02')]

In addition, every view object has two special properties, name and T. The name property provides the view’s unique name, or key, within the table. The T property is a reference back to the associated table containing the view:

>>> arp_item.name
'00:05:86:18:ec:02'
>>> arp_item.T
ArpTable:r0: 9 items

In this section, you’ve seen how to instantiate, populate, and access prepackaged tables and views. Now, let’s take a look at how to define a custom table and view rather than being restricted to the prepackaged tables and views that are included with PyEZ.

Creating New Operational Tables and Views

PyEZ tables and views are defined in .yml files using the YAML format. YAML uses a simple and intuitive human-readable syntax to define hierarchical data structures formed from scalars, lists, and associative arrays (similar to Python dictionaries). For more information on YAML, refer to the following sidebar, “YAML at a Glance”, and for comprehensive information on YAML refer to the YAML specification.

YAML at a Glance

YAML is a recursive acronym for “YAML Ain’t Markup Language.” YAML is a language-independent data serialization format that is easily parsed by humans. It is designed for data structures or configuration files that are often hand-edited. This is the case for the YAML files that define tables and views, and for the Ansible IT automation framework you will see in Chapter 10. Ansible also uses the YAML format for its configuration files.

YAML supports scalar, list, and associative array (similar to a Python dictionary) data types. It also supports the ability to combine these basic types into more complex hierarchical data structures. Similar to Python, YAML uses indentation to signal hierarchical relationships rather than using quotation marks, braces, brackets, or other balanced enclosure markers that some people may find hard to read. The amount of indentation is unimportant as long as elements at the same level of the hierarchy share the same left indentation. YAML does not allow tab characters to be used for indentation.

A file contains one or more YAML “documents.” Each document begins with three hyphens (---). Comments begin with # and continue to the end of the line. Blank lines may be added for readability.

YAML autodetects the data type of scalar values based on context. The core data types include integers, floating-point numbers, strings, and Booleans. Some examples of each data type:

--- # Example data types
answer: 42           # Integer
pi:     3.14159265   # Float
status: Foo Bar      # String - Internal whitespace is maintained
state:    Foobar     # String -  Leading and trailing whitespace is removed
response: Yes        # Boolean - True
countdown: "54321"   # String - Quotes used to disambiguate

A list is defined by placing each list item on a separate line at the same hierarchy level (same left indentation). Each list item begins with a hyphen followed by a space:

--- # List of most populated countries per continent
- China
- United States of America
- Brazil
- Nigeria
- Germany
- Australia

Alternatively, a list can be defined by enclosing the list in brackets and separating each item with a comma and space:

--- # List of most populated countries per continent
[China, United States of America, Brazil, Nigeria, Germany, Australia]

An associative array is specified as a set of key/value pairs. Each key/value pair is specified on a separate line at the same hierarchy level (same left indentation). The key is separated from the value using a colon and one or more spaces:

--- # Associative array of most populated countries per continent
Asia:          China
North_America: United States of America
South_America: Brazil
Africa:        Nigeria
Europe:        Germany
Australia:     Australia

Again, an alternative syntax exists for specifying an associative array. The associative array is surrounded by braces and each key/value pair is separated by a comma and a space. Key/value pairs continue to be separated by a colon and space:

--- # Associative array of most populated countries per continent
{Asia: China, North_America: United States of America, South_America: Brazil}

Associative arrays and lists can be combined into hierarchical data structures of arbitrary complexity, and both the indented and inline syntaxes can be can be combined:

--- # Country info
Asia:
    - Name: China
      Capital: Beijing
      Population: 1357000000
      Area:
          sq_km:    9597000
          sq_miles: 3705000
    - Name: India
      Capital: New Delhi
      Population: 1252000000
      Area:
          sq_km:    3288000
          sq_miles: 1269000
Africa:
    - Name: Nigeria
      Capital: Abuja
      Population: 173600000
      Area: {sq_km: 923768, sq_miles: 356669}
    - Name: Ethiopia
      Capital: Addis Ababa
      Population: 94100000
      Area: {sq_km: 1127000, sq_miles: 435200}

Notice that both the Asia and Africa keys have values that are two-item lists. Each list item is an associative array with Name, Capital, Population, and Area keys. The value of the Area key is an associative array with the keys sq_km and sq_miles. For China and India, the Area associative array is specified using the normal (indented) syntax, but for Nigeria and Ethiopia the Area associative array is specified using the alternative syntax.

PyEZ table and view definitions use a subset of the full YAML syntax. They primarily use associative arrays with multiple levels of hierarchy. Values are typically string data types. Here’s an example using the arp.yml file, which defines the ArpTable and ArpView classes used in the previous section:

---
ArpTable: 
  rpc: get-arp-table-information 
  item: arp-table-entry 
  key: mac-address 
  view: ArpView 

ArpView: 
  fields: 
    mac_address: mac-address 
    ip_address: ip-address 
    interface_name: interface-name

: The PyEZ YAML loader assumes that anything in the first column is a table or view definition. View definitions are distinguished from table definitions based on their keys. This line begins a table definition. The table name becomes the name of the corresponding Python class. The only restriction on the table name is that it must be a valid name for a Python class.
: The Junos XML RPC (get-arp-table-information) which is invoked to retrieve the table’s item data.
: An XPath expression used to select each table item from the RPC response.
: The name of an XML element within each table item. The value of this XML element becomes the key used to access each item within the native Python data structure.
: The name of the default view used to map a table item into a native Python data structure. This value must exactly match a view name defined in the same YAML document. In this document, the only view is ArpView, defined by the top-level associate array key.
: The PyEZ YAML loader assumes that anything in the first column is a table or view definition. View definitions are distinguished from table definitions based on their keys. This line begins a view definition.
: The value of the fields key is an associative array that maps XPath expressions to names. The names are used as the keys in the native Python data structure (the view object). The XPath expressions are used to fetch values from each item’s XML object.
: The mac-address XPath expression is used to set the value of the mac_address key in the native Python view object. Because mac_address is used as a key in a Python data structure, it must conform to Python variable naming requirements (it should use underscores, not hyphens).
: Again, ip-address is an XPath expression and ip_address is the name of the key in the Python view object.
: The final key in each Python view object is interface_name. The value of the interface_name key is determined by the interface-name XPath expression.

While the arp.yml file demonstrates the required structure and content of PyEZ table and view definitions, it does not include every possible key/value pair. Table 4-5 provides a description of every available key/value pair that may be used in a table definition.

Table 4-5. Keys for defining a PyEZ table
Key name	Required or optional	Description
`rpc`	Required	The name of the Junos XML RPC that is invoked to retrieve the table’s item data. This value should use the actual RPC XML tag name (with hyphens) rather than the PyEZ RPC on Demand method name (with underscores).
`args`	Optional	An associative array whose items are passed as default arguments to `rpc`. The `args` parameter should only be specified if `rpc` should always be called with `args` arguments. The keys of the associative array are the arguments passed to the PyEZ RPC on Demand method (with underscores, not hyphens). If an RPC argument is a flag, set the value of the associative array to `True`.
`args_key`	Optional	The name of one optional unnamed first argument to the `get()` method. For example, the prepackaged `RouteTable` definition includes: `RouteTable:` `rpc:` `get-route-information` `args_key:` `destination` This definition allows the user to call: >>> `route_table.get('10.0.0.0/8')` Which causes the following RPC to be sent to the Junos device: `<get-route-information>` `<destination>`10.0.0.0/8`</destination>` `</get-route-information>`
`item`	Required	An XML XPath expression that selects each table item (record of data) from the RPC response. Each XML element in the RPC response which matches the XPath expression becomes a table item. The XPath expression is relative to the first element in the response after the `<rpc-reply>` tag. This is the same behavior we observed in “lxml Elements” when using lxml methods with RPC responses.
`key`	Optional, but recommended	An XPath to select an XML element within each table item. The value of the XML element becomes the key used to access each item within the native Python data structure. If `key` is not specified, the XPath `name` is used as the key. If multiple XML elements are required to uniquely identify a table item, the value of this key is a YAML list containing an XPath for each XML element that forms the key. It is recommended to explicitly specify `key` even if its value is set to the default of `name`.
`view`	Required^a	The name of the default view used to map a table item into a native Python data structure. This value must exactly match the name of a view defined in the same YAML document.
^a If a `view` key is not present, then each table item is returned as an `lxml.etree.Element` object. This negates much of the benefit of tables and views, so the `view` key is effectively required.

Let’s put the information from Table 4-5 into practice by creating a new table definition for the show system users CLI command we used earlier in this chapter. First, the show system users CLI command maps to the get-system-users-information XML RPC. Our table definition will include the no-resolve RPC argument to avoid DNS lookups on the <from> elements in the RPC response:

user@r0> show system users no-resolve | display xml rpc 
<rpc-reply xmlns:junos="http://xml.juniper.net/junos/15.1R2/junos">
    <rpc>
        <get-system-users-information>
                <no-resolve/>
        </get-system-users-information>
    </rpc>
    <cli>
        <banner></banner>
    </cli>
</rpc-reply>

It is helpful to reference the structure of the expected RPC response when creating the new table and view definition. An abbreviated version of the get-system-users-information response is included here for your reference. This RPC response includes both system-wide information (<up-time>, <active-user-count>, <load-average-1>, etc.) and per-login-specific information (the <user-entry> elements):

user@r0> show system users no-resolve | display xml        
<rpc-reply xmlns:junos="http://xml.juniper.net/junos/15.1R2/junos">
    <system-users-information xmlns="http://xml.juniper.net/junos/15.1R2/junos">
        <uptime-information>
            <date-time junos:seconds="1437696318">5:05PM</date-time>
            <up-time junos:seconds="192060">2 days,  5:21</up-time>
            <active-user-count junos:format="8 users">8</active-user-count>
            <load-average-1>0.46</load-average-1>
            <load-average-5>0.50</load-average-5>
            <load-average-15>0.44</load-average-15>
            <user-table>
                <user-entry>
                    <user>root</user>
                    <tty>u0</tty>
                    <from>-</from>
                    <login-time junos:seconds="0">Wed08PM</login-time>
                    <idle-time junos:seconds="16860">4:41</idle-time>
                    <command>cli</command>
                </user-entry>
                <user-entry>
                    <user>user</user>
                    <tty>pts/2</tty>
                    <from>172.29.98.24</from>
                    <login-time junos:seconds="1437694518">4:35PM</login-time>
                    <idle-time junos:seconds="0">-</idle-time>
                    <command>-cli (cli)</command>
                </user-entry>
                <user-entry>
                    <user>user</user>
                    <tty>pts/3</tty>
                    <from>172.29.104.116</from>
                    <login-time junos:seconds="1437667698">9:08AM</login-time>
                    <idle-time junos:seconds="6420">1:47</idle-time>
                    <command>-cli (cli)</command>
                </user-entry>
                ... additional user-entry elements omitted ...
            </user-table>
        </uptime-information>
    </system-users-information>
    <cli>
        <banner></banner>               
    </cli>
</rpc-reply>

The table definition will focus on the login-specific information by extracting the <user-entry> elements into table items. Here is the table definition and explanations for each field. Follow along by typing this content into a users.yml file. Remember to use spaces, not tabs, for indentation:

---
### ------------------------------------------------------
### show system users no-resolve
### ------------------------------------------------------

UserTable:
    rpc: get-system-users-information
    args:
        no_resolve: True
    item: uptime-information/user-table/user-entry
    key:
        - user
        - tty
    view: UserView

This YAML file defines a table named UserTable. The table is populated by running the get-system-users-information RPC with the <no-resolve/> argument. The table will contain one item for each XML entry that matches the XPath expression uptime-information/user-table/user-entry. And, by default, the UserView view will be applied to each table item. This example shows an example of a multielement key. Each part of the key (tty and user) is an XPath expression relative to a table item (a <user-entry> element). The key for each table item will be a tuple formed from the user and tty values.

Note

In most cases, there is a single XML element that uniquely identifies each item. In this case, the <tty> element uniquely identifies each <user-entry> and could have been used as a simple key. However, for demonstration purposes, we’ve chosen to demonstrate a multielement key using the combination of both the <user> and <tty> elements.

The item key in the previous table definition is a relatively simply XPath that specifies a three-level hierarchy, uptime-information/user-table/user-entry, to select all <user-entry> elements. While advanced XPath expressions are outside the scope of this book, it can be helpful to see how a more complicated XPath expression is used to control table item selection. For example, substituting this XPath expression will select only the user logins that have been idle for more than 1 hour (3600 seconds):

item: "uptime-information/user-table/user-entry[idle-time[@seconds>3600]]"

Each table item still represents a <user-entry> element, but only certain <user-entry> elements are selected. Specifically, the <user-entry> elements that have an <idle-time>child element, and where the <idle-time> element has a seconds attribute with a value greater than 3600, are selected.

Note

The preceding XPath expression must be enclosed in double quotes. It contains brackets, which YAML would otherwise interpret as an inline list.

Now that we’ve seen how to define a more complex table, let’s focus our attention on the corresponding view definition. The sole purpose of a view definition is to map values to keys in a Python view object. The value for a given key comes from a corresponding XPath expression. The XPath expression is relative to each table item, and typically selects a single element from the table item. The selected element’s text node becomes the key’s value in the Python view object.

Append this view definition to the same users.yml file that contains the UserTable definition:

UserView:
    fields:
        from: from
        login_time: login-time
        idle_time: idle-time
        command: command

Do not put a YAML document separator (---) between the table and view definitions. They are both part of the same YAML document.

Each view definition begins with the name of the view in the first column. Again, view definitions are distinguished from table definitions based on their keys. If an rpc key is present, PyEZ assumes it’s a table definition. If no rpc is present, PyEZ assumes it’s a view definition.

In the example, the name of the view is UserView. The view name becomes the name of a corresponding Python class, so the name should follow Python’s conventions for class names. There also must be a view name that exactly matches the table definition’s view property. This is the default view for the table. Additional views may be defined, but only the default view is required.

Notice the colon after UserView. This colon indicates UserView is a key in an associative array. The value of the UserView key is another associative array, which can have four kinds of keys: fields, extends, groups, and fields_groupname. The example uses the simplest of these keys, fields. The value of the fields key is another associative array.

The fields associative array defines a set of names and corresponding XPath expressions (which are relative to the table item). Because the names become attributes of the corresponding Python view object, they must conform to Python naming conventions for variables. In the example, from, login_time, idle_time, and command are the field names.

Warning

View instance attributes (property or method names) cannot be used as the names of fields. Currently, view instance attributes include: asview, items, key, keys, name, refresh, to_json, updater, values, xml, D, FIELDS, GROUPS, ITEM_NAME_XPATH, and T. Do not use these names as field names. You should also avoid field names that begin with an underscore.

The XPath expressions are from, login-time, idle-time, and command. Each of these XPath expressions selects a single matching child element from a table item. Refer back to the XML output and note how each <user-entry> element contains <from>, <login-time>, <idle-time>, and <command> child elements.

It is also possible to use more complicated XPath expressions for field selection. Take this example field definition from the prepackaged RouteTableView in the routes.yml file:

    via: "nh/via | nh/nh-local-interface"

The field name is via. The XPath expression is nh/via | nh/nh-local-interface. This XPath expression selects all <via> and <nh-local-interface> elements under the item’s <nh> child elements. If only one matching element is present, then the via field will be the string value of the matching element. However, if the XPath expression selects more than one element, the value of the via field will be a list of string values. The string values are taken from each matching element.

By default, field values are Python strings. However, there are times when the XML response elements contain numeric values. In these cases, the field’s value can be defined as a Python int or float. As an example, consider the <login-time> and <idle-time> elements of each <user-entry>:

<login-time junos:seconds="0">Wed08PM</login-time>
<idle-time junos:seconds="16860">4:41</idle-time>

The value of each element is a date/time string, not a numeric value. These strings are the values of the login_time and idle_time fields in the previous UserView definition. However, these XML elements also contain a seconds attribute⁶ with a numeric value. Let’s define a new UserExtView that includes integer values for the login time and idle time. Append this new view definition to the same users.yml file:

UserExtView:
    extends: UserView
    fields:
        login_seconds: {"login-time/@seconds": int}
        idle_seconds: {"idle-time/@seconds": int}

Warning

The YAML specification reserves the @ character for future use. Currently, YAML will accept an @ that appears internal to an unquoted string. However, it will produce an error if the string begins with an @ character, such as @seconds. Enclosing these strings in quotes avoids any potential errors with future versions of YAML.

The new UserExtView view definition demonstrates several points. First, look at the values of the login_seconds and idle_seconds fields. Instead of specifying an XPath expression, an inline associative array is used. This associative array includes an XPath expression and the Python data type to be used for the resulting value (int, in this case). The field definitions also use more complex XPath expressions that select the value of the seconds attribute within each element.

Note

The type of a view field can be defined as int, float, or flag. The flag type sets a Boolean value that is True if the element is present and False if the element is not present. Here is an example from the prepackaged EthPortView in the ethport.yml file:

    running: { ifdf-running: flag }
    present: { ifdf-present: flag }

Now, notice the extends key in the UserExtView definition. The extends key is simply a way to create a new view that’s a superset of another view. The extends: UserView line causes all fields from the UserView definition to be included in UserExtView. In addition to the fields from UserView, UserExtView will also include the new login_seconds and idle_seconds fields defined under the fields key.

In addition to containing <user-entry> elements for each login, the get-system-users-information RPC response includes system-wide information such as the <active-user-count> and <load-average-1> elements. While a bit unusual, it is possible to include this system-wide information in each UserExtView object. This is done by appending two additional fields to the UserExtView definition. The full UserExtView definition is now:

UserExtView:
    extends: UserView
    fields:
        login_seconds: {"login-time/@seconds": int}
        idle_seconds: {"idle-time/@seconds": int}
        num_users: {../../active-user-count: int}
        load_avg:  {../../load-average-1: float}

Like all field definitions, the new num_users and load_avg fields each specify an XPath expression that is relative to each table item. These XPath expressions just happen to use the parent element notation (..) to traverse up the XML hierarchy of the response to select nodes that are not contained within the <user-entry> element. Because each <user-entry> element shares a common <user-table> parent element, the result is that each UserExtView object will contain num_users and load_avg fields with the exact same values. The other fields—from, login_time, idle_time, commands, login_seconds, and idle_seconds—continue to have login-specific values because those fields are selected from the per-item <user-entry> element. Notice how the load_avg field is defined to be a Python float.

There is one final tool available when defining views. That tool is groups. Groups are completely optional. They are simply a way of grouping together a set of fields that share XPath expressions with a common prefix. In the UserExtView definition, the num_users and load_avg fields share the ../.. prefix. Here is an alternate definition of UserExtView using the groups tool:

UserExtView:
    groups:
        common: ../..
    fields_common:
        num_users: {active-user-count: int}
        load_avg:  {load-average-1: float}
    fields:
        from: from
        login_time: login-time
        idle_time: idle-time
        command: command
        login_seconds: {"login-time/@seconds": int}
        idle_seconds: {"idle-time/@seconds": int}

Each group has a name and an XPath prefix. In this example, the name is common and ../.. is the XPath prefix. A corresponding fields_groupname key is then used to define the fields that share the XPath prefix. In this case, the fields_common key defines the num_users and load_avg fields. The full XPath expressions for these fields will be ../../active-user-count and ../../load-average-1, respectively.

Note

You may have noticed the preceding UserExtView did not include the extends key. Instead, each of the fields from UserView are copied into the UserExtView fields. At this time, the groups and extends keywords are mutually exclusive. You cannot use both keys in a view definition.

Before moving on, let’s combine the table and view definitions into the complete users.yml file. This file does not use groups in the UserExtView definition:

---
### ------------------------------------------------------
### show system users no-resolve
### ------------------------------------------------------

UserTable:
    rpc: get-system-users-information
    args:
        no_resolve: True
    item: uptime-information/user-table/user-entry
    key:
        - user
        - tty
    view: UserView

UserView:
    fields:
        from: from
        login_time: login-time
        idle_time: idle-time
        command: command

UserExtView:
    extends: UserView
    fields:
        login_seconds: {"login-time/@seconds": int}
        idle_seconds: {"idle-time/@seconds": int}
        num_users: {../../active-user-count: int}
        load_avg:  {../../load-average-1: float}

Using the New Operational Table and View

Now that we’ve created the complete users.yml file, let’s use those table and view definitions. Begin by creating and opening a device instance:

>>> from jnpr.junos import Device
>>> r0 = Device(host='r0',user='user',password='user123')
>>> r0.open()
Device(r0)

Table and view definitions are loaded using the loadyaml() function from the jnpr.junos.factory module. This function creates the Python classes and returns a dictionary that maps table and view names to their corresponding class functions:

>>> from jnpr.junos.factory import loadyaml
>>> user_defs = loadyaml('users.yml')
>>> from pprint import pprint
>>> pprint(user_defs)
{'UserExtView': <class 'jnpr.junos.factory.View.UserExtView'>,
 'UserTable': <class 'jnpr.junos.factory.OpTable.UserTable'>,
 'UserView': <class 'jnpr.junos.factory.View.UserView'>}

The loadyaml() function takes a path to the YAML file containing the table and view definitions. This can be an absolute or relative file path. Relative paths are relative to the current working directory.

Once the table and view definitions have been created, an instance of the table class is created. The class function is accessed by indexing the users_def dictionary with the table name. The class function takes a device instance as its sole argument:

>>> user_table = user_defs['UserTable'](r0)

Alternatively, copy the class names into the global namespace, and invoke the class function using the table name:

>>> globals().update(user_defs)
>>> user_table = UserTable(r0)

Either of these methods results in an empty UserTable instance. The instance is assigned to the user_table variable. Just like with a prepackaged table, the get() method populates the table by invoking the specified RPC:

>>> user_table.get()
UserTable:r0: 8 items
>>> pprint(user_table.items())
[(('root', 'u0'),
  [('command', 'cli'),
   ('idle_time', '4:41'),
   ('from', '-'),
   ('login_time', 'Wed08PM')]),
 (('foo', 'pts/0'),
  [('command', '-cli (cli)'),
   ('idle_time', '7:56'),
   ('from', '172.29.104.116'),
   ('login_time', '9:08AM')]),
 (('bar', 'pts/1'),
  [('command', '-cli (cli)'),
   ('idle_time', '7:56'),
   ('from', '172.29.104.116'),
   ('login_time', '9:08AM')]),
 (('user', 'pts/2'),
  [('command', '-cli (cli)'),
   ('idle_time', '-'),
   ('from', '172.29.98.24'),
   ('login_time', '4:35PM')]),
 (('user', 'pts/3'),
  [('command', '-cli (cli)'),
   ('idle_time', '1:47'),
   ('from', '172.29.104.116'),
   ('login_time', '9:08AM')]),
 (('user', 'pts/4'),
  [('command', '-cli (cli)'),
   ('idle_time', '29'),
   ('from', '172.29.98.24'),
   ('login_time', '4:35PM')]),
 (('foo', 'pts/5'),
  [('command', '-cli (cli)'),
   ('idle_time', '29'),
   ('from', '172.29.98.24'),
   ('login_time', '4:35PM')]),
 (('bar', 'pts/6'),
  [('command', '-cli (cli)'),
   ('idle_time', '29'),
   ('from', '172.29.98.24'),
   ('login_time', '4:35PM')])]
>>>

Take time to compare the output of pprint(user_table.items()) to the UserTable and UserView definitions in the users.yml file. Pay attention to the view fields and values. Also, notice the key for each table item. It’s a tuple formed from the user and tty values. You can access a property in a specific view by specifying the tuple as the table key:

>>> user_table[('foo','pts/5')]['from']
'172.29.98.24'

Another method for loading a user-defined operational table and view is to create a corresponding .py file that executes the loadyaml() function and updates the global namespace. Save the following content to a users.py file in the same directory as users.yml:

"""
Pythonifier for UserTable and UserView
"""
from jnpr.junos.factory import loadyaml
from os.path import splitext
_YAML_ = splitext(__file__)[0] + '.yml'
globals().update(loadyaml(_YAML_))

This code determines the YAML file to load by replacing the .py extension with .yml. It then uses the same loadyaml() and globals().update() functions demonstrated earlier. Here’s an example of using the users.py file to create and populate a UserTable instance:

>>> from users import UserTable
>>> user_table = UserTable(r0)
>>> user_table.get()
UserTable:r0: 8 items
>>>

With the addition of the <name>.py file, the procedure for creating a new table instance is the same regardless of whether the YAML definition file is prepackaged with PyEZ or user-defined.

Note

PyEZ users are encouraged to submit their own table and view definitions to the PyEZ project via a GitHub pull request. As the library of tables and views grows, you become more likely to find an existing table and view to reuse or modify to suit your needs.

Applying a Different View

Remember the UserExtView defined in the users.yml file? The UserExtView defines additional fields for each table item. How do you apply that view to the user_table table instance? Simply import the view class and set the user_table’s view property to the new UserExtView class:

>>> from users import UserExtView
>>> user_table.view = UserExtView
>>> pprint(user_table.items())
[(('root', 'u0'),
  [('idle_seconds', 16860),
   ('from', '-'),
   ('idle_time', '4:41'),
   ('login_seconds', 0),
   ('num_users', 8),
   ('command', 'cli'),
   ('load_avg', 0.51),
   ('login_time', 'Wed08PM')]),
 (('foo', 'pts/0'),
  [('idle_seconds', 28560),
   ('from', '172.29.104.116'),
   ('idle_time', '7:56'),
   ('login_seconds', 1437667701),
   ('num_users', 8),
   ('command', '-cli (cli)'),
   ('load_avg', 0.51),
   ('login_time', '9:08AM')]),
 (('bar', 'pts/1'),
  [('idle_seconds', 28560),
   ('from', '172.29.104.116'),
   ('idle_time', '7:56'),
   ('login_seconds', 1437667701),
   ('num_users', 8),
   ('command', '-cli (cli)'),
   ('load_avg', 0.51),
   ('login_time', '9:08AM')]),
 (('user', 'pts/2'),
  [('idle_seconds', 0),
   ('from', '172.29.98.24'),
   ('idle_time', '-'),
   ('login_seconds', 1437694521),
   ('num_users', 8),
   ('command', '-cli (cli)'),
   ('load_avg', 0.51),
   ('login_time', '4:35PM')]),
 (('user', 'pts/3'),
  [('idle_seconds', 6420),
   ('from', '172.29.104.116'),
   ('idle_time', '1:47'),
   ('login_seconds', 1437667701),
   ('num_users', 8),
   ('command', '-cli (cli)'),
   ('load_avg', 0.51),
   ('login_time', '9:08AM')]),
 (('user', 'pts/4'),
  [('idle_seconds', 1740),
   ('from', '172.29.98.24'),
   ('idle_time', '29'),
   ('login_seconds', 1437694521),
   ('num_users', 8),
   ('command', '-cli (cli)'),
   ('load_avg', 0.51),
   ('login_time', '4:35PM')]),
 (('foo', 'pts/5'),
  [('idle_seconds', 1740),
   ('from', '172.29.98.24'),
   ('idle_time', '29'),
   ('login_seconds', 1437694521),
   ('num_users', 8),
   ('command', '-cli (cli)'),
   ('load_avg', 0.51),
   ('login_time', '4:35PM')]),
 (('bar', 'pts/6'),
  [('idle_seconds', 1740),
   ('from', '172.29.98.24'),
   ('idle_time', '29'),
   ('login_seconds', 1437694521),
   ('num_users', 8),
   ('command', '-cli (cli)'),
   ('load_avg', 0.51),
   ('login_time', '4:35PM')])]
>>>

Alternatively, you can apply the asview() method to a single view instance (which represents a single table item), as shown in the following example. First, the table’s view is reset to UserView:

>>> from users import UserView
>>> user_table.view = UserView

Next, the first table item is assigned to one_view:

>>> one_view = user_table[0]

The items within one_view are printed using the default UserView, and then printed again using the UserExtView:

>>> pprint(one_view.items())
[('command', 'cli'),
 ('idle_time', '4:41'),
 ('from', '-'),
 ('login_time', 'Wed08PM')]
>>> pprint(one_view.asview(UserExtView).items())
[('idle_seconds', 16860),
 ('from', '-'),
 ('idle_time', '4:41'),
 ('login_seconds', 0),
 ('num_users', 8),
 ('command', 'cli'),
 ('load_avg', 0.51),
 ('login_time', 'Wed08PM')]
>>>

Saving and Loading XML Files from Tables

One final note on tables and views before moving on to configuring Junos devices with PyEZ is that PyEZ tables are normally populated by executing an XML RPC over an open NETCONF connection. However, it is also possible to save, and later reuse, the XML RPC response. Saving the XML RPC response is accomplished by invoking the savexml() method on the table instance. The path argument is required and specifies where (an absolute or relative file path) to store the XML output. The optional hostname and timestamp flags can be used to save multiple tables into unique XML files:

>>> user_table.savexml(path='/tmp/user_table.xml',hostname=True,timestamp=True)
>>> quit()
user@h0$ ls /tmp/*.xml
/tmp/user_table_r0_20150723151807.xml

Later, an XML file can be used to populate a table. Loading a table from an XML file does not require a NETCONF connection to the device that originally produced the RPC response. To load the table from an XML file, provide the path argument to the table’s class function:

>>> user_table = UserTable(path='/tmp/user_table_r0_20150723151807.xml')
>>> user_table.get()
UserTable:/tmp/user_table_r0_20150723151807.xml: 8 items
>>>

Note

The get() method must still be called to populate the table.

Now that we’ve seen multiple ways to invoke operational RPCs and parse the resulting responses, let’s turn our attention to device configuration.

Configuration

PyEZ provides a Config class which simplifies the process of loading and committing configuration changes to a Junos device. In addition, PyEZ integrates with the Jinja2 templating engine to simplify the process of creating the actual configuration snippet. Finally, PyEZ offers utilities for comparing configurations, rolling back configuration changes, and locking or unlocking the configuration database.

The first step in using the Config class is creating an instance variable. Here’s an example of creating a configuration instance variable using the already-opened r0 device instance variable:

>>> from jnpr.junos.utils.config import Config
>>> r0_cu = Config(r0)
>>> r0_cu
jnpr.junos.utils.Config(r0)

Alternatively, the configuration instance can be bound as a property of the device instance:

>>> from jnpr.junos.utils.config import Config
>>> r0.bind(cu=Config)
>>> r0.cu
jnpr.junos.utils.Config(r0)

For the rest of this section, we will use this r0.cu device property syntax. However, there’s nothing special about the name of the device property. In our examples, we have chosen the name cu, but any valid Python variable name (that isn’t already a property of the device instance) could be used in its place. Let’s begin by loading configuration snippets into the device’s candidate configuration.

Loading Configuration Changes

The load() method can be used to load a configuration snippet, or full configuration, into the device’s candidate configuration. The configuration can be specified in text (aka “curly brace”), set, or XML syntax. Alternatively, the configuration may be specified as an lxml.etree.Element object.

Configuration snippets in text, set, or XML syntax can be loaded from a file on the automation host or a Python string object. Files are specified using the path argument to the load() method. Strings are passed as the first unnamed argument to the load() method. The load() method attempts to determine the format of the configuration content automatically. The format of a configuration string is determined by the string’s content. The format of a configuration file is determined by the path’s filename extension, per Table 4-6. In either case, the automatic format can be overridden by setting the format argument to text, set, or xml.

Table 4-6. Mapping path filename extensions to configuration format
Path filename extension	Configuration format
.conf, .text, or .txt	Text in “curly brace” configuration format
.set	Text in set configuration format
.xml	Text in XML configuration format

The load() method’s overwrite and merge Boolean flags control how the new configuration affects the current configuration. The default behavior is equivalent to the load replace CLI configuration mode command. If the overwrite flag is set, the behavior is equivalent to load override, and if the merge flag is set the behavior is equivalent to the load merge command. The overwrite and merge flags are mutually exclusive. You cannot set both at the same time. In addition, you cannot set the overwrite flag when the configuration is in set format.

Here’s a simple example of changing the device’s hostname by passing a configuration string in set format:

>>> from lxml import etree
>>> new_hostname = "set system host-name r0.new" 
>>> result = r0.cu.load(new_hostname)
>>> etree.dump(result)
<load-configuration-results>
<ok/>
</load-configuration-results>

>>>

The load() method returns an lxml.etree.Element object indicating the result.

Here are several additional examples of using the load() method:

# load merge set
result = r0.cu.load(path='hostname.set', merge=True)

# load override
result = r0.cu.load(path='new_config.txt', overwrite=True)

# load merge
result = r0.cu.load(path='hostname.conf', merge=True)

# load replace xml
result = r0.cu.load(path='hostname.xml')

If an error occurs, a jnpr.junos.exception.ConfigLoadError exception is raised:

>>> r0.cu.load('bad config syntax', format='set')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/jnpr/junos/utils/config.py", l...
    return try_load(rpc_contents, rpc_xattrs)
  File "/usr/local/lib/python2.7/dist-packages/jnpr/junos/utils/config.py", l...
    raise ConfigLoadError(cmd=err.cmd, rsp=err.rsp, errs=err.errs)
jnpr.junos.exception.ConfigLoadError: ConfigLoadError(severity: error, bad_el...
>>>

By default, the load() method modifies the shared configuration database. The equivalent of the configure exclusive CLI command can be achieved by first calling the lock() method. The lock() method returns True if the configuration is successfully locked, and raises a jnpr.junos.exception.LockError exception if the configuration database lock fails because the configuration is already locked:

>>> r0.cu.lock()
True
>>> result = r0.cu.load(path='hostname.xml')
>>> r0.cu.pdiff()

[edit system]
-  host-name r0;
+  host-name r0.new;

>>> r0.cu.unlock()
True
>>> r0.cu.pdiff()
None
>>>

Notice the unlock() method can be used to unlock the configuration database and discard the candidate configuration changes. The pdiff() method displays configuration difference and is used for debugging. It is explained further in “Viewing Configuration Differences”.

Note

The equivalent of the configure private CLI command can be achieved by calling the open-configuration and close-configuration RPCs. An example is shown in “Build, Apply, and Commit the Candidate Configuration”.

Configuration Templates

Templates allow large blocks of configuration to be generated with minimal effort. PyEZ utilizes the Jinja2 templating engine to generate configuration files from templates. Jinja2 is a popular, open source Python library from the Pocoo team. For more detailed documentation on Jinja2, you can visit the Jinja2 website.

Jinja2 templates combine a reusable text form with a set of data used to fill in the form and render a result. Jinja2 offers variable substitution, conditionals, and simple loop constructs. In effect, Jinja2 is its own “mini programming language.” Jinja2 templates are not unique to Junos; they allow any text file to be built from a “template” or “form” and set of data. Any text-based configuration file, including Junos configuration files in curly brace, set, or even XML syntax, can be generated from a Jinja2 template.

Let’s look at a very simple template example that configures a hostname on the router based on a template. In this example, the configuration is expressed using a set statement. It could just as easily have been expressed in text or XML format. Here is the configuration set statement to configure a hostname of r0.new:

set system host-name r0.new

Save this statement in a file named hostname.set in the current working directory.

As you saw in the previous section, a configuration file is loaded into the device’s candidate configuration with the load() method:

>>> result = r0.cu.load(path='hostname.set')
>>>

Jinja2 uses double braces to indicate an expression, and the simplest Jinja2 expression is just the name of the variable:

{{ variable_name }}

A Jinja2 variable expression evaluates to the variable’s value. Jinja2 variable names follow the same syntax rules as Python variable names.

Apply this syntax to the one-line hostname.set file. Replace the hardcoded r0.new hostname with a reference to the host_name variable. The updated hostname.set file is now:

set system host-name {{ host_name }}

As you just saw, a configuration file is loaded into the device’s candidate configuration by passing the path argument to the load() method. In a similar fashion, a templated configuration is loaded into the device’s candidate configuration by passing the template_path argument to the load() method. However, templates also require an additional argument to the load() method. The template_vars argument takes a dictionary as its value. Each variable in the Jinja2 template must be a key in this dictionary. Here’s an example that uses the hostname.set template to configure the hostname foo:

>>> result = r0.cu.load(template_path='hostname.set',
...                     template_vars= {'host_name' : 'foo'})
>>> r0.cu.pdiff()

[edit system]
-  host-name r0;
+  host-name foo;

>>>

The template_path argument may specify an absolute or relative filename. Relative filenames are searched against the current working directory, and a templates sub-directory of the module path.⁷ Just like with the path argument, the filename extension of the template_path argument determines the expected format of the template. The same filename extension–to–configuration format mapping specified in Table 4-6 also applies to the template_path.

Now that you’ve seen the basics of creating and loading a template, let’s look at some additional Jinja2 syntax. First, an expression can contain zero or more filters separated by the | character. Similar to a Unix pipeline or Junos pipe (|) display filters, Jinja2 filters are sets of functions that can be chained to modify an expression. Jinja2 filters modify the variable at the beginning of an expression.

For example, you can use a Jinja2 filter to provide a default value for a variable. Observe what happens with the previous template when no host_name key is present in the template_vars argument to the load() method:

>>> result = r0.cu.load(template_path='hostname.set',tempate_vars={})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/jnpr/junos/utils/config.py", l...
    return try_load(rpc_contents, rpc_xattrs)
  File "/usr/local/lib/python2.7/dist-packages/jnpr/junos/utils/config.py", l...
    raise ConfigLoadError(cmd=err.cmd, rsp=err.rsp, errs=err.errs)
jnpr.junos.exception.ConfigLoadError: ConfigLoadError(severity: error, bad_el...

The missing host_name variable results in the generation of an incomplete configuration set statement, which causes a jnpr.junos.exception.ConfigLoadError exception to be raised. In this case, an error might be an appropriate response to indicate to the template’s user that host_name is a mandatory variable. However, in other cases, it’s better to provide a default value when a variable is missing. The Jinja2 provided default() filter can be used for this purpose.

Here the default() filter is used to provide a default value of Missing.Hostname. The Jinja2 lower filter is also used to lowercase the hostname:

set system host-name {{ host_name | default('Missing.Hostname') | lower }}

Now look at the result of omitting the host_name key from the template_vars argument:⁸

>>> result = r0.cu.load(template_path='hostname.set')
>>> r0.cu.pdiff()

[edit system]
-  host-name r0;
+  host-name missing.hostname;

>>>

The hostname missing.hostname is configured as a result of chaining the default() and lower filters.

Specifying a hostname of Foo in template_vars results in a hostname of foo being configured:

>>> result = r0.cu.load(template_path='hostname.set',
...                     template_vars= {'host_name' : 'Foo'})
>>> r0.cu.pdiff()

[edit system]
-  host-name r0;
+  host-name foo;

>>>

Consult the Jinja2 template documentation for a list of the built-in filters available. It is also possible to write your own Python function that operates as a custom filter. Reference the Jinja2 API documentation for more details on custom filters.

In addition to variable substitution and filters, Jinja2 offers tags that control the logic of template rendering. Tags are enclosed in {% tag %} delimiters. One such tag is the conditional if statement. An if statement allows different content to be included in the rendered file depending on whether an expression evaluates to true or false.

The following example demonstrates configuring the device’s hostname depending on whether or not the device has dual routing engines. If two routing engines are present, the hostname is configured within the special re0 and re1 configuration groups and has re0. or re1. prepended. If the device has a single routing engine, the hostname is configured at the [edit system] configuration hierarchy level as before. This time, the configuration is specified in text (aka curly brace) syntax and saved into the file hostname.conf in the current working directory. Here is the content of hostname.conf:

{% if dual_re %}
groups {
    re0 {
        system {
            host-name {{ "re0." + host_name }};
        }
    }
    re1 {
        system {
            host-name {{ "re1." + host_name }};
        }
    }
}
{% else %}
system {
    host-name {{ host_name }};
}
{% endif %}

Notice the template begins with an {% if expression %} statement, which is dependent on the value of the dual_re variable. The {% else %} statement marks the end of the text that will be rendered if dual_re is true; it also marks the beginning of the text that will be rendered if dual_re is false. The {% endif %} statement closes the conditional block.

Also notice the expression used to render the hostname in the dual-RE case. It uses a static string and Python’s + string concatenation character to prepend the appropriate value to the host_name variable.

Here’s an example of rendering this template on a device with dual REs. Notice the value of the dual_re key is set from the value of r1.facts['2RE'] supplied by PyEZ’s default facts gathering:

>>> r1.facts['2RE']
True
>>> result = r1.cu.load(template_path='hostname.conf',
...                     template_vars= { 'dual_re' : r1.facts['2RE'],
...                                      'host_name' : 'r1' })
>>> r1.cu.pdiff()

[edit groups re0 system]
+   host-name re0.r1;
[edit groups re1 system]
+   host-name re1.r1;

>>>

The same template applied to a device with a single RE sets the hostname at the [edit system] configuration hierarchy level:

>>> r0.facts['2RE']
False
>>> result = r0.cu.load(template_path='hostname.conf',
...                     template_vars= { 'dual_re' : r0.facts['2RE'],
...                                      'host_name' : 'r0' })
>>> r0.cu.pdiff()

[edit system]
+  host-name r0;

>>>

Notice the dual_re key is again supplied from the facts information. Using a template with a conditional block generates the correct configuration for the device, automatically generated based on whether or not dual routing engines are present.

While Jinja2 templates offer a plethora of additional capabilities, let’s wrap up this section by looking at the {% for scalar_var in list_var %} looping construct. In this example, the loop actually iterates over the items in a dictionary and uses the dictionary key and values to configure a set of IPv4 addresses on a set of interfaces. This time, the configuration is specified in XML syntax and saved into the file interface_ip.xml in the current working directory. Here is the content of interface_ip.xml:

<interfaces>
{% for key, value in interface_ips.iteritems() %}
    <interface>
        <name>{{ key }}</name>
        <unit>
            <name>0</name>
            <family>
                <inet>
                    <address>{{ value }}</address>
                </inet>
            </family>
        </unit>
    </interface>
{% endfor %}
</interfaces>

The content of the for loop is very similar to a Python for statement. In fact, this example uses the iteritems() dictionary method to iterate over each key/value pair in the dictionary. In Jinja2 syntax, the loop ends with a {% endfor %} statement.

Now create an interface_ips dictionary with a set of interface names as the keys and a set of IPv4 addresses and prefix lengths as the values:

>>> interface_ips = {
...     'ge-0/0/0' : '10.0.4.1/30',
...     'ge-0/0/1' : '10.0.2.1/30',
...     'ge-0/0/2' : '10.0.3.1/30',
...     'ge-0/0/3' : '10.0.5.1/30' }

Let’s use this dictionary and template to configure IP addresses on r0. The interface_ips dictionary becomes the value of the interface_ips key in the template_vars argument:

>>> result = r0.cu.load(template_path='interface_ip.xml',
...                     template_vars= { 'interface_ips' : interface_ips })
>>> r0.cu.pdiff()

[edit interfaces ge-0/0/0 unit 0 family inet]
+       address 10.0.4.1/30;
[edit interfaces ge-0/0/1 unit 0 family inet]
+       address 10.0.2.1/30;
[edit interfaces ge-0/0/2 unit 0 family inet]
+       address 10.0.3.1/30;
[edit interfaces ge-0/0/3 unit 0 family inet]
+       address 10.0.5.1/30;

>>>

As expected, the resulting configuration adds the correct addresses on multiple interfaces using the for loop.

Jinja2 templates offer many additional features that we are unable to cover directly in this book. Take a look at the Jinja2 documentation and experiment with the features that might be applicable in your particular environment.

Viewing Configuration Differences

You’ve seen the pdiff() method used in previous examples to display the differences that have been configured in the candidate configuration. The pdiff() method is intended for debugging purposes, which is how we’ve been using it in those examples. It simply prints the differences between the candidate configuration and a rollback configuration. By default, the pdiff() method compares the candidate configuration to the rollback 0 configuration. The rollback 0 configuration is the same as the currently committed configuration:

>>> r0.cu.pdiff()

[edit system]
-  host-name r0;
+  host-name r0.new;

>>>

Rather than printing the differences, the diff() method returns a string containing the differences between the candidate and rollback configurations:

>>> r0.cu.diff()
'\n[edit system]\n-  host-name r0;\n+  host-name r0.new;\n'

Both the diff() and pdiff() methods take an unnamed optional argument. The argument is an integer representing the rollback ID:

>>> r0.cu.pdiff(1)

[edit system]
-  host-name r0;
+  host-name r0.new;
[edit system login]
+    user bar {
+        uid 2002;
+        class super-user;
+        authentication {
+            encrypted-password "$5$0jDrGxJT$PXj0TWwtu5LPJ4Nvlc1YpmCKy7yAwOUH...
+        }
+    }
+    user foo {
+        uid 2003;
+        class super-user;
+        authentication {
+            encrypted-password "$5$WfsXdd11$4LXuWOIwQA5HsWvF8oxMZGyzodIHsnZP...
+        }
+    }

>>>

Now that we’ve seen how to view configuration differences, let’s look at how to commit the modified candidate configuration.

Committing Configuration Changes

The PyEZ Config class provides a commit_check() method to validate the candidate configuration and a commit() method to commit the changes. The commit_check() method returns True if the candidate configuration passes all commit checks, and raises a jnpr.junos.exception.CommitError exception if the candidate configuration fails any commit checks, including warnings:

>>> r0.cu.load("set system host-name r0")
<Element load-configuration-results at 0x7f6503180ea8>
>>> r0.cu.commit_check()
True
>>> r0.cu.load("set protocols bgp export FooBar")
<Element load-configuration-results at 0x7f650317d488>
>>> r0.cu.commit_check()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/jnpr/junos/utils/config.py", l...
    raise CommitError(cmd=err.cmd, rsp=err.rsp)
jnpr.junos.exception.CommitError: CommitError(edit_path: [edit groups junos-d...
>>>

The commit() method takes several optional arguments, which are detailed in Table 4-7. It returns True if the candidate configuration is successfully committed and raises a jnpr.junos.exception.CommitError exception if there is an error or warning when committing the configuration.

Table 4-7. Arguments to the commit() method
Argument	Description
`comment`	The value is a comment string describing the commit.
`confirm`	The value is an integer specifying the number of minutes to wait for a confirmation of the commit. The commit is confirmed by invoking `commit()` again before the `confirm` timer expires.
`sync`	A Boolean flag. If `True`, performs a `commit synchronize`, which commits the new configuration on both routing engines of a dual-RE system. It is possible that a `commit synchronize` will happen anyway if the user has configured the `synchronize` statement at the `[edit system commit]` configuration hierarchy level.
`detail`	A Boolean flag. If `True`, the `commit()` method returns an `lxml.etree.Element` object with additional details about the commit process. This argument should only be used for debugging purposes.
`force_sync`	A Boolean flag. If `True`, performs a `commit synchronize force`. This argument should only be used for debugging purposes.
`full`	A Boolean flag. If `True`, all Junos daemons are notified and reparse their full configuration, even if no configuration changes have been made that affect the daemons. This argument should only be used for debugging purposes.

Here is an example of both a successful and a failed commit:

>>> r0.cu.load("set system host-name r0")
<Element load-configuration-results at 0x7f650317b5a8>
>>> r0.cu.commit(comment='New hostname',sync=True)
True
>>> r0.cu.load("set protocols bgp export FooBar")
<Element load-configuration-results at 0x7f65031765f0>
>>> r0.cu.commit()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/jnpr/junos/utils/config.py", l...
    raise CommitError(cmd=err.cmd, rsp=err.rsp)
jnpr.junos.exception.CommitError: CommitError(edit_path: [edit groups junos-d...
>>>

Note

If the configuration database has been locked with the lock() method, it is not unlocked by calling commit(). You must still invoke the unlock() method to release the configuration lock.

Using the Rescue Configuration

The Config class also provides a rescue() method for performing actions on the Junos device’s rescue configuration. The rescue configuration is intended to aid device recovery in the event of an erroneous configuration change. The idea is that the user defines a minimal configuration needed to restore the device to a known good state and saves that minimal configuration as the “rescue” configuration. The rescue() method takes an unnamed argument that indicates the action to be taken on the rescue configuration. The valid values of this argument are get, save, delete, and reload. If the action is get, a second optional named argument, format, may be specified. The default format is text, which indicates the configuration is in the text (aka curly brace) syntax. The format argument may also be set to xml, which retrieves the configuration as an lxml.etree.Element object.

The following example demonstrates deleting, saving, getting, and then reloading the rescue configuration using the existing r0.cu device configuration attribute:

>>> r0.cu.rescue('delete')
True
>>> r0.cu.rescue('save')
True
>>> r0.cu.rescue('get',format='xml')
<Element rescue-information at 0x7f885ac76128>
>>> resp = r0.cu.rescue('reload')
>>> from lxml import etree
>>> etree.dump(resp)
<load-configuration-results>
<ok/>
</load-configuration-results>

>>>

Notice the save and delete actions return a Boolean value indicating success or failure. The save action saves the current committed configuration as the rescue configuration, while the delete action deletes the current rescue configuration. The get action returns the current rescue configuration. In the preceding example this configuration is returned as an lxml.etree.Element object because format='xml' was specified. If the format argument had not been specified, a string containing the rescue configuration would have been returned instead. The reload action loads the rescue configuration into the candidate configuration. It returns the same response as the load() method. Like the load () method, rescue('reload') only modifies the candidate configuration. A commit() must be issued to activate the rescue configuration.

Utilities

PyEZ also supplies a set of utility methods that can be used to perform common tasks on the Junos operating system. These tasks provide filesystem access or access to the Junos Unix-level shell, perform secure copies, and execute Junos software upgrades. The filesystem utilities defined in the FS class of the jnpr.junos.utils.fs module provide common commands that access the filesystem on the Junos device. The jnpr.junos.utils.scp module defines an SCP class for performing secure copies to or from the Junos device. The jnpr.junos.utils.start_shell module provides a StartShell class that allows an SSH connection to be initiated to a Junos device. Additional StartShell methods are provided to execute commands over the SSH connection and wait for an expected response. Finally, the SW class in the jnpr.junos.utils.sw module provides a set of methods for upgrading the Junos software on a device as well as rebooting or powering off the device.

Because these utilities are outside the base functionality of PyEZ, and because they are being expanded with each new PyEZ release, this book does not attempt to cover each utility in detail. Instead, you are encouraged to use Python’s built-in help() function to display the documentation strings for these classes. Here’s an example of displaying the documentation strings for the FS class:

>>> from jnpr.junos.utils.fs import FS
>>> help(FS)
Help on class FS in module jnpr.junos.utils.fs:

class FS(jnpr.junos.utils.util.Util)
 |  Filesystem (FS) utilities:
 |  
 |  * :meth:`cat`: show the contents of a file
 |  * :meth:`checksum`: calculate file checksum (md5,sha256,sha1)
 |  * :meth:`cp`: local file copy (not scp)
 |  * :meth:`cwd`: change working directory
 |  * :meth:`ls`: return file/dir listing
 |  * :meth:`mkdir`: create a directory
 |  * :meth:`pwd`: get working directory
 |  * :meth:`mv`: local file rename
 |  * :meth:`rm`: local file delete
 |  * :meth:`rmdir`: remove a directory
 |  * :meth:`stat`: return file/dir information
 |  * :meth:`storage_usage`: return storage usage
 |  * :meth:`storage_cleanup`: perform storage storage_cleanup
...

The same recipe can be followed to display the documentation strings for the other PyEZ utility classes.

A PyEZ Example

Now that you’ve seen the capabilities of PyEZ, it is time to put this knowledge into practice by rewriting the lldp_interface_descriptions_rest.py example covered in “Using the RESTful APIs in Python”. This example uses PyEZ’s RPC on Demand feature to query the current LLDP neighbors and interface descriptions. It handles responses using both the lxml and jxmlease libraries. The new configuration is applied using a Jinja2 template.

The purpose and overall architecture of the script closely follows those of the previous lldp_interface_descriptions_rest.py example. The new script is named lldp_interface_descriptions_pyez.py and uses PyEZ to discover and monitor the network topology of one or more Junos devices running the Link Layer Discovery Protocol (LLDP). Information discovered from LLDP is stored in the interface description fields of the device’s configuration. The current LLDP-discovered information is compared against the previous information that is stored in the interface description field, and the user is notified of any changes in LLDP state (up, change, or down) since the previous snapshot.

The script is invoked by a user at the command line and takes one or more device names or IP addresses as command-line arguments. The syntax is:

user@h0$ python lldp_interface_descriptions_pyez.py r0 r1 r2 r3 r4 r5

In order to execute successfully, the NETCONF-over-SSH service should be configured on each device, and a common username and password with appropriate authorization should be configured on each device:

user@r0> show configuration system services 
netconf {
    ssh;
}

user@r0> show configuration system login                       
user user {
    uid 2001;
    class super-user;
    authentication {
        encrypted-password "$1$jCvocDbA$KeOycEvIDtSV/VOdPRHo5."; ## SECRET-DATA
    }
}

The script prompts for the username and password to use to connect to the devices and then prints its output to the user’s terminal. It prints a notification for each device being checked. If the script detects any changes in LLDP state since the last snapshot, those changes are printed to the terminal. The new interface descriptions are configured and a message indicates whether or not the device’s configuration was successfully updated. The following example shows a sample output from the script:

user@h0$ ./lldp_interface_descriptions_pyez.py r0 r1 r2 r3 r4 r5
Device Username: user
Device Password: 
Connecting to r0...
Getting LLDP information from r0...
Getting interface descriptions from r0...
    ge-0/0/4 LLDP Change. Was: r7 ge-0/0/6 Now: r1 ge-0/0/0
    ge-0/0/3 LLDP Up. Now: r5 ge-0/0/0
    ge-0/0/2 LLDP Up. Now: r3 ge-0/0/2
    ge-0/0/0 LLDP Up. Now: r4 ge-0/0/2
    ge-0/0/5 LLDP Down. Was: r6 ge-0/0/8
    Successfully committed configuration changes on r0.
    Closing connection to r0.
Connecting to r1...
Getting LLDP information from r1...
Getting interface descriptions from r1...
    ge-0/0/2 LLDP Down. Was: r2 ge-0/0/0
    Successfully committed configuration changes on r1.
    Closing connection to r1.
Connecting to r2...
Getting LLDP information from r2...
Getting interface descriptions from r2...
    ge-0/0/2 LLDP Down. Was: r0 ge-0/0/1
    ge-0/0/1 LLDP Down. Was: r3 ge-0/0/0
    ge-0/0/0 LLDP Down. Was: r1 ge-0/0/2
    Successfully committed configuration changes on r2.
    Closing connection to r2.
Connecting to r3...
Getting LLDP information from r3...
Getting interface descriptions from r3...
    ge-0/0/0 LLDP Down. Was: r2 ge-0/0/1
    Successfully committed configuration changes on r3.
    Closing connection to r3.
Connecting to r4...
Getting LLDP information from r4...
Getting interface descriptions from r4...
    No LLDP changes to configure on r4.
    Closing connection to r4.
Connecting to r5...
Getting LLDP information from r5...
Getting interface descriptions from r5...
    ge-0/0/2 LLDP Up. Now: r4 ge-0/0/0
    ge-0/0/0 LLDP Up. Now: r0 ge-0/0/3
    Successfully committed configuration changes on r5.
    Closing connection to r5.
user@h0$

As the script sequentially loops through each device specified on the command line, it performs the following steps:

Gather LLDP neighbor information.
Gather interface descriptions; parse the LLDP neighbor information that was previously stored in the interface descriptions.
Compare current and previous LLDP neighbor information; print LLDP up, change, and down messages; calculate new interface descriptions.
Build, load, and commit a candidate configuration with updated interface descriptions.

Note

Like the RESTful API example script, this example tries to provide a useful and realistic function while also concentrating on code that demonstrates PyEZ. The same caveats apply. Specifically, the example script is invoked by the user at the command line, but a more complex program might be invoked by an event or on a schedule. In addition, the output might integrate with an existing network monitoring or alerting system rather than simply being printed to the terminal. A more realistic implementation would store the LLDP information gathered from the device in an off-device database or in on-device apply-macro configuration statements rather than in the interface description configuration statements. Finally, each device could be queried and configured in parallel to speed the program’s execution.

Let’s analyze the Python code used to perform each of these steps. Again, we recommend following along and typing each line of the program listings into your own script file named lldp_interface_descriptions_pyez.py. After completing “Putting It All Together”, you will have a working example to execute against your own network.

The Preamble

The first step in our example script is to import required libraries and perform some one-time initialization. The callouts give more information on each line of the program listing:

#!/usr/bin/env python 
"""Use interface descriptions to track the topology reported by LLDP.

This includes the following steps:
1) Gather LLDP neighbor information.
2) Gather interface descriptions.
3) Parse LLDP neighbor information previously stored in the descriptions.
4) Compare LLDP neighbor info to previous LLDP info from the descriptions.
5) Print LLDP Up / Change / Down events.
6) Store the updated LLDP neighbor info in the interface descriptions.

Interface descriptions are in the format:
[user-configured description ]LLDP: <remote system> <remote port>[(DOWN)]

The '(DOWN)' string indicates an LLDP neighbor which was previously
present, but is now not present.
"""

import sys 
import getpass

import jxmlease 

from jnpr.junos import Device 
from jnpr.junos.utils.config import Config
import jnpr.junos.exception

TEMPLATE_PATH = 'interface_descriptions_template.xml' 

# Create a jxmlease parser with desired defaults.
parser = jxmlease.EtreeParser() 

class DoneWithDevice(Exception): pass

: The #! line (sometimes called the hashbang or shebang line) allows the option of running the script without specifying the python command. (This mechanism works on Unix-like platforms and on Windows platforms using the Python Launcher for Windows.) In other words, you could execute the script with path/lldp_interface_descriptions_pyez.py r0 r1 r2 r3 r4 r5 instead of python path/lldp_interface_descriptions_pyez.py r0 r1 r2 r3 r4 r5. Executing the script directly does require executable permissions to be set on the lldp_interface_descriptions_pyez.py file. Using the /usr/bin/env shell command to invoke Python means the script is not dependent on the location of the python command.
: Two standard Python modules, sys and getpass, are imported. The sys module provides access to objects maintained by the Python interpreter, and the getpass module allows the script to interactively prompt for the required password without echoing the input to the user’s terminal.
: The jxmlease library parses XML into native Python data structures. You must ensure this module is installed on your system.
: Three PyEZ modules are imported. The PyEZ Device class is used to create a device instance. This is the basic PyEZ class for interacting with a Junos device. The PyEZ Config class offers methods for dealing with the Junos device configuration. The exception module defines several PyEZ-specific exceptions that may be raised to indicate a potential problem.
: The TEMPLATE_PATH variable is used as a constant to identify the name of the Jinja2 configuration template used to generate the configuration of new interface descriptions.
: A jxmlease parser instance is created for parsing lxml.etree.Element objects into Python data structures. The jxmlease.EtreeParser() method creates an instance of the jxmlease.EtreeParser class with a set of default parameters. Whereas a parser created with the jxmlease.Parser() method expects an XML input document as a string, an instance of the jxmlease.EtreeParser class expects an XML input document as an lxml.etree.Element object.
: A custom DoneWithDevice class is created for indicating when processing on each device in the main device loop has been completed. This new class is a subclass of Exception.

Each of the script’s major steps has been encapsulated into a Python function which will be executed from the main() function of the lldp_interface_descriptions_pyez.py file.

Loop Through Each Device

Let’s begin by analyzing the main() function, which prompts for a username and password and then loops over each device specified on the command line. The main() function calls several functions, which are each analyzed in later sections:

def main():
    """The main loop.

    Prompt for a username and password.
    Loop over each device specified on the command line.
    Perform the following steps on each device:
    1) Get LLDP information from the current device state.
    2) Get interface descriptions from the device configuration.
    3) Compare the LLDP information against the previous snapshot of LLDP
       information stored in the interface descriptions. Print changes.
    4) Build a configuration snippet with new interface descriptions.
    5) Commit the configuration changes.

    Return an integer suitable for passing to sys.exit().
    """

    if len(sys.argv) == 1: 
        print("\nUsage: %s device1 [device2 [...]]\n\n" % sys.argv[0])
        return 1

    rc = 0 

    # Get username and password as user input.
    user = raw_input('Device Username: ')
    password = getpass.getpass('Device Password: ') 

    for hostname in sys.argv[1:]: 
        try:
            print("Connecting to %s..." % hostname)
            dev = Device(host=hostname, 
                         user=user,
                         password=password,
                         normalize=True)
            dev.open() 

            print("Getting LLDP information from %s..." % hostname)
            lldp_info = get_lldp_neighbors(device=dev) 
            if lldp_info == None: 
                print("    Error retrieving LLDP info on " + hostname +
                      ". Make sure LLDP is enabled.")
                rc = 1
                raise DoneWithDevice

            print("Getting interface descriptions from %s..." % hostname)
            desc_info = get_description_info_for_interfaces(device=dev) 
            if desc_info == None:
                print("    Error retrieving interface descriptions on %s." %
                      hostname)
                rc = 1
                raise DoneWithDevice

            desc_changes = check_lldp_changes(lldp_info, desc_info) 
            if not desc_changes:
                print("    No LLDP changes to configure on %s." % hostname)
                raise DoneWithDevice

            if load_merge_template_config( 
                device=dev,
                template_path=TEMPLATE_PATH,
                template_vars={'descriptions': desc_changes}):
                print("    Successfully committed configuration changes on %s." %
                      hostname)
            else:
                print("    Error committing description changes on %s." %
                      hostname)
                rc = 1
                raise DoneWithDevice
        except jnpr.junos.exception.ConnectError as err: 
            print("    Error connecting: " + repr(err))
            rc = 1
        except DoneWithDevice:
            pass
        finally: 
            print("    Closing connection to %s." % hostname)
            try:
                dev.close()
            except:
                pass
    return rc

: The script requires that at least one device be specified as a command-line argument. The sys.argv list will contain the name of the script at index 0. User-specified arguments begin at index 1. If no user-specified arguments are present (len(sys.argv) == 1), the usage message is printed and the script exits with a status code of 1 to indicate an error.
: The rc variable holds the status code to be returned at the end of the script. The value is initialized to 0 which indicates success. Later, if an error is encountered, rc will be set to 1 and processing will continue with the next device specified on the command line.
: This statement and the previous statement prompt the user for the username and password used to establish the NETCONF session to the Junos devices. It is assumed each device is configured with the same user authentication credentials. The getpass() function from the Python standard library prompts the user for a password and disables echoing the response to the screen.
: This for loop iterates over every device name (or IP address) specified in the command-line arguments. Because sys.argv[0] is the name of the script, the slice sys.argv[1:] is used to return the list of devices.
: A device instance is created and assigned to the variable dev. The username and password entered by the user are passed as the user and password arguments. The normalize=True argument ensures that response normalization is applied to every RPC on Demand call using the dev device instance.
: The open() method is invoked on the dev device instance. This establishes a NETCONF session to the device and invokes PyEZ’s default facts gathering. If the open() call fails, a jnpr.junos.exception.ConnectError (or one of its subclasses) is raised. This potential exception is handled later, in the main() function.
: The get_lldp_neighbors() function is invoked on the current device instance, dev. The result is stored in the lldp_info dictionary.
: If get_lldp_neighbors() reports an error (by returning the value None), an error message is printed, rc is set to 1 to indicate the error, and the DoneWithDevice exception is raised. This exception causes execution to jump to the except DoneWithDevice line toward the end of the main() function.
: The get_description_info_for_interfaces()function is executed on the current device, dev. The result is stored in the desc_info dictionary. Similar to the get_lldp_neighbors() function, if the get_description_info_for_interfaces() function reports an error (by returning the value None), an error message is printed, rc is set to 1 to indicate the error, and the DoneWithDevice exception is raised. This exception causes execution to jump to the except DoneWithDevice line toward the end of the main() function.
: The get_lldp_description_changes() function parses the lldp_info and desc_info dictionaries and returns the new interface descriptions in the desc_changes dictionary. If no descriptions have changed, the DoneWithDevice exception is raised. In this case, the exception does not indicate an error. It simply skips the section of code that loads and commits the configuration change if there is no new configuration to be applied.
: The load_merge_template_config() function is called. The template_path argument is set to the value of TEMPLATE_PATH and the descriptions key of the template_vars argument is set to the desc_changes dictionary. The return value of the function indicates whether the new configuration was successfully committed. If an error occurs, an error message is printed, rc is set to 1 to indicate the error, and DoneWithDevice is raised.
: A jnpr.junos.exception.ConnectError (or any subclass) exception indicates the NETCONF session to the device was not opened. In this case, the error is printed, rc is set to 1 to indicate the error, and the script continues on to the finally block.
: Placing the dev.close() method invocation inside a finally block ensures the NETCONF session is gracefully closed regardless of whether there was an exception raised in gathering information or committing the new configuration. Because dev.close() might raise an exception itself, the statement is placed inside a try block. The corresponding except block simply ignores any exceptions that were raised by dev.close().

Now, let’s look at each of the functions that are called from the for loop in more detail.

Gather LLDP Neighbor Information

The first function, get_lldp_neighbors(), uses the PyEZ RPC on Demand feature to query the get-lldp-neighbors-information RPC. The RPC response is the default format of lxml Element objects. The function gathers the LLDP neighbor’s system and port information for each local interface using lxml methods, and returns the information to the caller in a dictionary:

def get_lldp_neighbors(device): 
    """Get current LLDP neighbor information.

    Return a two-level dictionary with the LLDP neighbor information.
    The first-level key is the local port (aka interface) name.
    The second-level keys are 'system' for the remote system name
    and 'port' for the remote port ID. On error, return None.

    For example:
    {'ge-0/0/1': {'system': 'r1', 'port', 'ge-0/0/10'}}
    """

    lldp_info = {}
    try: 
        resp = device.rpc.get_lldp_neighbors_information()
    except (jnpr.junos.exception.RpcError,
            jnpr.junos.exception.ConnectError)as err:
        print "    " + repr(err)
        return None

    for nbr in resp.findall('lldp-neighbor-information'): 
        local_port = nbr.findtext('lldp-local-port-id') 
        remote_system = nbr.findtext('lldp-remote-system-name')
        remote_port = nbr.findtext('lldp-remote-port-id')
        if local_port and (remote_system or remote_port):
            lldp_info[local_port] = {'system': remote_system, 
                                     'port': remote_port}

    return lldp_info

: The get_lldp_neighbors() function requires a device argument. This argument is a PyEZ device instance that has an active NETCONF session open.
: The RPC on Demand call and the processing of the RPC response are surrounded by a try block to avoid unexpected script termination. If an exception is raised, an error message is printed and the value None is returned to the caller to indicate the error. The PyEZ RPC on Demand feature invokes the get-lldp-neighbors-information XML RPC.
: The lxml findall() method will return a list of all lxml.etree.Element objects that match the XPath expression lldp-neighbor-information. This results in looping through each LLDP neighbor. The nbr variable is assigned the lxml.etree.Element object representing the current LLDP neighbor.
: The lxml findtext() method selects the first XML element matching an XPath expression. It is used to select the values of the local_port, remote_system, and remote_port variables.
: The remote_system and remote_port values are stored in the lldp_info dictionary. This dictionary is keyed on the local_port value extracted from the current LLDP neighbor.

Gather and Parse Interface Descriptions

The next function, get_description_info_for_interfaces(), makes another PyEZ RPC on Demand query using the get-interface-information RPC. The descriptions parameter is added to the RPC in order to retrieve only the interface descriptions. This time, the output is parsed by the jxmlease library to produce a jxmlease.XMLDictNode object. The content of the interface description field is then parsed from this response based on a simple convention that has been chosen for this example script. Refer back to “Gather and Parse Interface Descriptions” if you need a refresher on the convention used. The function is defined as follows:

def get_description_info_for_interfaces(device): 
    """Get current interface description for each interface.

    Parse the description into the user-configured description, remote
    system, and remote port components.

    Return a two-level dictionary. The first-level key is the
    local port (aka interface) name. The second-level keys are
    'user_desc' for the user-configured description, 'system' for the
    remote system name, 'port' for the remote port, and 'down', which is
    a Boolean indicating if LLDP was previously down. On error, return None.

    For example:
    {'ge-0/0/1': {'user_desc': 'test description', 'system': 'r1',
                  'port': 'ge-0/0/10', 'down': True}}
    """

    desc_info = {}
    try: 
        resp = parser(device.rpc.get_interface_information(descriptions=True))
    except (jnpr.junos.exception.RpcError,
            jnpr.junos.exception.ConnectError) as err:
        print "    " + repr(err)
        return None

    try:
        pi = resp['interface-information']['physical-interface'].jdict() 
    except KeyError:
        return desc_info

    for (local_port, port_info) in pi.items(): 
        try:
            (udesc, _, ldesc) = port_info['description'].partition('LLDP: ') 
            udesc = udesc.rstrip()
            (remote_system, _, remote_port) = ldesc.partition(' ')
            (remote_port, down_string, _) = remote_port.partition('(DOWN)')
            desc_info[local_port] = {'user_desc': udesc,
                                     'system': remote_system,
                                     'port': remote_port,
                                     'down': True if down_string else False}
        except (KeyError, TypeError): 
            pass
    return desc_info

: get_description_info_for_interfaces() requires a device argument. This argument is a PyEZ device instance that has an active NETCONF session open.
: Again, the RPC on Demand call is surrounded by a try block to handle exception conditions. The RPC on Demand feature is used to execute the get-interface-information XML RPC. The descriptions argument is passed to the RPC and specifies that only interface descriptions should be returned. The lxml.etree.Element object is then passed to the jxmlease.EtreeParser() instance, parser().
: The pi variable is assigned a dictionary based on the physical interface information in the RPC response. The jdict() method (discussed in “jxmlease objects”) is used to return a dictionary from the physical interface information. The jdict() method automatically produces a dictionary keyed on the value of the <name> elements in each <pyhsical-interface> element of the response. A KeyError exception indicates that either the <interface-information> or the <physical-interface> element is missing from the response. This simply indicates that no interface descriptions are currently configured.
: The for loop iterates over the local_port, the key to the pi dictionary, and the port_info, the value of the pi dictionary.
: The existing interface description is parsed into components by the next several lines. The descriptions are stored in the desc_info dictionary, which is keyed on the local_port value. Reference “Gather and Parse Interface Descriptions” in the RESTful API example script if this code is unclear.
: When accessing the data, a KeyError exception is raised when the specified key does not exist. A TypeError exception is raised when the type being accessed doesn’t match the data structure. If either of these conditions occurs, the exception is ignored. Processing continues with the next local_port in the pi dictionary.

Compare Current and Previous LLDP Neighbor Information

The check_lldp_changes() function compares the previous LLDP information found in the description fields and now stored in the desc_info dictionary to the LLDP information now stored in the lldp_info dictionary. Because this function operates solely on the desc_info and lldp_info dictionaries returned by the previous functions, its content is exactly the same as the RESTful API version. The function is included here without repeating the explanations of each line. If you’re unsure of the purpose or behavior of any line, refer back to the explanations in “Compare Current and Previous LLDP Neighbor Information”. The function definition is as follows:

def check_lldp_changes(lldp_info, desc_info):
    """Compare current LLDP info with previous snapshot from descriptions.

    Given the dictionaries produced by get_lldp_neighbors() and
    get_description_info_for_interfaces(), print LLDP up, change,
    and down messages.

    Return a dictionary containing information for the new descriptions
    to configure.
    """

    desc_changes = {}

    # Iterate through the current LLDP neighbor state. Compare this
    # to the saved state as retrieved from the interface descriptions.
    for local_port in lldp_info:
        lldp_system = lldp_info[local_port]['system']
        lldp_port = lldp_info[local_port]['port']
        has_lldp_desc = desc_info.has_key(local_port)
        if has_lldp_desc:
            desc_system = desc_info[local_port]['system']
            desc_port = desc_info[local_port]['port']
            down = desc_info[local_port]['down']
            if not desc_system or not desc_port:
                has_lldp_desc = False
        if not has_lldp_desc:
            print("    %s LLDP Up. Now: %s %s" %
                  (local_port,lldp_system,lldp_port))
        elif down:
            print("    %s LLDP Up. Was: %s %s Now: %s %s" %
                  (local_port,desc_system,desc_port,lldp_system,lldp_port))
        elif lldp_system != desc_system or lldp_port != desc_port:
            print("    %s LLDP Change. Was: %s %s Now: %s %s" %
                  (local_port,desc_system,desc_port,lldp_system,lldp_port))
        else:
            # No change. LLDP was not down. Same system and port.
            continue
        desc_changes[local_port] = "LLDP: %s %s" % (lldp_system,lldp_port)

    # Iterate through the saved state as retrieved from the interface
    # descriptions. Look for any neighbors that are present in the
    # saved state, but are not present in the current LLDP neighbor
    # state.
    for local_port in desc_info:
        desc_system = desc_info[local_port]['system']
        desc_port = desc_info[local_port]['port']
        down = desc_info[local_port]['down']
        if (desc_system and desc_port and not down and
            not lldp_info.has_key(local_port)):
            print("    %s LLDP Down. Was: %s %s" %
                  (local_port,desc_system,desc_port))
            desc_changes[local_port] = "LLDP: %s %s(DOWN)" % (desc_system,
                                                              desc_port)

    # Iterate through the list of interface descriptions we are going
    # to change. Prepend the user description, if any.
    for local_port in desc_changes:
        try:
            udesc = desc_info[local_port]['user_desc']
        except KeyError:
            continue
        if udesc:
            desc_changes[local_port] = udesc + " " + desc_changes[local_port]

    return desc_changes

Build, Apply, and Commit the Candidate Configuration

The load_merge_template_config() function takes a device instance, a Jinja2 template, and template variables as its arguments. It uses RPC on Demand as well as the PyEZ configuration load() and commit() methods to perform the equivalent of the CLI commands configure private, load merge, and commit. It also checks the results for potential errors:

def load_merge_template_config(device,
                               template_path,
                               template_vars): 
    """Load templated config with "configure private" and "load merge".

    Given a template_path and template_vars, do:
        configure private,
        load merge of the templated config,
        commit,
        and check the results.

    Return True if the config was committed successfully, False otherwise.
    """

    class LoadNotOKError(Exception): pass 

    device.bind(cu=Config) 

    rc = False

    try:
        try:
            resp = device.rpc.open_configuration(private=True) 
        except jnpr.junos.exception.RpcError as err: 
            if not (err.rpc_error['severity'] == 'warning' and
                    'uncommitted changes will be discarded on exit' in
                    err.rpc_error['message']):
                raise

        resp = device.cu.load(template_path=template_path,
                              template_vars=template_vars,
                              merge=True) 
        if resp.find("ok") is None: 
            raise LoadNotOKError
        device.cu.commit(comment="made by %s" % sys.argv[0]) 
    except (jnpr.junos.exception.RpcError, 
            jnpr.junos.exception.ConnectError,
            LoadNotOKError) as err:
        print "    " + repr(err)
    except:
        print "    Unknown error occurred loading or committing configuration."
    else: 
        rc = True
    try: 
        device.rpc.close_configuration()
    except jnpr.junos.exception.RpcError as err:
        print "    " + repr(err)
        rc = False
    return rc

: The device, template_path, and template_vars arguments are required parameters of the load_merge_template_config() function.
: A new Exception subclass is defined. This class is raised, and handled, when there is a problem loading the new configuration.
: A new PyEZ configuration instance is created and bound to the device’s cu attribute.
: The open-configuration XML RPC is called by the RPC on Demand method. The private argument makes this RPC equivalent to the configure private CLI command.
: jnpr.junos.exception.RpcError exceptions are caught and handled by this except block. It is normal for the open-configuration XML RPC to return an uncommitted changes will be discarded on exit warning when the private argument is specified. This expected warning is ignored. All other jnpr.junos.exception.RpcError exceptions are raised and handled by the enclosing try/except block.
: This statement passes the template_path and template_vars arguments to the configuration instance’s load() method. The configuration is generated from the template and user-supplied values. The merge=True argument causes the new configuration to be merged with the existing candidate configuration.
: An <ok> XML element in the load() method’s XML response indicates the new configuration was loaded successfully. A LoadNotOKError exception, defined earlier in this function, is raised when the <ok> XML element is not found in the response. In most situations, an error loading the new candidate configuration will raise some subclass of the PyEZ RpcError exception. This block simply handles the possible situation where the load() method does not raise an exception and returns an unexpected RPC response.
: The configuration’s commit() method is used to commit the new candidate configuration. A comment that includes the script’s name is added to the commit with the comment argument.
: jnpr.junos.exception.RpcError, jnpr.junos.exception.ConnectError, and LoadNotOKError exceptions and subclasses are caught and handled by this except block. The contents of these exceptions are simply printed. A different message is printed for all other exceptions that may have been raised during the try block.
: This else block is executed only if no exception has been raised. This indicates the configuration has been successfully loaded and committed. In this case, rc is assigned the value of True to indicate the success. (Earlier, rc was initialized to False.)
: Regardless of whether a previous exception was raised, this try block is executed and the private candidate configuration is closed by the close-configuration RPC. If there is an error closing the configuration, the exception is printed and rc is assigned False to indicate the error.

The load_merge_template_config() function is written to be independent of the configuration being applied. By passing the device instance, Jinja2 template file, and template variables, you can use it to apply any configuration to the device. The real work of producing the configuration is done in the Jinja2 template. In this example, the template lives in the interface_descriptions_template.xml file in the current working directory. As the .xml filename extension implies, this template generates a configuration in XML format. The callouts explain each line of the template:

<configuration>
    <interfaces> 
    {% for key, value in descriptions.iteritems() %} 
        <interface> 
            <name>{{ key }}</name> 
            <description>{{ value }}</description> 
        </interface>
    {% endfor %} 
    </interfaces>
</configuration>

: This template generates a configuration in XML syntax. Only the configuration-related XML tags need to be included. The <interfaces> XML tag corresponds to the [edit interfaces] level of the Junos configuration hierarchy.
: This line is a Jinja2 for loop that iterates over the items in the descriptions dictionary. The key to the descriptions dictionary is the interface’s name, and the value is the new description to be configured.
: The opening XML tag for each interface. This tag will be repeated for each interface in the descriptions dictionary.
: The name of the interface. The {{ key }} expression evaluates to each key in the descriptions dictionary.
: The description of the interfaces. The {{ value }} expression evaluates to each value in the descriptions dictionary.
: The {% endfor %} tag identifies the end of the for loop that iterates over each interface in the descriptions dictionary.

Putting It All Together

The final block of code in the example script calls the main() function when the script is executed. After entering this code block, the example script is functional:

if __name__ == "__main__":
  sys.exit(main())

At this point, the script is complete and ready to test on a set of Junos devices running the NETCONF-over-SSH service and the LLDP protocol. If you encounter errors running the script, review and carefully compare your code with the example. The full working lldp_interface_descriptions_pyez.py file is also available on GitHub.

Limitations

Like all of the automation tools discussed in this book, PyEZ fills a very useful role in automating Junos networks, but it is not the solution to every network automation problem. The first, and most obvious, limitation is that PyEZ is Python-specific. If you’re fluent in Python or augmenting a current system written in Python, you may see this as an asset. However, if your project requires another language, there may be other options. Refer to the next section for more information on ways to access Junos devices using NETCONF in the language of your choice.

Another (almost as obvious) limitation is that PyEZ requires NETCONF. Because all currently supported Junos devices support NETCONF, this requirement isn’t a big limitation. However, it does require that the NETCONF-over-SSH service, or the standalone SSH service, be configured and reachable from your automation host. When using NETCONF over SSH, ensure that TCP destination port 830 is not being filtered along the network path between your automation host and the target Junos device.

NETCONF Libraries for Other Languages

While this book doesn’t discuss them in detail, there are various libraries for several languages that implement the NETCONF protocol and provide some level of abstraction that avoids you having to send direct NETCONF RPCs. Most of these libraries do not support the higher-level abstractions, like tables and views, that Junos PyEZ supports, but they can still be extremely useful.

Table 4-8 provides a current survey of some of these libraries. Refer to the corresponding link for more information on each library.

Table 4-8. Available NETCONF libraries
Language	Description	Link
Ruby	Popular open source NETCONF library for Ruby. Easy to install, limited dependencies, and active support.	http://rubygems.org/gems/netconf
Java	Open source NETCONF library for Java. Already in use by enterprise customers. Easy installation with zero dependencies.	http://www.juniper.net/support/downloads/?p=netconf
Perl	Supported by Juniper Networks JTAC. Oldest of the NETCONF libraries. Installation can be difficult, with multiple dependencies.	http://www.juniper.net/support/downloads/?p=netconf#sw
PHP	Open source NETCONF library for PHP. Undergoing active development and may not be ready for production use.	https://github.com/Juniper/netconf-php
Python	Vendor-agnostic open source NETCONF library for Python. Utilized by PyEZ.	https://github.com/leopoul/ncclient

These libraries can be helpful if your project requires direct NETCONF support or a specific development language; however, PyEZ is highly recommended for new Junos automation development.

Chapter Summary

In summary, Junos PyEZ provides a friendly balance between simplicity and power. This balance makes it the obvious choice for users already familiar with Python. PyEZ offers fact gathering and utility functions that simplify many common tasks, and it simplifies RPC execution by providing an abstraction layer built on top of the NETCONF protocol.

Much of PyEZ’s power involves dealing with the resulting RPC responses. For the most power and flexibility, use XPath expressions with the lxml library. For a quick and easy mapping between XML and native Python data structures, use the jxmlease library to parse the lxml response. For a reusable mapping between complex XML elements and simple Python data structures, choose the table and view mechanism.

In addition to operational RPCs, PyEZ offers tools for making configuration changes. These tools include a powerful “template building” feature that allows large and complex configurations to be generated with minimal effort.

¹ ncclient is a community-led and community-supported open source project hosted on GitHub.

² In addition to the Junos PyEZ software requirements, installing from the GitHub repository requires git to be installed.

³ The string representation of a jnpr.junos.Device class instance is simply Device(host).

⁴ It is more secure to protect the private key with a passphrase. In fact, a passphrase should be considered best practice for production network environments.

⁵ Be aware that PyEZ strips XML namespaces from the response. So, while the CLI output includes XML elements with attributes in the junos namespace, such as <up-time junos:seconds="102120">1 day, 4:22</up-time>, the attribute in the response object is seconds rather than junos:seconds.

⁶ Remember that PyEZ strips namespaces from XML responses. So, the junos:seconds attribute in the raw XML response is specified simply as seconds within PyEZ.

⁷ On the example automation host, the module path is /usr/local/lib/python2.7/dist-packages/jnpr/junos, but the location is installation-specific and may be different on your machine. You can discover the correct location for your machine using the instructions at the beginning of “Prepackaged Operational Tables and Views”.

⁸ If the template_vars argument is omitted completely, as in this example, it defaults to {}, an empty Python dictionary.

Get Automating Junos Administration now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Chapter 4. Junos PyEZ

Installation

Note

Warning

Device Connectivity

Note

Creating a Device Instance

Note

Making the Connection

Authentication and Authorization

Connection Exceptions

Note

Facts Gathering

Closing the Connection

RPC Execution

RPC on Demand

Note

Warning

RPC Parameters

RPC Timeout

RPC Exceptions

Note

RPC Responses

lxml Elements

Note

Note

Response Normalization

jxmlease

JSON

Operational Tables and Views

Prepackaged Operational Tables and Views

Note

Note

Note

Creating New Operational Tables and Views

Note

Note

Warning

Warning

Note

Note

Using the New Operational Table and View

Note

Applying a Different View

Saving and Loading XML Files from Tables

Note

Configuration

Loading Configuration Changes

Note

Configuration Templates

Viewing Configuration Differences

Committing Configuration Changes

Note

Using the Rescue Configuration

Utilities

A PyEZ Example

Note

The Preamble

Loop Through Each Device

Gather LLDP Neighbor Information

Gather and Parse Interface Descriptions

Compare Current and Previous LLDP Neighbor Information

Build, Apply, and Commit the Candidate Configuration

Putting It All Together

Limitations

NETCONF Libraries for Other Languages

Chapter Summary

Don’t leave empty-handed

It’s yours, free.

Check it out now on O’Reilly