Identity, Authentication, and Access Management in OpenStack

Chapter 4. LDAP

As OpenStack grew and started to gain attention from enterprises, one of the most common initial requests around the Identity space was for LDAP support. This was first introduced, in a basic form, in the Essex release. In this chapter, we will discuss the approach to LDAP integration, the new, comprehensive, configuration options for LDAP, using multiple domains for identity, and, finally, tips, common pitfalls, and troubleshooting.

4.1 Approach to LDAP Integration

LDAP and Microsoft’s Active Directory (AD) are two of the most common existing corporate directories in the enterprise and are used by countless corporations. Given that enterprises already use these as their identity store, it is neither desirable nor practical for this to be replicated in Keystone. Instead, we want Keystone’s Identity APIs to be a front to these corporate enterprise directories, allowing the information they hold to be accessed in-place. Keystone was designed to provide this kind of support by the very nature of its architecture. In essence, it has drivers for different backend technologies that can be loaded to allow components of Keystone to be backed by external data stores (such as LDAP). As an aside, although we will talk predominantly about LDAP in this chapter, since AD supports the LDAP protocol, everything here applies equally to AD. Where there are differences, we will point them out.

As can be seen in “1.5 Backends and Services”, the Keystone architecture is composed of a number of backends and services. Although conceptually they all could have LDAP drivers, in reality only three have such support:

• Identity: Storing users and groups
• Resource: Storing domains and projects
• Assignment: Storing roles and their assignments

In the vast majority of installations, only Identity is mapped to LDAP and usually in read-only mode, since most enterprises have separate, well-established management systems for their corporate directory. The mapping of the Resource and Assignment backends to LDAP is usually problematic; for example, there is no obvious LDAP schema that matches that needed by Keystone, and this needs to be fully read-write from a Keystone perspective. In fact, given these problems, and that so few companies attempted to use these drivers, both the Resource and Assignment support for LDAP is now deprecated in Keystone, and support is planned to be removed in the Mitaka release of OpenStack. Given this, it is highly recommended that LDAP not be used for these drivers and that enterprises only map the Keystone Identity backend to LDAP.

4.2 Configuring Keystone to Integrate with LDAP

In its basic form, Keystone LDAP integration has been included in OpenStack since the Essex release. By providing your specific LDAP options in the Keystone configuration file (keystone.conf), Keystone will translate any user and group API calls into something your LDAP server will understand. There is one huge restriction in this classic LDAP support: you can only have one domain in Keystone, which is the default domain. We’ll discuss how we use multi-domains in the newer LDAP support (introduced in Juno) later in this chapter, but it is still worthwhile to look through the classic support, since this is the underpinning on which such follow-on enhancements are built. Further, there are some installations for which a single domain is sufficient.

In looking at the task of mapping Keystone users and groups to LDAP, let’s first examine what Keystone (and the rest of OpenStack) needs. A user entity in Keystone (that is returned by a GET /users call) looks like this:

{
    "domain_id": "default",
    "enabled": true,
    "id": "ef7ab1f3fe2745fd91b3d62fd3b7309b",
    "links": {
        "self": "http://localhost:5000/v3/users/ef7ab1f3fe2745fd91b3d62fd3b7309b"
    },
    "name": "GeorgeSmiley"
}

Now, the links attribute is actually filled in by Keystone as part of formatting the entity, and, as described above, in classic LDAP we only support the default domain, so that bit’s easy. However, the other attributes need to come from the identity store (be it SQL, LDAP, or whatever)— and in fact, there is one more—password! For security reasons, this is not returned by the Keystone API—but it does need to be accessible to Keystone so that it can check an authentication request. So that’s the job of our classic LDAP support to somehow map enabled, id, name, and password to some attributes already stored in an existing LDAP database (but whose schema and exact attribute naming will vary from customer to customer). Further, since customers will already have processes for disabling a user in the corporate directory, how is it that we can ensure such a user is disabled in Keystone? And, of course, there are a similar set of attributes for the Keystone group entity as well, which also need to be mapped:

{
    "description": "A group of spies",
    "domain_id": "default",
    "id": " a2ffab1b3fe3745fdaab3d62fd3b75093",
    "links": {
        "self": "http://localhost:5000/v3/users/a2ffab1b3fe3745fd...d62fd3b75093"
    },
    "name": "Undercover"
}

To solve these issues, Keystone provides a set of configuration options where you specify for each of the needed Keystone entity attributes where in the LDAP schema you will find the equivalent. For example, for the user entity, the configuration file might include the following:

user_id_attribute = mail
user_name_attribute = mail
user_enabled_attribute = enabled
user_pass_attribute = userPassword

The left-hand side specifies the Keystone entity attribute; the right-hand side the name of the LDAP attribute to use. Similarly for groups:

group_id_attribute = cn
group_name_attribute = cn
group_desc_attribute = description

There are other things Keystone also needs to know, of course. For example, the URL of the LDAP server, the location in the LDAP tree where the user and group objects are stored, etc. So let’s look at a complete set of LDAP options in a typical configuration file:

[identity]
driver = ldap

[ldap]
url = ldap://myservice.acme.com
query_scope = sub

user_tree_dn = "ou=users,o=acme.com"
user_objectclass = person
user_id_attribute = mail
user_name_attribute = mail
user_mail_attribute = mail
user_pass_attribute = userPassword
user_enabled_attribute = enabled
group_tree_dn = "ou=memberlist,ou=groups,o=acme.com"
group_objectclass = groupOfUniqueNames
group_id_attribute = cn
group_name_attribute = cn
group_member_attribute = uniquemember
group_desc_attribute = description

user_allow_create = false
user_allow_update = false
user_allow_delete = false
group_allow_create = false
group_allow_update = false
group_allow_delete = false

Let’s look, one by one, at the key options in the above configuration settings to see what’s being done here:

[identity]
driver = ldap

This tells Keystone to use the LDAP driver for the [identity] backend (the default is SQL).

The next section, [ldap] , adds the specific LDAP settings:

url = ldap://myservice.acme.com

This tells Keystone the URL of your server.

query_scope = sub

This tells Keystone what type of depth of search it should do when looking for objects in LDAP. The default is one meaning just search at the top level of the specified tree, whereas sub means search for any objects within the sub-tree. 

user_tree_dn = "ou=users,o=acme.com"

This specifies the root of the tree in the LDAP server in which Keystone will search for users.

user_objectclass = person

This specifies the LDAP object class that Keystone will filter on within user_tree_dn to find user objects. Any objects of other classes will be ignored.

user_id_attribute = mail
user_name_attribute = mail
user_pass_attribute = userPassword

This set of options define the mapping to LDAP attributes for the three key user attributes supported by Keystone. The LDAP attribute chosen for user_id must be something that is immutable for a user and no more than 64 characters in length. Although one might think that the Distinguished Name (DN) would be a good choice here, there are problems with this (e.g., it can be longer than 64 characters). Often, an LDAP schema will include a uid of some type, which is a good substitute, or alternatively mail is a popular choice. The user name must also be unique within the user objects that will be accessed by Keystone.

user_mail_attribute = mail

Similar to the attributes above, this gives the mapping to the LDAP email attribute. In fact, email is not a standard attribute supported by Keystone (i.e., you will not find it defined in the OpenStack Identity API specification), since it can be considered Personally Identifiable Information (PII). As such, Keystone does not automatically provide this for storage in its default SQL database. However, if your LDAP already provides this, then setting the mapping above will cause this to appear in the Keystone user entity as the email attribute.

user_enabled_attribute = enabled

In Keystone, a user entity can be either enabled or disabled. Setting the above option will give a mapping to an equivalent attribute in LDAP, allowing your LDAP management tools to disable a user and for that fact to be reflected in OpenStack (since a disabled user is prevented from authenticating by Keystone).

group_tree_dn = "ou=memberlist,ou=groups,o=acme.com"

Similar to user_tree_dn, this specifies the root of the tree in LDAP in which to search for group objects.

group_objectclass = groupOfUniqueNames

Similar to user_objectclass, this defines the LDAP object class of a group object.

group_member_attribute = uniquemember

The LDAP attribute within the group object that holds the users who are members.

group_id_attribute = cn
group_name_attribute = cn
group_desc_attribute = description

This set of options specify the LDAP attributes mapped to the Keystone group entity attributes. Again, as in the case for users, the group id must be unique for the group objects that will be accessed by Keystone, as must the group name.

user_allow_create = false
user_allow_update = false
user_allow_delete = false
group_allow_create = false
group_allow_update = false
group_allow_delete = false

The above set of options dictate whether any of the Keystone writeable operations should be supported against LDAP (by default they are set to true).

With the above configuration (once you have restarted Keystone), you should be able to use Keystone’s user and group APIs to read the information in your corporate LDAP, for example:

$ openstack user show [e-mail] --domain default
+-----------+------------------------------------------------------------------+
| Field     | Value                                                            |
+-----------+------------------------------------------------------------------+
| domain_id | default                                                          |
| email     | [e-mail]                                               |
| id        | [e-mail]                                               |
| name      | [e-mail]                                               |
+-----------+------------------------------------------------------------------+

Issuing the above command will cause Keystone to:

1. Search for the objects of class person within the tree rooted by ou=users,o=acme.com that have the mail attribute equal to [e-mail] (since we defined mail as the user name)
2. Having found such an object, return a Keystone user entity with the attributes of id, email, and name all set to the value of the mail attribute (since that is what we specified in the configuration options)

Note that for the above to work, the user accessing Keystone needs to have anonymous query rights to the dn_tree specified in the configuration options. If this is not possible, then you can provide a special user and password account that will be used for that purpose in the configuration file in the [ldap] section:

user = cn=Manager,ou=users,o=acme.com
password = ga9s86d3das98dd

The above account is the only user to bind and search for users/groups.

4.2.1 Other Keystone Configuration Options in Classic LDAP Support

The Keystone configuration supports a number of other additional options and shortcuts that allow integration with other common LDAP schemas and configurations:

use_tls = True
tls_cacertfile = /etc/keystone/ssl/certs/cacert.pem
tls_cacertdir = /etc/keystone/ssl/certs/
tls_req_cert = demand

The above four options all relate to enabling Transport Layer Security (TLS) for providing a secure connection from Keystone to LDAP—and we highly recommend this. Setting use_tls to true will cause Keystone to upgrade the connection to SSL. Note that an alternative to specifying use_tls is to specify an alternate port in the URL by using the form “ldaps://myserver.com:636”. Whichever method you choose (and you should use one or the other, but not both), the subsequent SSL connection is governed by the other three tls options. tls_cacertfile gives the name of the file on the Keystone server where the certificate(s) is/are to be found—if you specify this option, then all certificates must be placed inside this one PEM file. Alternatively, the certifications can be stored as individual files in a directory specified in tls_cacertdir. Finally, you can control how the certificates are checked for validity in the client (i.e., Keystone end) of the secure connection (this doesn’t affect what level of checking the server is doing on the certificates it receives from Keystone). The client checking is really designed to spot man-in-the-middle (MITM) attacks—and the options for tls_req_cert (which corresponds to the standard TLS option TLS_REQCERT) are demand, never, and allow. The default of demand means the client always checks the certificate and will drop the connection if it is not provided or invalid. never is the opposite—it never checks it, nor requires it to be provided. allow means that if it is not provided then the connection is allowed to continue, but if it is provided it will be checked—and if invalid, the connection will be dropped.

suffix = "o=acme.com"

By default, if you don’t specify user_tree_dn or group_tree_dn, Keystone will construct these by adding ou=Users and ou=UserGroups, respectively, on the front of the suffix (e.g., ou=Users,o=acme.com).

use_dumb_member = false
dumb_member = "cn=dumb,dc=nonexistent"

dumb_member is only used when LDAP groups are writeable and the LDAP class chosen requires the member attribute to be defined. If use_dumb_member is true, the LDAP member attribute will be set to the value defined in dumb_member upon creation of a group.

user_filter = "objectCategory=person"
group_filter = "objectCategory=group"

The two filter options allow additional filters (over and above user_objectclass and group_objectclass) to be included (i.e., ANDed) into the search of user and group objects, respectively. One common use of this is to provide more efficient searching within AD installations, where the recommended search for user objects is (&(objectCategory=person)(objectClass=user)). By specifying user_objectclass as user and user_filter as objectCategory=person in the Keystone configuration file, this can be achieved. Similarly, for group searching, the recommended search is (objectCategory=group), which can be achieved by specifying group_objectclass as * and group_filter as objectCategory=group.

user_enabled_mask = 2

Some LDAP schemas, rather than having a dedicated attribute for user enablement, use a bit within a general control attribute (such as userAccountControl) to indicate this. Setting user_enabled_mask will cause Keystone to look at only the status of this bit in the attribute specified by user_enabled_attribute, with the bit set indicating the user is enabled.

user_enabled_default = 512

Most LDAP servers use a boolean or bit in a control field to indicate enablement. However, some schemas might use an integer value in an attribute. In this situation, set user_enabled_default to the integer value that represents a user being enabled.

user_enabled_invert = true

Some LDAP schemas have an “account locked” attribute, which is the equivalent to account being “disabled.” In order to map this to the Keystone enabled attribute, you can utilize the user_enabled_invert setting in conjunction with user_enabled_attribute to map the lock status to disabled in Keystone. Note that this option cannot be used in conjunction with either user_enabled_emulation or user_enabled_mask (see below):

user_enabled_emulation = true
user_enabled_emulation_dn = "ou=enabledusers,o=acme.com"

Finally, some LDAP schemas do not hold a suitable attribute that can be mapped directly to the Keystone user-enabled attribute, but nonetheless want to allow their LDAP management tools to cause Keystone users to be enabled/disabled. If user_enabled_emulation is true then Keystone treats the DN specified by user_enabled_emulation_dn as a group object containing the list of enabled users. Adding/removing users to/from this group using your LDAP management tools will cause those users to be enabled/disabled in OpenStack.

allow_subtree_delete = false

Some of the data Keystone stores in LDAP is in tree form, e.g., user and group roles on projects in assignments (and is relevant to Identity if you are also using the LDAP assignment driver). When it comes to deleting groups or projects, by default Keystone will essentially walk the tree deleting the objects. Some LDAP servers support the ability to delete a tree in a single command, and setting allow_subtree_delete to true will cause Keystone to try and use this facility. If you set this to true and your LDAP server does not support tree deletion, then it will respond with an error—in which case Keystone will fall back to tree walking.

page_size = 0

By default, when searching data, Keystone will talk to the LDAP server in a synchronous fashion. Setting page_size to something other than zero will result in Keystone asking the LDAP server to send results in chunks of page_size number of entries. For this to work, your LDAP server must support paging (or Keystone will log an error). Note that although Keystone is now talking to the LDAP server in an asynchronous fashion, Keystone will only return from its API call once all the data has been returned by the LDAP server. In other words, this level of paging does not enable paging at the API level.

alias_dereferencing = default

Aliasing in an LDAP server is a way of linking objects (i.e., part of one object is an alias to another object). When querying a server, an LDAP client can specify whether these alias links are followed or “dereferenced,” so that the aliased object is returned in the query. The default for Keystone is just to use the underlying configuration of your LDAP client. However, that can be overridden by specifying any of the standard options for LDAP alias dereferencing, which are:

• never (which means never dereference)
• always (which means always dereference)
• finding (which means dereference in finding the base object, but not when returning objects from the subsequent search of subordinate objects)
• searching (which means dereference the objects returned from the search of subordinate objects, but not in finding the initially specified base object)

debug_level = 0

A non-zero value of debug_level will enable debugging at the LDAP library level, with the value being passed directly into the LDAP calls themselves. You will need to consult your LDAP library and server documentation for the appropriate settings depending on what level of debugging you want to set.

chase_referrals = true

Most LDAP and AD servers support the concept of referrals. This is where you ask the LDAP server for the value of some object, and it replies “Well, I don’t have that, but I think I know who does, try that server over there.” The LDAP client library can automatically “chase the referral” for us—and your LDAP client library will most likely be configured as to whether it should do that or not. Setting chase_referrals in the Keystone configuration file will override the setting in your LDAP client. Note that if referrals are not being chased, and Keystone receives a referral response when querying your LDAP server, it will discard this response (and log a debug error message indicating that you should enable referral chasing). Also note that referral chasing is an LDAP client-side activity, as opposed to chaining, which is where the LDAP server does this for you. If you are using chaining, you do not need to set chase_referrals.

user_attribute_ignore = ["default_project_id"]

This specifies a list of Keystone user entity attributes that should not be mapped to LDAP, and should be ignored for both read and write operations. The default list for the user object contains default_project_id, since this rarely maps to the typical LDAP schema. With this being ignored, in order to obtain a project-scoped token, you must always explicitly provide the project_id or name when authenticating to Keystone. 

user_default_project_id_attribute = None

This defines the LDAP attribute that should be mapped to the default_project_id attribute in the Keystone user entity. In reality, as described above, this is rarely used since it does not match most LDAP schemas—and it is entirely optional for Keystone, since with the v3 API it simply represents a shortcut from having to define a project scope when authenticating.

user_additional_attribute_mapping = []

Keystone supports the dynamic adding of attributes to its entities, allowing cloud deployers and developers to, for instance, extend the user object. When the backend is SQL, these are stored in a hidden (at the API level) column called extra. For LDAP, since an additional attribute needs to be mapped to the schema, you can set user_additional_attribute_mapping to indicate the name of the Keystone attribute and the LDAP attribute to which it is to be mapped. This option is a list of pairs of mappings for the form "keystone_attrib":"LDAP attribute", for example:

user_additional_attribute_mapping = ["cell_number":"mobileTelephoneNumber"]

With the above specified in the configuration, the user entity returned by Keystone from a GET /users API call would look like something like this:

{
    "cell_number": "925-482-7336",
    "domain_id": "default",
    "enabled": true,
    "id": "ef7ab1f3fe2745fd91b3d62fd3b7309b",
    "links": {
        "self": "http://localhost:5000/v3/users/ef7ab1f3fe2745fd91b3d62fd3b7309b"
    },
    "name": "GeorgeSmiley"
}

group_attribute_ignore = []

This specifies a list of Keystone group entity attributes that should not be mapped to LDAP, and should be ignored for both read and write operations.

group_additional_attribute_mapping = []

Similar to user_additional_attribute_mapping, this allows you to define additional attributes to be mapped from your LDAP group entity into the one Keystone will return via its API.

use_pool = false
pool_size = 10
pool_retry_max = 3
pool_retry_delay = 0.1
pool_connection_timeout = -1
pool_connection_lifetime = 600

This batch of options relates to performance optimizations by instructing Keystone to maintain a pool of connections to the LDAP server, and hands these out to be used by incoming request processing rather than having to build and tear down a connection for each API request. The main pool is used for all requests except authentication requests (see below). Use of the pool is disabled by default, and setting use_pool to true will enable it. Set pool_size to the maximum likely number of concurrent requests.

use_auth_pool = false
auth_pool_size = 100
auth_pool_connection_lifetime = 60

The above allows you to further refine performance by having a separate pool of connections for authentication requests, as opposed to those used for other API processing.

4.3 Multiple Domains and LDAP

4.3.1 Requirements for Multi-Domain Corporate Directory Support

There are two main use cases for multi-domain LDAP with Keystone:

• A cloud provider who wants to support multiple customers on their cloud, each of which wants to integrate with their own corporate directory.
• An enterprise wants to model their internal company structure around domains—for example, each division of the company might be a domain. Each of these divisions might have their own LDAP server, or perhaps a separate tree of users within a common company LDAP server.

In both cases, the first key issue for Keystone is how to hook different LDAP URLs (and other configuration options) to different domains. The domain definitions are stored in the Keystone Resource backend, but somehow the Identity backend needs to know on a domain-by-domain basis to which LDAP server to send the request. Easy, you might think: just have an index by domain, right? Unfortunately, most user and group API calls do not include a domain identifier, so that’s not so easy after all. One of Keystone’s functions for Identity is to be the source (for other OpenStack services) of a globally unique ID (for each user and group). Once a user is created, this unique ID is used in most Keystone calls to identify the user or group in question; for example, to get a user entity you would issue a Keystone API call specifying just the user ID:

GET /v3/users/d19cac8f7ef3e6de36a47b85f306f137

How will Keystone know which domain/LDAP server to talk to in order to find this user? For performance reasons, searching them all is not a viable option. This problem was the main reason it took until Juno to support multi-domain LDAP. Further, as we saw from the classic (mono-domain) LDAP support in the previous section, the Keystone configuration file specifies the LDAP attribute to map to the user_id or group_id. Given the (OpenStack) constraint that this ID must be globally unique (i.e., across all the LDAPs being interfaced by one Keystone), a second issue is whether we would trust any given LDAP to be providing such a globally unique attribute. Imagine if we just let whatever ID the underlying LDAP administrator chose for a user to become the ID that had to be unique across the LDAP directories of all the other customers (who may even be their competitors) who were hosted on the same cloud? Sounds like a recipe for disaster! The solution to both issues was to get Keystone to provide an intermediate ID mapping, which creates a “public” unique ID that is published to the rest of OpenStack, and builds a table that will map this public ID back to its Domain, LDAP server, and local ID within that LDAP server. This directory-mapping layer is essentially invisible to callers of the Keystone API, and the mapping is built on the fly as Keystone encounters entities in its identity drivers. The only thing a caller would notice is that the IDs for users and groups have gotten bigger. (With multi-domain enabled, they are, by default, 64-byte SHA256 hashes as opposed to the classic 32-byte UUIDs.) It should be noted that this directory mapping is not related to the Keystone federation mapping used to handle the mapping of identity provider assertion attributes into Keystone attributes.

So let’s take a look at an example of using domain-specific directories in practice. In a service provider that offered cloud services to its customers, it would be common to represent a customer as a domain in Keystone. This allows customers to have their own users, groups, and projects. What the provider would like to do is to be able to offer customers the ability to use their existing corporate directory for authentication to their domain—enabling each customer domain to point to that customer’s own LDAP/AD service. The first thing to do is to enable domain-specific configuration support in Keystone (it is disabled by default) by setting the following option in the main Keystone configuration file:

[identity]
domain_specific_drivers_enabled=true

The next step is to decide how you will provide the domain-specific LDAP configuration options (e.g., URL to the LDAP server). Keystone supports two methods for doing this:

• By providing a configuration override file for each domain (this method has been supported since Juno).
• By using some new additions to the Keystone API to specify the LDAP overrides for each domain. (This was introduced in Kilo, and is currently marked as experimental. Despite being marked as experimental, it is fully functional and is likely to become the primary supported mechanism in the Mitaka release.)

Both methods achieve the same goal: telling Keystone how to handle LDAP for a given domain. You can’t mix the two approaches—it’s either/or—so you need to decide which one to use. The API approach has the benefit that you can onboard a domain (including setting its LDAP parameters) purely via the Keystone API. The configuration file approach, while being familiar to anyone setting up an OpenStack service, means you must ensure the file is created on your Keystone server for each LDAP-backed domain you set up.

4.3.2 Setting Up Multi-Domain Using the Configuration File–Based Approach

To use this approach, you specify to Keystone a directory on the Keystone server in which to locate the domain-specific configuration files (one per domain).

[identity]
domain_config_dir=/etc/keystone/domains

When domain-specific configuration files are enabled, each time Keystone is started it will scan the contents of domain_config_dir for any domain-specific configuration files. For each domain that requires its unique corporate directory, a domain-specific configuration file is required that contains the specific details of the corporate directory for that domain. For example, if the service provider creates a domain called “customerA” to represent that customer, then they create a domain-specific configuration file called keystone.customerA.conf. (The name is important and is always of the form: keystone.{domain-name}.conf—the domain name must match the name of the domain created in Keystone.) The contents of this configuration file might look something like this:

[ldap]
url = ldap://ldapservice.thecustomer.com
query_scope = sub

user_tree_dn = ou=Users,dc=openstack,dc=org
user_objectclass = MyOrgPerson
user_id_attribute = uid
user_name_attribute = mail
user_mail_attribute = mail
user_pass_attribute = userPassword
user_enabled_attribute = enabled
group_tree_dn = ou=UserGroups,dc=openstack,dc=org
group_objectclass = groupOfUniqueNames
group_id_attribute = cn
group_name_attribute = cn
group_member_attribute = uniquemember

As you can see, these are exactly the same types of configuration options that are normally set in the main Keystone configuration file—except in this case they are specified in the specific configuration file for this domain and act as overrides to any options specified in the main Keystone configuration file. Note that only the options that are specific to the domain need to be specified here; common options can stay in the main Keystone configuration file.

Once a domain has its own configuration file, Keystone will build a mapping for the user and group entities in that backend as it encounters them, and route any requests issued for those entities to the appropriate corporate directory.

While it is primarily designed to support different corporate directories, you can also specify a configuration file for a domain to use an SQL backend. This is particularly useful if you want to have some local users (such as service users or local admin users) that you don’t want to place in a corporate directory. This is done in exactly the same way as in the LDAP/AD case: Create a domain-specific configuration file for a domain using the same naming convention, but in this case it will probably just contain the name of the driver. For example, if you want domain CloudAdmin to be backed by SQL while all your other domains perhaps use LDAP, then create a domain-specific configuration file called keystone.CloudAdmin.conf that contains:

[identity]
driver = sql

The big restriction today, however, is that Keystone currently allows only one SQL driver to be loaded at a time—so either this can be the catch-all driver (by specifying the Identity driver as SQL in the main Keystone configuration file), or it can be assigned to a specific domain (using a domain-specific configuration file). If you try specifying more than one domain with its own SQL configuration, Keystone will raise an exception when starting up. Future versions of Keystone may lift this limitation.

As was stated at the start of this subsection, the configuration files are only read when starting Keystone—so every time you add or change any of the LDAP configuration parameters, you need to restart Keystone. The alternative approach is to use an extension that is part of the Keystone API, which modifies the operation of a running Keystone—this is described in the next subsection.

4.3.3 Setting Up Multi-Domain Using the Keystone API–Based Approach

The Keystone API approach is similar to the configuration file approach, except that rather than specify the LDAP override options in a file, you can specify them by issuing an API call against the domain in question. This support is (in Liberty) marked as experimental, although it is fully functional, and is targeted to be a part of the core API in the Mitaka release. It is enabled by the (rather poorly named) option in the main Keystone configuration file:

[identity]
domain_configurations_from_database=true

Having enabled the capability, you can now use the domain configuration management part of the Identity API, e.g., to get or modify the overrides for a given domain:

[GET | PUT | PATCH | DELETE] /domains/{domain_id}/config

As of the Liberty release, these APIs are not yet supported from OpenStackClient (the changes to add them are currently under review), so using a cURL request to create the configuration overrides for a given domain might look like:

$ curl -s -X PUT -H "X-Auth-Token: $OS_TOKEN" \
  -H "Content-Type: application/json" - d '
{
    "config": {
        "identity": {
            "driver": "ldap"
        },
        "ldap": {
            "url" = "ldap://ldapservice.thecustomer.com"
            "query_scope" = "sub"
            "user_tree_dn" = "ou=Users,dc=openstack,dc=org"
            "user_objectclass" = "MyOrgPerson"
            "user_id_attribute" = "uid"
            "user_name_attribute" = "mail"
            "user_mail_attribute" = "mail"
            "user_pass_attribute" = "userPassword"
            "user_enabled_attribute" = "enabled"
            "group_tree_dn" = "ou=UserGroups,dc=openstack,dc=org"
            "group_objectclass" = "groupOfUniqueNames"
            "group_id_attribute" = "cn"
            "group_name_attribute" = "cn"
            "group_member_attribute" = "uniquemember"
        }
    }
}' http://localhost:5000/v3/domains/421370cd78234413bbeb5d7ce1c73077/config 

The above is the equivalent to the configuration file described in “4.3.2 Setting Up Multi-Domain Using the Configuration File–Based Approach”.

In fact, the domain configuration management API extensions are quite powerful, and can be used to read and modify individual options, not just to bulk load/read the entire block. For instance, let’s say that the LDAP server URL for the domain in the above example was changed, you could update just that option using the following API command (via cURL):

$ curl -s -X PATCH -H "X-Auth-Token: $OS_TOKEN" \
  -H "Content-Type: application/json" - d '
{
    "url" = "ldap://ldapservice2.thecustomer.com"
}' \
http://localhost:5000/v3/domains/421370cd78234413bbeb5d7ce1c73077/config/ldap/url

Similarly, you can just get the value of an individual option:

$ curl -s -H "X-Auth-Token: $OS_TOKEN" -H "Content-Type: application/json" \ 
  http://localhost:5000/v3/domains/421370cd78234413bbeb5d7ce1c73077/config/ldap/u
  rl | python -mjson.tool
{
    "url" = "ldap://ldapservice2.thecustomer.com"
}

It is likely that over time this API-based method of domain configuration management will become the primary method, and that the configuration file–based method will be deprecated. However, as stated above, currently this API method is still marked as experimental. This is mainly due to the fact that there are some corner cases that are not yet well handled within its implementation—primarily to do with what happens in a multiprocess installation of Keystone (i.e., one where multiple Keystone processes are used to improve the response time and throughput). The challenge for Keystone in this case is to ensure that all processes see the update (and any cache copies are updated), and further that they see the update in an atomic fashion (i.e., if you are updating multiple configuration options for a domain, that all processes see all the updates or none of the updates). The implementation for Liberty solves the first of these challenges, but it is possible that if you try and access the LDAP users or groups at the same time you are updating the domain configuration options that for a short period that other process could see a partial update (until the caches timeout). So if you choose to use this capability, then either ensure creating/updating the LDAP options are done when the domain is not in use, or worst case, ensure that you wait for at least a period of cache_time before using the domain (by default this is 300 seconds, i.e., 5 minutes). The cache_time is itself configurable in the main Keystone file using the option:

[domain_config]
cache_time=300

As stated, these restrictions are planned to be resolved in the Mitaka release, and the API management method will be declared as stable.

In summary, the API method of setting up the domain LDAP configuration options has several advantages over the file-based method:

• It is all done via the Keystone API, so there’s no need to separately distribute the domain-specific configuration file. Given that the API can also be used in a fine-grained fashion, this is also easier to integrate into your management systems.
• Does not require a restart of the Keystone server, ensuring no interruption of service when onboarding a domain.

4.3.4 Restrictions When Using Multi-Domain Identity

The main restriction relates to one of the common queries people want to make when first starting to test Identity, which is “list me all the users in the system,” using the GET /users API call. Now, for small systems when all the users are in an SQL database, such a query doesn’t seem unreasonable.  However, imagine a setup where there are many tens (or even hundreds) of customers supported in a single cloud, each in their own domain, each using their own corporate LDAP. In such situations, satisfying “list me all the users in the system” would require a scan of all the separate corporate LDAP servers and merging the results. But, more importantly, does asking such a question really make any sense? The primary use of multi-domain is to enable each customer to have their own LDAP support, so the use of such a “list everything” command would have to be restricted to administrators of the cloud provider, who, in reality, simply do not need to have a list of all the users of all their customers. As such, if you enable multi-domain support, Keystone will require you to specify a domain as part of a list all users request in one of two ways:

1. Add a domain filter to the request:
   GET /users?domain_id=421370cd78234413bbeb5d7ce1c73077
2. Use a domain-scoped token when issuing the request, and the scope of the token will be used as the domain to search

Hence, with multi-domain enabled, you can no longer ask to “list me all the users in the system,” but you can ask to “list me all the users in this domain.”

A second restriction, which is probably already apparent, is that multi-domain is only supported when using the Keystone v3 API. The v2 API does not support the concept of domains, so there is no way to separate the different namespaces.

4.3.5 Choosing the Right Mix of SQL and LDAP Domains

As discussed already, the main use of multi-domain LDAP is to allow the users and groups of newly created domains to be drawn from specific LDAP corporate directories. In setting up multi-domain to achieve this, there are a couple of decisions you need to make:

1. What backing store will be used for the default domain? Before trying to answer this, a little background is needed. Originally, before Keystone v3, there were no domains—users were “global” across a Keystone installation. With the introduction of domains, users and groups are owned by a given domain; however, that begged the question as to what happens to the users created before the v3 API was introduced? Also, a modern Keystone supports both the v3 and v2 APIs (and although v2 will be deprecated soon, you can still call it), so what happens to users created via the v2 API today (since you can’t specify what domain they are in)? To solve these issues, Keystone automatically creates a “default” domain (which has an ID of default and a name of Default). Any users created using the v2 API will be placed in this domain.
2. Where are the service users going to be created? Service users are the user accounts that other services (e.g., Nova, Glance, etc.) utilize to make calls to Keystone. Since these already existed pre-v3, these typically live in the default domain, particularly since until recently some services still made v2 API calls to Keystone (and a v2 API call is only allowed to see users in the default domain). With the latest keystonemiddleware (the Keystone client code that is used by services to call Keystone), services can now specify the domain ID for the service user, enabling cloud providers to move their service users into a separate service domain.

So given the above, you have three basic choices relating to the default domain, as discussed in the following.

Use SQL for the Default Domain

The most obvious solution for new installations is that the default domain is backed by a local Keystone SQL database, and all other domains use LDAP. One important advantage to this approach is that service users can be created locally, without the need for them to be added to a corporate directory.

In fact, there are two ways of configuring Keystone so that the default domain is SQL. Depending on which you choose there will be implications for any other SQL domains you want to support:

1. Set the Identity driver to sql in the main Keystone configuration file, and then add domain-specific LDAP configurations (using the file- or API-based mechanisms) for those domains that have their own LDAP. What this configuration allows is for the SQL driver to handle not just the default domain, but any other domain that doesn’t have explicit LDAP configurations defined. This can be quite a common multi-customer cloud scenario—some smaller customers just want their users and groups managed by the cloud provider, some larger customers want to link to their existing corporate directory. Using this SQL configuration, this gives a cloud provider the option to choose to use SQL for a set of smaller customer domains if that was more cost effective (these customers’ users and groups are still in their own domains, so their namespaces are maintained). Of course, there’s no reason the cloud provider could not also use an internal LDAP server with different tree attach points for these smaller customers, if that suited their requirements better.
2. Create a specific domain configuration for the default domain and in it set the Identity driver to sql. Using the API-based method you would issue the following call: 

   $ curl -s --request PUT -H "X-Auth-Token: $OS_TOKEN" \
     -H "Content-Type: application/json" - d '
   {
       "config": {
           "identity": {
               "driver": "sql"
           },
       }
   }' http://localhost:5000/v3/domains/default/config 
   

   Alternatively, if you are using the file-based configuration method you would create a domain-specific configuration file of the name keystone.Default.conf (note how the name of the default domain is used in the filename) that includes just the [identity] section and set the driver to sql. Also set the identity driver in the main Keystone configuration file to ldap (but don’t worry about any of the other LDAP configuration options in the main configuration file). This will tell Keystone that only the default domain is backed by SQL and all other domains must be given their own LDAP configuration—and access to a domain won’t work without creating a specific LDAP configuration for it.

This method is probably the simplest to set up. But one thing to keep in mind, however, is that if you still need to support users accessing Keystone via the v2 API, then they will access the default domain, which is SQL. There is no way of allowing v2 clients to access any of the LDAP domains.

Use LDAP for All Domains, Except an SQL Service Domain

This is a new capability using keystonemiddleware plus enhancements to multi-domain support that was added to Keystone in Liberty. To configure this you need to:

• Set the identity driver to ldap in the main Keystone configuration file.
• Create a domain in Keystone that will hold your service users, and create a domain-specific configuration for this service domain, setting the identity driver to sql.
• Create your service users in this new domain (which will be stored in the Keystone SQL database).
• Modify the configuration file for each of your other services to specify the domain_id for the service users, e.g., in nova.conf you would set the following:

  [keystone_authtoken]
  username = nova
  user_domain_id = 7c0df68c6b734735a22b2365f241b9b8
  

  For any additional domains, create LDAP domain-specific configurations for each one, using either the file- or API-based mechanisms.

This SQL/LDAP mix requires that all services be configured to use the domain for service users as described above. Provided you can do this, then you kind of get the best of both worlds—LDAP for both the default and additional domains, with your service users stored locally in a separate service domain. In addition, you preserve the ability (if you need it) for any user access via the v2 API to successfully access the LDAP-backed default domain.

Use LDAP for All Domains

The third approach is LDAP for all domains, including the default domain. This is often used in installations where LDAP was already in place prior to multiple domains being supported (with the Identity driver set to ldap in the main keystone configuration file). In these scenarios, the service users will have already been created in the LDAP directory that is backing the default domain. Under this circumstance, multi-domain can be enabled and additional domains provided with their own configurations. Note that in this setup, all domains (except the default domain) will require their LDAP configuration to be defined using either the file- or API-based mechanisms.

Finally, as an aside, remember that in all of the above methods, we are talking about whether to use SQL or LDAP for the store of Identities. Whichever mix of SQL and LDAP you chose for your domains, you can always use SQL for the storage of resources and assignments (and indeed this is recommended; we will discuss this more in “4.5 Projects, Roles, and Assignments from LDAP (Just Say NO!)”).

4.4 A Practical Guide to Using Multi-Domains and Keystone

In this section, we show how to use Keystone against IBM’s internal corporate directory (“bluepages”), which is a standard LDAP server. Although the particular LDAP attributes might be different in your setup, the principles will be the same.

4.4.1 Setting Up LDAP

Like the early examples from Chapter 2, we start from a fresh DevStack installation of OpenStack, which will have set up the default domain as a service domain containing the service accounts OpenStack needs. Hence, the first task is to add a new domain that will subsequently be configured so its users and groups are mapped to the bluepages directory, where the user accounts for all IBMers are stored.

$ openstack domain create ibm
+---------+----------------------------------+
| Field   | Value                            |
+---------+----------------------------------+
| enabled | True                             |
| id      | 421370cd78234413bbeb5d7ce1c73077 |
| name    | ibm                              |
+---------+----------------------------------+

Let’s also create a project within that domain (both the domain and project records are actually being stored in SQL in Keystone since the Resource backend is backed by SQL).

$ openstack project create ibmcloud --domain ibm
+-------------+----------------------------------+
| Field       | Value                            |
+-------------+----------------------------------+
| description |                                  |
| domain_id   | 421370cd78234413bbeb5d7ce1c73077 |
| enabled     | True                             |
| id          | 95ae2a9c259b4012b8b7e8ad7dc9a939 |
| name        | ibmcloud                         |
| parent_id   | None                             |
+-------------+----------------------------------+

Now we need to tell Keystone to use multi-domains and that for this specific IBM domain, to go and talk to the IBM bluepages server. First, we update the main Keystone configuration file (keystone.conf) to enable the domain-specific driver option:

[identity]
domain_specific_drivers_enabled = true

Then we need to set the LDAP options for the specific IBM domain. As described earlier, there are two ways to do this: using the Keystone API or by creating a separate configuration file for this domain. We’ll show both ways here.

To use the API method, we first need to enable this option (in addition to setting domain_specific_drivers_enabled to true):

[identity]
domain_configurations_from_database=true

Then we simply call the domain configuration management PUT API to create the options all in one go:

$ curl -s --request PUT -H "X-Auth-Token: $OS_TOKEN" \
  -H "Content-Type: application/json" - d '
{
    "config": {
        "identity": {
            "driver": "ldap"
        },
        "ldap": {
            "url" = "ldap://bluepages.ibm.com",
            "query_scope" = "sub",
            "user_tree_dn" = "ou=bluepages,o=ibm.com",
            "user_objectclass" = "ibmPerson",
            "user_id_attribute" = "uid",
            "user_name_attribute" = "mail",
            "user_mail_attribute" = "mail",
            "user_pass_attribute" = "userPassword",
            "user_enabled_attribute" = "enabled",
            "group_tree_dn" = "ou=memberlist,ou=ibmgroups,o=ibm.com",
            "group_objectclass" = "groupOfUniqueNames",
            "group_id_attribute" = "cn",
            "group_name_attribute" = "cn",
            "group_member_attribute" = "uniquemember",
            "group_desc_attribute" = "description",
            "user_allow_create" = false,
            "user_allow_update" = false,
            "user_allow_delete" = false,
            "group_allow_create" = false,
            "group_allow_update" = false,
            "group_allow_delete" = false
        }
    }
}' http://localhost:5000/v3/domains/421370cd78234413bbeb5d7ce1c73077/config 

That’s it for the API method—we’re done. Skip to “4.4.2 Running Admin Commands” to try some commands to test our new setup.

The alternative to the above is to create a domain-specific configuration file. To enable this, we need to tell Keystone where to find this file:

[identity]
domain_config_dir = /etc/keystone/domains

Now we create the IBM domain-specific configuration file (keystone.ibm.conf) in the /etc/keystone/domains directory, and use it to specify the bluepages-specific settings:

[identity]
driver = ldap

[ldap]
url = ldap://bluepages.ibm.com
query_scope = sub

user_tree_dn = "ou=bluepages,o=ibm.com"
user_objectclass = ibmPerson
user_id_attribute = uid
user_name_attribute = mail
user_mail_attribute = mail
user_pass_attribute = userPassword
user_enabled_attribute = enabled
group_tree_dn = "ou=memberlist,ou=ibmgroups,o=ibm.com"
group_objectclass = groupOfUniqueNames
group_id_attribute = cn
group_name_attribute = cn
group_member_attribute = uniquemember
group_desc_attribute = description

user_allow_create = false
user_allow_update = false
user_allow_delete = false
group_allow_create = false
group_allow_update = false
group_allow_delete = false

Before continuing, we need to restart Keystone for the changes to take effect:

$ sudo service apache2 restart

4.4.2 Running Admin Commands

Finding a user

In this example, we attempt to find an LDAP user within the domain that is backed by LDAP.

$ openstack user show [e-mail] --domain ibm
+-----------+------------------------------------------------------------------+
| Field     | Value                                                            |
+-----------+------------------------------------------------------------------+
| domain_id | c3c24e60d57e4461ad64b372c14128b7                                 |
| email     | [e-mail]                                              |
| id        | c29cac8f7003e6de36a47b85f306f137bea8026e3d55c35aed27dfbf09c8fb28 |
| name      | [e-mail]                                              |
+-----------+------------------------------------------------------------------+

Finding groups a user is a member of

In this example, we list all LDAP groups a user is a member of. Similar to the above command, we need to specify the domain the user can be found in. Note that if a user is not specified as a filter, then Keystone will attempt to list all LDAP users, an operation that will likely result in a timeout.

$ openstack group list --user [e-mail] --user-domain ibm
+----------------------------------------------------------+--------------------+
| ID                                                       | Name               |
+----------------------------------------------------------+--------------------+
| 178e5df37393dd6695c4d93382...60b21b393373ff8360a191b74bb | SWG_Canada         |
| 5f620ab5abe2f0b14e3112357d...3ac637e7416b7d562152f73e938 | Toronto_Lab_VPN    |
| ed9f9cb5dbefd6a6e5808ca979...b5d160e0a7308c199680c595a96 | HiPODS-OpenCloud   |
+----------------------------------------------------------+--------------------+

List all members of a group

In this example, we attempt to list all users in a group. We can group by ID or by using the group’s name and owning domain. Similar to the previous command, if a group is not specified as a filter, then Keystone will attempt to list all LDAP groups, an operation that will likely result in a timeout. Note that the ability to list users in a group by group and domain name will be available in the Mikata release of OpenStackClient.

$ openstack user list \
  --group ed9f9cb5dbefd6a6e5808ca97921734da3b90b5d160e0a7308c199680c595a96
+---------------------------------------------------------+---------------------+
| ID                                                      | Name                |
+---------------------------------------------------------+---------------------+
| c29cac8f7003e6de36a47b...a8026e3d55c35aed27dfbf09c8fb28 | [e-mail] |
| f3ca1de06fbe100e71577a...d792686ae75812becfe43d5b4aa09c | [e-mail]    |
+---------------------------------------------------------+---------------------+

Assigning a group a role on a project

In this example, we want to provide authorization to all members of a group. All users within the group will now have the role on the project specified.

$ openstack role add member --group "HiPODS-OpenCloud" --group-domain ibm \
  --project ibmcloud --project-domain ibm

4.4.3 Running LDAP User Commands

Setting up LDAP credentials

In this example, we attempt to authenticate as an LDAP user. To do this, we should set our environment variables to our LDAP credentials, and update the project domain and user domain values as necessary. The resultant settings should look like:

$ env | grep OS
OS_IDENTITY_API_VERSION=3
OS_PASSWORD=My5uper5ecretPa$$word
OS_AUTH_URL=http://172.16.240.134:5000/v3
OS_USERNAME=[e-mail]
OS_USER_DOMAIN_NAME=ibm
OS_PROJECT_NAME=ibmcloud
OS_PROJECT_DOMAIN_NAME=ibm

Getting a token

In this example, we attempt to authenticate and retrieve a token using our newly set credentials.

$ openstack token issue
+------------+------------------------------------------------------------------+
| Field      | Value                                                            |
+------------+------------------------------------------------------------------+
| expires    | 2015-08-13T21:54:15.411376Z                                      |
| id         | 7c0df68c6b734735a22b2365f241b9b8                                 |
| project_id | e68bd223fb8045f6b7b2b7aa926433c8                                 |
| user_id    | c29cac8f7003e6de36a47b85f306f137bea8026e3d55c35aed27dfbf09c8fb28 |
+------------+------------------------------------------------------------------+

Listing images

In this example, we attempt to list all public images that are available to the user. This highlights the fact that LDAP users are able to communicate with other OpenStack services.

$ openstack image list
+--------------------------------------+---------------------------------+
| ID                                   | Name                            |
+--------------------------------------+---------------------------------+
| 1f8a0809-5aa5-454a-916b-6077c00e20e1 | cirros-0.3.4-x86_64-uec         |
| 1d181a59-7ab7-4b5c-a351-118383d0b58b | cirros-0.3.4-x86_64-uec-ramdisk |
| b8aa23ee-1791-4a0e-b42a-cac6f7607b21 | cirros-0.3.4-x86_64-uec-kernel  |
+--------------------------------------+---------------------------------+

Creating a VM

In this example, we create a VM using the Nova service, once again illustrating that we are able to use other OpenStack services.

$ openstack server create myVM --image cirros-0.3.4-x86_64-uec --flavor 1
+--------------------------------------+---------------------------------------+
| Field                                | Value                                 |
+--------------------------------------+---------------------------------------+
| OS-EXT-AZ:availability_zone          | nova                                  |
| OS-EXT-STS:task_state                | scheduling                            |
| OS-EXT-STS:vm_state                  | building                              |
| adminPass                            | PQDNiboGaNQ2                          |
| created                              | 2015-08-14T07:35:36Z                  |
| flavor                               | m1.tiny (1)                           |
| id                                   | fe2b4ab0-80cf-48fe-ad5b-3e8b9624edd1  |
| image                                | cirros-0.3.4-x86_64-uec (1f8a0809-... |
| name                                 | myVM                                  |
| project_id                           | e68bd223fb8045f6b7b2b7aa926433c8      |
| security_groups                      | [{u'name': u'default'}]               |
| status                               | BUILD                                 |
| updated                              | 2015-08-14T07:35:36Z                  |
| user_id                              | c29cac8f7003e6de36a47b85f306f137be... |
+--------------------------------------+---------------------------------------+

4.4.4 Authenticating with Horizon

Updating the Horizon configuration file

In order for LDAP users to authenticate through Horizon, certain options need to be enabled. In the horizon/openstack_dashboard/local/local_settings.py file, make the following changes:

OPENSTACK_API_VERSIONS = {
    "data-processing": 1.1,
    "identity": 3,
    "volume": 2,
}
OPENSTACK_KEYSTONE_MULTIDOMAIN_SUPPORT = True
OPENSTACK_KEYSTONE_URL="http://172.16.240.134:5000/v3"

Log in with LDAP credentials and specify the domain name

In the screenshots below we can see the modified Horizon landing page, which now expects a user to specify the domain they want to authenticate against. 

Figure 4-1. Horizon landing page with an additional text box for the domain name

Once logged in, a user is able to see their username and domain along the top of the dashboard.

Figure 4-2. Once logged in, the user’s email address should be visible at the top right.

4.5 Projects, Roles, and Assignments from LDAP (Just Say NO!)

Identities in LDAP work well. They are typically managed by an LDAP administrator, using existing enterprise management tools (that most likely ensure an enterprise’s other systems are also maintained; e.g., human resources, etc.). When employees are hired or fired from the enterprise company, they are added and removed from LDAP accordingly. Similarly, when users are added and removed from groups in the corporate LDAP, Keystone’s view of the world is also updated (since, of course, Keystone is just using the corporate LDAP as its Identity store). This is a great fit for Identity and a common approach used by many applications.

Initially, when the LDAP driver was written in the Essex release, there was no separation between the Project, Role, and User services—they were all lumped together in the same backend together. This resulted in the first few iterations of LDAP support in Keystone forcing administrators to keep this data in their LDAP, a very unpopular restriction. Since then, the Keystone team has worked hard to separate the services and allow each to use their own backends (as described earlier, called, Identity, Assignment, and Resource). While Keystone still supports an LDAP driver for each of these three backends, we overwhelming suggest that you not store Assignment and Resource data in LDAP, but rather use a database such as SQL. The reasons for this are as follows:

• While there is a pretty good schema match for Identity with LDAP, the same is not true for Assignment and Resource. You will most likely have to modify your schema significantly to support OpenStack data.
• In general, Keystone needs the backends for Assignment and Resource to be read-write, something most corporate directory groups will feel uneasy about.
• Due to the lack of usage, the actual backend support for LDAP in Assignment and Resource is not as up to date as SQL. In particular, you can only use the default domain (the multi-domain support implemented in Identity is not provided with Assignment and Resource). In addition, domain role assignments are not supported on the default domain.

Given the above, as stated earlier, the LDAP backends for Assignment and Resource are likely to marked as deprecated in Liberty, for removal in the Mitaka release. For clarity, let us emphasize that this does not mean any reduction in support or commitment from Keystone to LDAP for Identity. This remains a fundamental part of Keystone and represents the most common backend for Identity in use in larger enterprise deployments.

4.6 Tips, Common Pitfalls, and Troubleshooting

In this section, we describe some commonly experienced pitfalls that occur when using LDAP and the multi-backend approach to identity in Keystone and provide troubleshooting tips to address these issues.

4.6.1 General LDAP Issues

Missing Python LDAP libraries

If a user is seeing either of the following errors, then they may be missing LDAP Python libraries (these libraries are not installed by default).

$ openstack user show [e-mail] --domain ibm
ERROR: openstack An unexpected error prevented the server from fulfilling your 
request: No module named ldap.filter (Disable debug mode to suppress these 
details.) (HTTP 500) (Request-ID: req-74bb843e-cc52-482e-9a25-4366a276ba3d)

or

$ openstack user show [e-mail] --domain ibm
ERROR: openstack An unexpected error prevented the server from fulfilling your 
request: No module named ldappool (Disable debug mode to suppress these 
details.) (HTTP 500) (Request-ID: req-74529f55-079b-4afd-b71d-6270fe4c1b2d)

To resolve either of these issues, install the required libraries:

$ sudo pip install python-ldap
$ sudo pip install ldappool

Or (as of the Liberty release of Keystone):

$ sudo pip install keystone[ldap]

Use tools to help you determine LDAP attributes

If you are unaware of the values for LDAP attributes that are necessary to configure Keystone, we suggest downloading software to help you determine these values. There are several free tools available, such as Xplorer, which have a graphical user interface that are easy to use. Using a simple value like name or email to find a single entry in LDAP is usually sufficient, since this will allow you to see all the fields available to every user.

4.6.2 Tips for Using Multi-Domain LDAP

When using the configuration file–based method, make sure you set up things in the right order

If you are starting with the classic configuration and you only have the default domain that points at a single LDAP/AD corporate directory, then the order of steps should be:

1. Set domain_specific_drivers_enabled to “true” in the main Keystone configuration file and restart Keystone.
2. Create the domain for which you want to have specific configurations.
3. Create the domain-specific configuration files for each of these domains.
4. Restart Keystone again.

With LDAP as the default driver, if you don’t first set domain_specific_drivers_enabled to true, Keystone will prevent you from creating new domains (since it knows the default LDAP driver can only handle the default domain).

If you have set up your domain and the configuration file but you still can’t access the data in the corporate directory, then it might be worth checking the Keystone log file. At startup, as Keystone is scanning the domain_config_dir, it will examine each file. If it doesn’t match the form keystone.{domain-name}.conf, then it will be ignored. If Keystone can’t find the domain of the name specified in the file name, then it will also ignore it. In both situations, Keystone will log a warning with the name of the problematic file. For example:

Invalid domain name (CustomerB) found in config file name

Remember, you can’t list all the users

As described earlier, with multi-domain, Keystone requires you to specify a domain when listing “all users.” If you fail to do this (and are not using a domain-scoped token, in which case this will imply the domain), then Keystone will return an error. This error can sometimes be a bit confusing—“Request Unauthorized” (HTTP error code 401), when one might expect it to return a “Bad Request” (HTTP error code 400).

You can’t move users between domains

As described earlier, with multi-domain LDAP, when Keystone encounters a user it will assign it a unique external ID it will publish to the rest of OpenStack (and itself, for assigning roles). This unique ID is effectively a hash of the domain and the local ID within the respective LDAP server). Hence, if you change the local ID or “move” that user to an LDAP server that is mapped to a different Keystone domain, as far as Keystone (and the rest of OpenStack) is concerned, this is a different user. None of the old assignments or project relationships will be carried across.

Occasional maintenance of the directory-mapping table

As we mentioned earlier, with multi-domain LDAP, Keystone dynamically builds the directory mapping of the external published user and group IDs to the underlying local entities in the appropriate corporate directory service. Since these corporate directories are usually read-only from a Keystone perspective, any user management will happen out-of-band via the tooling used to maintain the corporate directory in question. In practice, this means that over time as user and group entities are deleted out of the corporate directory, old mapping entries will remain in Keystone’s directory-mapping table. While this is benign, over time it could bloat the Keystone table. To manage this, the keystone-manage tool has been enhanced to support several options for purging entries out of the Keystone directory-mapping table. The most specific option is to purge the mapping for a given local identifier in a given domain. For example:

$ keystone-manage mapping_purge --domain-name CustomerA --local-id [e-mail]

If this is a relatively frequent occurrence, then you might want to automate such individual purging.

An alternative, however, is to simply purge all the mappings periodically for a given domain. A concern might be whether this will cause you to lose the externally published ID for local entities. In fact, that’s not an issue, since the algorithm for the mapping is designed such that Keystone can regenerate the mapping and guarantee to re-assign the same externally published user/group ID as before. Note that this assumes a user has not moved between corporate directories; if they have, then they are considered a different user. To purge all the mappings for a given domain:

$ keystone-manage mapping_purge --domain-name CustomerA