Keystone: LDAP for Identity, SQL for Assignment

This week I volunteered to work on playing around with integrating keystone with LDAP (via a FreeIPA box). Since I basically knew nothing about LDAP nor keystone before starting, I learned a few lessons along the way. And here they are:

The Setup

I started with a FreeIPA server that our team uses, I have full admin rights to it. I was also running an “All In One” openstack Havana instance on a VM that I setup using puppet_openstack_builder.

The Goal

My basic goal here was to learn more about both LDAP and keystone and in the process get the AIO instance to authenticate against LDAP as a preliminary test to authenticating against AD. In our environment, I wanted LDAP to manage Idenity (users, groups, and group memberships) and Keystone’s SQL backend to manage Assignment (roles, tenants, domains). This will work well when we integrate with AD since we cannot just create accounts on the corporate AD server willy-nilly. This is called Read Only LDAP and is covered in more detail here.

Diving In

There are several awesome blog entries about using FreeIPA and Keystone from Adam Young and these helped me get started. I’d configure keystone, restart it, then tail the logs while running keystone user-list.

Configuration

The very basic config is done like this:

First, enable the LDAP identity driver:
[identity]
driver = keystone.identity.backends.ldap.Identity
#driver = keystone.identity.backends.sql.Identity

Then you need to tell Keystone to use SQL for assignment:
[assignment]
driver = keystone.assignment.backends.sql.Assignment

Next we setup the user to which AD will authenticate since we’re using ldaps:

[ldap]
url = ldaps://example.com:636
user = uid=service_acct,cn=users,cn=accounts,dc=example,dc=com
password = UbuntuRulez

Then we tell it about the user and group schema:

user_tree_dn = cn=users,cn=accounts,dc=example,dc=com
user_filter = (memberOf=cn=openstack,cn=groups,cn=accounts,dc=example,dc=com)
user_objectclass = inetUser
user_id_attribute = uid
# this is what is searched on
user_name_attribute = uid
user_mail_attribute = mail
user_pass_attribute =
# XXX FIXME -mfisch: wont work on freeIPA like this
#user_enabled_attribute = (nsAccountLock=False)
user_allow_create = False
user_allow_update = False
user_allow_delete = False
...
group_tree_dn = cn=groups,cn=accounts,dc=example,dc=com
group_filter =
group_objectclass = groupOfNames
group_id_attribute = cn
group_name_attribute = cn
group_member_attribute = member
group_desc_attribute = description
# group_attribute_ignore =
group_allow_create = False
group_allow_update = False
group_allow_delete = False

Lastly we point at the cert info:
use_tls = False
tls_cacertfile = /etc/ssl/certs/ca-certificates.crt

The First Problem

The first stumbling block I hit is that I needed to use ldaps. This is a pretty basic fix, just grab the cert and put it on your box where keystone is running. Unfortunately I found out that unless I had the cert path also defined in /etc/ldap/ldap.conf, the query didn’t work. The fix is to make a pretty much empty /etc/ldap/ldap.conf and add this one line that points to the cert you pulled down:

TLS_CACERT /etc/ssl/certs/ca-certificates.crt

user_name_attribute

My next issue is that the default user_name_attribute is not right for FreeIPA. I checked this using Apache Directory Studio which made it easy for me to browse the tree and test queries. I needed to use uid and not cn. The issue here is that if you have this wrong, then the initial authentication of your LDAP user for the ldaps query fails and the resulting output provides no clue as to the issue, even with debug enabled. Once I solved this, I had a permission issue.

Bootstrapping

The problem once you switch the authentication mechanism to LDAP, you’ve “lost” all the old roles and users that puppet setup, like the admin user who actually has permissions to do stuff like list users. The fix as any keystone expert will know is to bypass the main API and use the service token directly. In the keystone.conf a service token is defined as admin_token. To use it, unset everything like OS_USERNAME and set:

export SERVICE_TOKEN=
export SERVICE_ENDPOINT=http://localhost:35357/v2.0

Then you need to give your user, in my case, “id=mfischer” permission to do stuff. I ended up giving mfischer the admin and _member_ role in both of my tenants since he/me is the new admin. Finally, I switched my settings back to what’s in my rc file, mfischer as the user, my LDAP password, and the normal keystone endpoint, and… finally my user-list query worked and I got results from LDAP.

root@landscape-03:/var/log/keystone# keystone user-list
+------------+------------+---------+--------------------------+
| id | name | enabled | email |
+------------+------------+---------+--------------------------+
| mfischer | mfischer | | matt.fischer@example.com |
....

User Enabled?

keystone wants to know if a user is enabled and shows this as a column in the output of user-list. All my results were blank, and it’s not clear that FreeIPA has a straightforward “enabled” column in our setup. At a glance this stuff seemed optimized for AD’s user-enabled mask stuff. (If you know a fix, let me know please).

Trying Other Services

I decided to try nova list next and guess what, it failed. It’s obvious to me now, but the puppet AIO has users created for almost all the services, like nova, cinder, glance, etc. I needed users for these. This is easy to do in FreeIPA, so I made a batch of users and restarted the node. I bet you can guess, but stuff still failed because these users didn’t have the roles that they needed. At this point, I switched the identity backend back to SQL, restarted keystone and made a map of roles to users and tenants. In my case it was pretty basic, everything needed admin in the services tenant and some needed more, here’s how to do it simply:

for I in 'glance' 'nova' 'cinder' 'neutron' 'heat' 'heat-cfn' 'swift';
do; keystone user-role-add --user-id=$I --tenant-id=de4442c6e54a43459eaab97e26dc21f8 --role id=a5a8ea228b1942e28289ba63fba9b3c0; done

I did this and then bounced the node again (which is frankly simpler than restarting everything but unlikely to work in the real world). And again nothing worked! I’d forgotten one more step, each service has a password defined in it’s config file as “admin_password”. I signed back into FreeIPA and set the passwords and then bounced the node a final time.

This time, I could sign into Horizon and everything worked great! Finally, I was using LDAP only for identity and didn’t need to switch back.

Obviously this is much better to be done before you start your puppet install but in my case it was a great learning experience about roles/tenants/users and the tools that keystone provides.

Debugging Tools/Hints

Here are some more debugging tools and hints that I came across and maybe will help you if you’re dealing with LDAP and keystone:

  1. Enable debug and verbose in keystone, of course.
  2. I’ve mentioned Apache Directory Studio so you can test LDAP queries. This command also helps show what fields are available and is simpler than using ADS: “ipa user-show –all –raw”.
  3. The LDAP code in keystone has double secret logging which you can enable in /usr/lib/python2.7/dist-packages/keystone/common/ldap/core.py. You can look for “ldap.set_option(ldap.OPT_DEBUG_LEVEL, 4095)” and uncomment it. These logs only show on stdout, so you’ll need to stop the service and run it by hand to see this output.
  4. I also traced some code in pdb, in addition to the file listed above you should also look at /usr/lib/python2.7/dist-packages/keystone/identity/backends/ldap.py

Good luck everyone!

Tagged , ,

13 thoughts on “Keystone: LDAP for Identity, SQL for Assignment

  1. Yuriy Taraday says:

    Hello.

    We’ve created ‘enabled_emulation’ configuration options exactly for LDAP backends that don’t have any specific ‘enabled’ fields. You can turn it on for users and/or for projects. Just add ‘user_enabled_emulation=True’ to ‘[ldap]‘ section of your config.

  2. If you’re using FreeIPA you could use this:

    user_enabled_attribute = nsAccountLock
    user_enabled_mask = 1

    That is in accordance to FreeIPA directory.

    However I noticed problems in Python (keystone) depending on where I created the user. If I used ipa on the command line to create the user I had a type error on Python. If I created it using the web interface of FreeIPA I had not.

  3. I also wanted to comment about enable emulation. I don’t like very much that solution but if one want to use it one should configure it something like this:

    user_enabled_emulation = True
    user_enabled_emulation_dn = cn=enabled_users,cn=Groups,cn=Accounts,dn=example,dn=org

    were enabled_users is a group (it can be created with the FreeIPA UI). Then one adds a “member” attribute to enabled_users for each user one wants to enable. The member attribute needs to have as value the user id.

    The same should work for enabled emulation on tenants (projects).

    I am actually using Read Only LDAP, therefore my assignment backend is still SQL. The identity backend is LDAP.

  4. Mahesh Sawaiker says:

    Worked great, only problem is looks like open-ldap client wont pick up any cert file unless its in the certs directory.

  5. Mahesh Sawaiker says:

    I also noted that it works without tls_cacertfile = /etc/ssl/certs/ca-certificates.crt in the keystone.conf, somehow all it really cares about is the setting in the openldap client config

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>