[Buildbot] #3510: LdapUserInfo groupMemberPattern barfs on non ASCII DNs

[Buildbot trac]

#3510: LdapUserInfo groupMemberPattern barfs on non ASCII DNs
----------------------+-------------------
Reporter:  gracinet   |      Owner:
    Type:  undecided  |     Status:  new
Priority:  minor      |  Milestone:  0.9.0
 Version:  0.9.0b7    |   Keywords:
----------------------+-------------------
 A typical ```groupMemberPattern``` would be ```'(member=%(dn)s)'```
 If the user DN happens to have non ASCII characters, this leads to

 {{{
           File "/srv/buildbot9/buildbot-
 git/master/buildbot/www/ldapuserinfo.py", line 68, in search
             c.search(base, filterstr, ldap3.SEARCH_SCOPE_WHOLE_SUBTREE,
 attributes=attributes)
           File "/srv/buildbot9/local/lib/python2.7/site-
 packages/ldap3/core/connection.py", line 568, in search
             request = search_operation(search_base, search_filter,
 search_scope, dereference_aliases, attributes, size_limit, time_limit,
 types_only, self.server.schema if self.server else None)
           File "/srv/buildbot9/local/lib/python2.7/site-
 packages/ldap3/operation/search.py", line 365, in search_operation
             request['filter'] = build_filter(search_filter, schema)  #
 parse the searchFilter string and compile it starting from the root node
           File "/srv/buildbot9/local/lib/python2.7/site-
 packages/ldap3/operation/search.py", line 312, in build_filter
             return compile_filter(parse_filter(search_filter,
 schema).elements[0])
           File "/srv/buildbot9/local/lib/python2.7/site-
 packages/ldap3/operation/search.py", line 220, in parse_filter
 current_node.append(evaluate_match(search_filter[start_pos:end_pos],
 schema))
           File "/srv/buildbot9/local/lib/python2.7/site-
 packages/ldap3/operation/search.py", line 180, in evaluate_match
             assertion = {'attr': left_part, 'value':
 validate_assertion_value(schema, left_part, right_part)}
           File "/srv/buildbot9/local/lib/python2.7/site-
 packages/ldap3/protocol/convert.py", line 126, in validate_assertion_value
             return value.encode('utf-8')
         exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte
 0xc3 in position 9: ordinal not in range(128)

 }}}

 Explanation: ldap3 returns the DN as an UTF-8 encoded ```str```. The
 resulting search filter is then also an UTF-8 encoded ```str```.

 The ```groupMemberPattern``` should have been an unicode string from the
 very beginning (```LdapUserInfo``` could promote it in its constructor
 since it's easy to forget in configuration), but that's still not enough:

 {{{

           File "/srv/buildbot9/buildbot-
 git/master/buildbot/www/ldapuserinfo.py", line 93, in thd
             pattern = self.groupMemberPattern % dict(dn=dn)
         exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte
 0xc3 in position 9: ordinal not in range(128)

 }}}

 That one is easy to fix: transcode ```dn``` if needed beforehand.
 This also will promote ```groupMemberPattern``` to unicode at line 93
 (pattern construction)

 In my opinion, ```ldap3``` should always return an unicode string, but
 buildbot can easily accept both ```unicode``` and UTF-8 encoded ```str```

--
Ticket URL: <http://trac.buildbot.net/ticket/3510>
Buildbot <http://buildbot.net/>
Buildbot: build/test automation