LDAP groups_from_user auth


#1

Hi

Since 2 days I’ve been trying to setup RoR with LDAP auth to use memberOf of the users to get their groups (to no avail).

In the tcpdump I see my users being authenticated successfully, then RoR asks LDAP for memberOf of this user, and LDAP correctly responds with several groups, e.g.:

CN=group1,OU=ou4,OU=ou3,OU=ou2,OU=ou1,DC=organization,DC=com
CN=group2,OU=ou4,OU=ou3,OU=ou2,OU=ou1,DC=organization,DC=com
CN=group3,OU=ou4,OU=ou3,OU=ou2,OU=ou1,DC=organization,DC=com

In my understanding now this rule should be matched and user authorized:

 - name: 'rule1'
   ldap_auth:
     name: 'ldap1'                                    
     cache_ttl_in_sec: 60
     groups: [ 'group2' ]                              
   indices: [ 'logstash*' ]
   actions: [ 'cluster:monitor/main', 'indices:admin/mappings/fields/get', 'indices:admin/validate/query', 'indices:admin/get' ]

But then it for some reason doesn’t, and I get “the request matches no rules in this block”, [rule1->[ldap_authentication->false]] , and authorization fails.

my ldaps config:

   ldaps:
    - name: ldap1
      host: 'ldap.organization.com'
      ssl_enabled: false                                         
      ssl_trust_all_certs: true                                
      bind_dn: 'cn=ldap,DC=organization,DC=com'                   
      bind_password: 'XXXXX'                                 
      search_user_base_DN: 'OU=ou5,DC=organization,DC=com'
      user_id_attribute: 'SAMAccountName'                                 
      search_groups_base_DN: 'OU=ou4,OU=ou3,OU=ou2,OU=ou1,DC=organization,DC=com'
      groups_from_user: true
      connection_pool_size: 10                                 
      connection_timeout_in_sec: 10                             
      request_timeout_in_sec: 10                               
      cache_ttl_in_sec: 60                                     

I’m using readonlyrest-1.16.19_es5.6.8.zip with ES 5.6.8

What am I doing wrong? Should I put something else in the rule’s ‘groups’ to match it?

Thank you very much!


(Simone Scarduzio) #2

Hi @metsaviha!

Interesting, can you paste the log line that says “FORBIDDEN” for such request?


#3

Sure. here you go

[2018-06-02T07:11:05,267][INFO ][t.b.r.a.ACL ] ESC[35mFORBIDDEN by default req={ ID:789645391--2094074344#2487, TYP:SearchRequest, CGR:N/A, USR:my.user, BRS:false, KDX:null, ACT:indices:data/read/search, OA:10.1.1.2, DA:10.1.1.1, IDX:.kibana, MET:POST, PTH:/.kibana/_search, CNT:<OMITTED, LENGTH=279>, HDR:{authorization=Basic XXXXXXXXXXXXXXXXXXXXXX, Connection=keep-alive, Authorization=<OMITTED>, content-type=application/json, Host=10.1.1.1:9200, Content-Length=279}, HIS:[Kibana system user->[auth_key->false]], [Logstash rule->[auth_key->false]], [rule1->[ldap_authorization->false]] }

Thank you


(Simone Scarduzio) #4

Hey @metsaviha, we had some fixes about LDAP and groups handling, could you check if you can reproduce this with ROR 1.16.20-pre18?


#5

Hi @sscarduzio

Thank you for the suggestion, but nothing changed after I re-tested with 1.16.20-pre18

Here I’ll try to show my setup and logs as close to my real settings as possible:

So here is the rule and ldaps section:

- name: 'My rule'
  ldap_auth:
    name: 'ldap1'
    groups: ['GROUP-2']
  kibana_access: rw
  kibana_index: '.kibana*'
  indices: [ '*logstash*', '.kibana*' ]
  actions: [ 'cluster:monitor/main', 'indices:admin/mappings/fields/get', 'indices:admin/validate/query', 'indices:admin/get' ]

ldaps:
- name: ldap1
  host: 'ldap.organization.com'
  #port: 636                                                 # optional, default 389
  ssl_enabled: false                                       # optional, default true
  ssl_trust_all_certs: true                                 # optional, default false
  bind_dn: 'cn=ldap,ou=xxxx,ou=smth,OU=Place\, Country,OU=ZZZ and VVVV,DC=organization,DC=com'
  bind_password: 'bindpassword'
  search_user_base_DN: 'OU=This and That,DC=organization,DC=com'
  user_id_attribute: 'SAMAccountName'                                 
  search_groups_base_DN: 'OU=Floor,OU=Place,OU=My Groups,OU=XXX & YYYY,DC=organization,DC=com'
  #unique_member_attribute: 'uniqueMember'               
  groups_from_user: true
  connection_pool_size: 10                                  # optional, default 30
  connection_timeout_in_sec: 10                             # optional, default 1
  request_timeout_in_sec: 10                                # optional, default 1
  cache_ttl_in_sec: 60                                      # optional, default 0 - cache disabled

tcpdump -A of the memberof request:

08:15:59.025934 IP 10.0.0.1.35702 > 192.168.1.1.ldap: Flags [P.], seq 105:244, ack 2719, win 271, options [nop,nop,TS val 562671899 ecr 886277594], length 139
[email protected]@.Ne
..H..B..v....t..t.k.... ......
!...4...0.....c...XCN=My User,OU=Users,OU=Floor,OU=Place,OU=This and That,DC=organization,DC=com
..
.............objectClass0
..memberOf
08:15:59.441694 IP 192.168.1.1.ldap > 10.0.0.1.35702: Flags [P.], seq 2719:3155, ack 244, win 258, options [nop,nop,TS val 886277595 ecr 562671899], length 436
[email protected]{R..B.
..H...v.t.k..u(.....].....
4...!...0........d......XCN=My User,OU=Users,OU=Floor,OU=Place,OU=This and That,DC=organization,DC=com0..../0....)..memberOf1......]CN=GROUP-1,OU=Floor,OU=Place,OU=My Groups,OU=XXX & YYYY,DC=organization,DC=com.ZCN=GROUPA&B,OU=Floor,OU=Place,OU=My Groups,OU=XXX & YYYY,DC=organization,DC=com.\CN=GROUP-2,OU=Floor,OU=Place,OU=My Groups,OU=XXX & YYYY,DC=organization,DC=com0........e.....

And the debug log of the same request:

[2018-06-03T08:15:58,902][DEBUG][t.b.r.a.b.r.i.LdapAuthenticationAsyncRule] Attempting Login as: my.user rc: { ID:310685897--663038645#1010, TYP:SearchRequest, CGR:N/A, USR:my.user(?), BRS:false, KDX:null, ACT:indices:data/read/search, OA:10.0.0.2, DA:10.0.0.1, IDX:.kibana, MET:POST, PTH:/.kibana/_search, CNT:{"version":true,"size":1,"query":{"bool":{"should":[{"bool":{"must":[{"term":{"_id":"5.6.8"}},{"term":{"_type":"config"}}]}},{"bool":{"must":[{"term":{"_id":"config:5.6.8"}},{"term":{"type":"config"}}]}},{"bool":{"must":[{"term":{"_id":"5.6.8"}},{"term":{"type":"config"}}]}}]}}}, HDR:{authorization=Basic XXXXXXXXXXXXXXXXXXXXX, Connection=keep-alive, Content-Length=279, content-type=application/json, Host=10.0.0.1:9200}, HIS:[Admin user->[auth_key->false]], [Kibana system user->[auth_key->false]], [Logstash rule->[auth_key->false]] }
[2018-06-03T08:15:59,024][DEBUG][t.b.r.a.d.l.u.UnboundidGroupsProviderLdapClient] LDAP search string: CN=My User,OU=Users,OU=Floor,OU=Place,OU=This and That,DC=organization,DC=com  |  groupsFromUserAttribute: memberOf
[2018-06-03T08:15:59,044][DEBUG][t.b.r.a.b.Block          ] ESC[33m[My rule] the request matches no rules in this block: { ID:310685897--663038645#1010, TYP:SearchRequest, CGR:N/A, USR:my.user, BRS:false, KDX:null, ACT:indices:data/read/search, OA:10.0.0.2, DA:10.0.0.1, IDX:.kibana, MET:POST, PTH:/.kibana/_search, CNT:{"version":true,"size":1,"query":{"bool":{"should":[{"bool":{"must":[{"term":{"_id":"5.6.8"}},{"term":{"_type":"config"}}]}},{"bool":{"must":[{"term":{"_id":"config:5.6.8"}},{"term":{"type":"config"}}]}},{"bool":{"must":[{"term":{"_id":"5.6.8"}},{"term":{"type":"config"}}]}}]}}}, HDR:{authorization=Basic XXXXXXXXXXXXXXXXXXXX, Connection=keep-alive, Content-Length=279, content-type=application/json, Host=10.0.0.1:9200}, HIS:[Admin user->[auth_key->false]], [Kibana system user->[auth_key->false]], [Logstash rule->[auth_key->false]], [My rule->[ldap_authorization->false]] }ESC[0m
[2018-06-03T08:15:59,050][DEBUG][r.suppressed             ] path: /.kibana/_search, params: {index=.kibana}
tech.beshu.ror.es.IndexLevelActionFilter$1$1: Forbidden
        at tech.beshu.ror.es.IndexLevelActionFilter$1.onForbidden(IndexLevelActionFilter.java:176) ~[?:?]

[...skip...]

[2018-06-03T08:15:59,088][INFO ][t.b.r.a.ACL              ] ESC[35mFORBIDDEN by default req={ ID:310685897--663038645#1010, TYP:SearchRequest, CGR:N/A, USR:my.user, BRS:false, KDX:null, ACT:indices:data/read/search, OA:10.0.0.2, DA:10.0.0.1, IDX:.kibana, MET:POST, PTH:/.kibana/_search, CNT:{"version":true,"size":1,"query":{"bool":{"should":[{"bool":{"must":[{"term":{"_id":"5.6.8"}},{"term":{"_type":"config"}}]}},{"bool":{"must":[{"term":{"_id":"config:5.6.8"}},{"term":{"type":"config"}}]}},{"bool":{"must":[{"term":{"_id":"5.6.8"}},{"term":{"type":"config"}}]}}]}}}, HDR:{authorization=Basic XXXXXXXXXXXXXXXXXXXXXXX, Connection=keep-alive, Content-Length=279, content-type=application/json, Host=10.0.0.1:9200}, HIS:[Admin user->[auth_key->false]], [Kibana system user->[auth_key->false]], [Logstash rule->[auth_key->false]], [My rule->[ldap_authorization->false]] } ESC[0m

Thank you


(Simone Scarduzio) #6

Ok let’s go with order and we clean this up as well as possible so mistakes and anomalies will pop up better.

FIRST AND FOREMOST

This “actions” rule has no place in your ACL block, as you are already using “kibana_access” which is a macro rule which underpins the correct actions set for a Kibana session to happen.

NEXT ISSUE

The “kibana_index” rule does not accept wildcards, as a Kibana session can be backed by the settings in a single kibana index at a time. Just remove it if you mean the kibana index to be “.kibana” which is anyways the deafault.

NEXT ISSUE

As above, replace .kibana* with .kibana or even better, remove this rule entirely, as we keep things as simple as possible to identify another issue.

FOR EASIER TESTING

Set this to 0 for debugging, so we can see things change faster when we tweak stuff.


#7

Hi @sscarduzio

Thank you for your reply.

Well, not that I think it is relevant to the LDAP issue, but ok, here I have re-tested it with your suggestions (with the very same result).

Config (full this time, just in case):

readonlyrest:

ssl:
  enable: true
  keystore_file: '/etc/es.jks'
  keystore_pass: password

response_if_req_forbidden: Forbidden

access_control_rules:

- name: 'Admin rule'
  auth_key: admin_user:password
  verbosity: error

- name: 'Kibana rule'
  auth_key: kibana_system_user:password
  verbosity: error

- name: 'Logstash rule'
  auth_key: logstash_user:password
  indices: [ '*logstash*' ]
  actions: [ 'indices:admin/template/get', 'indices:admin/template/put', 'indices:data/write/index', 'indices:data/write/bulk', 'indices:data/write/delete', 'indices:data/write/update', 'indices:data/read/search', 'indices:data/read/scroll', 'indices:admin/mapping/put' ]
  verbosity: error

- name: 'My rule'
  ldap_auth:
    name: 'ldap1'
    groups: ['GROUP-2']
  kibana_access: rw
  kibana_index: '.kibana'

ldaps:
- name: ldap1
  host: 'ldap.organization.com'
  ssl_enabled: false
  ssl_trust_all_certs: true
  bind_dn: 'cn=ldap,ou=xxxx,ou=smth,OU=Place\, Country,OU=ZZZ and VVVV,DC=organization,DC=com'
  bind_password: 'bindpassword'
  search_user_base_DN: 'OU=This and That,DC=organization,DC=com'
  user_id_attribute: 'SAMAccountName'                         
  search_groups_base_DN: 'OU=Floor,OU=Place,OU=My Groups,OU=XXX & YYYY,DC=organization,DC=com'
  groups_from_user: true

tcpdump -A (memberOf request/response):

12:50:57.960084 IP 10.0.0.1.36472 > 192.168.1.1.ldap: Flags [P.], seq 105:244, ack 2719, win 271, options [nop,nop,TS val 586844745 ecr 895947079], length 139
E…<@[email protected]
…H…B…x…1…zv..... ...... "..I5g.G0.....c...XCN=My User,OU=Users,OU=Floor,OU=Place,OU=This and That,DC=organization,DC=com .. .............objectClass0 ..memberOf 12:50:57.962072 IP 192.168.1.1.ldap > 10.0.0.1.36472: Flags [P.], seq 2719:3155, ack 244, win 258, options [nop,nop,TS val 895947080 ecr 586844745], length 436 [email protected] ..H...xzv.1…TU…
5g.H"…I0…d…XCN=My User,OU=Users,OU=Floor,OU=Place,OU=This and That,DC=organization,DC=com0…/0…)…memberOf1…]
CN=GROUP-1,OU=Floor,OU=Place,OU=My Groups,OU=XXX & YYYY,DC=organization,DC=com.ZCN=GROUPA&B,OU=Floor,OU=Place,OU=My Groups,OU=XXX & YYYY,DC=organization,DC=com.\CN=GROUP-2,OU=Floor,OU=Place,OU=My Groups,OU=XXX & YYYY,DC=organization,DC=com0…e…

ES debug log:

[2018-06-04T12:50:57,959][DEBUG][t.b.r.a.d.l.u.UnboundidGroupsProviderLdapClient] LDAP search string: My User,OU=Users,OU=Floor,OU=Place,OU=This and That,DC=organization,DC=com | groupsFromUserAttribute: memberOf
[2018-06-04T12:50:58,000][DEBUG][t.b.r.a.b.Block ] ESC[33m[My rule] the request matches no rules in this block: { ID:2116931118–430133616#4593, TYP:SearchRequest, CGR:N/A, USR:my.user, BRS:false, KDX:null, ACT:indices:data/read/search, OA:10.1.1.2, DA:10.1.1.1., IDX:.kibana, MET:POST, PTH:/.kibana/_search, CNT:{“version”:true,“size”:1,“query”:{“bool”:{“should”:[{“bool”:{“must”:[{“term”:{"_id":“5.6.8”}},{“term”:{"_type":“config”}}]}},{“bool”:{“must”:[{“term”:{"_id":“config:5.6.8”}},{“term”:{“type”:“config”}}]}},{“bool”:{“must”:[{“term”:{"_id":“5.6.8”}},{“term”:{“type”:“config”}}]}}]}}}, HDR:{authorization=Basic XXXXXXXXXXXXXXXXXXXXX, Connection=keep-alive, Content-Length=279, content-type=application/json, Host=10.1.1.1:9200}, HIS:[Admin rule->[auth_key->false]], [Kibana rule->[auth_key->false]], [Logstash rule->[auth_key->false]], [My rule->[ldap_authorization->false]] }ESC[0m
[2018-06-04T12:50:58,001][DEBUG][r.suppressed ] path: /.kibana/_search, params: {index=.kibana}
tech.beshu.ror.es.IndexLevelActionFilter$1$1: Forbidden

[2018-06-04T12:50:58,014][INFO ][t.b.r.a.ACL ] ESC[35mFORBIDDEN by default req={ ID:2116931118–430133616#4593, TYP:SearchRequest, CGR:N/A, USR:my.user, BRS:false, KDX:null, ACT:indices:data/read/search, OA:10.1.1.2, DA:10.1.1.1, IDX:.kibana, MET:POST, PTH:/.kibana/_search, CNT:{“version”:true,“size”:1,“query”:{“bool”:{“should”:[{“bool”:{“must”:[{“term”:{"_id":“5.6.8”}},{“term”:{"_type":“config”}}]}},{“bool”:{“must”:[{“term”:{"_id":“config:5.6.8”}},{“term”:{“type”:“config”}}]}},{“bool”:{“must”:[{“term”:{"_id":“5.6.8”}},{“term”:{“type”:“config”}}]}}]}}}, HDR:{authorization=Basic XXXXXXXXXXXXXXXXXXXX, Connection=keep-alive, Content-Length=279, content-type=application/json, Host=10.1.1.1:9200}, HIS:[Admin rule->[auth_key->false]], [Kibana rule->[auth_key->false]], [Logstash rule->[auth_key->false]], [My rule->[ldap_authorization->false]] } ESC[0m

Thank you very much


(Simone Scarduzio) #8

Hi @metsaviha,

can you read these log lines being printed at all?

logger.debug("Trying to fetch user [id=" + user.getUid() + ", dn" + user.getDN() + "] groups from LDAP [" + name + "]");

logger.debug("LDAP [" + name + "] returned for user [" + user.getUid() + "] following groups: " +
                       "[" + Joiner.on(", ").join(groups.stream().map(LdapGroup::getName).collect(Collectors.toSet())) + "]");

#9

No, I don’t have these in my logs.

Just in case, I set ES debug by PUTting {“transient”:{“logger”: {"_root":“DEBUG”}}}’. Is it ok?

Thank you


(Simone Scarduzio) #10

Nope, in log4j2.properties

rootLogger.level=debug

#11

Thanks, now I have these messages:

[2018-06-05T20:01:41,624][DEBUG][t.b.r.a.d.l.l.AuthenticationLdapClientLoggingDecorator] Trying to authenticate user [my.user] with LDAP [ldap1]
[2018-06-05T20:01:41,642][DEBUG][t.b.r.a.d.l.l.AuthenticationLdapClientLoggingDecorator] User [my.user] authenticated by LDAP [ldap1]
[2018-06-05T20:01:41,642][DEBUG][t.b.r.a.d.l.l.AuthenticationLdapClientLoggingDecorator] Trying to fetch user with identifier [my.user] from LDAP [
ldap1]
[2018-06-05T20:01:41,647][DEBUG][t.b.r.a.d.l.l.AuthenticationLdapClientLoggingDecorator] User with identifier [my.user] found [dn = CN=My User,OU=Users,OU=Floor,OU=Place,OU=This and That,DC=organization,DC=com]
[2018-06-05T20:01:41,647][DEBUG][t.b.r.a.d.l.l.GroupsProviderLdapClientLoggingDecorator] Trying to fetch user [id=my.user, dnCN=My User,OU=Users,OU=Floor,OU=Place,OU=This and That,DC=organization,DC=com] groups from LDAP [ldap1]
[2018-06-05T20:01:41,648][DEBUG][t.b.r.a.d.l.u.UnboundidGroupsProviderLdapClient] LDAP search string: CN=My User,OU=Users,OU=Floor,OU=Place,OU=This and That,DC=organization,DC=com | groupsFromUserAttribute: memberOf
[2018-06-05T20:01:41,653][DEBUG][t.b.r.a.d.l.l.GroupsProviderLdapClientLoggingDecorator] LDAP [ldap1] returned for user [my.user] following groups:
[]

So it doesn’t detect any groups in the ldap response for some reason (and ldap does indeed return the groups - see the tcpdump above)

Thank you


(Simone Scarduzio) #12

I found a few undocumented options (good time now for fixing it).

private static final String UNIQUE_MEMBER = “unique_member_attribute”;
private static final String GROUP_SEARCH_FILTER = “group_search_filter”;
private static final String GROUP_NAME_ATTRIBUTE = “group_name_attribute”;
private static final String GROUPS_FROM_USER = “groups_from_user”;
private static final String GROUPS_FROM_USER_ATTRIBUTE = “groups_from_user_attribute”;

You could try to add in your ldap connector definition: group_from_user: true


#13

But I already have group_from_user set in my ldap conf.
And that’s exactly what i am trying to do - to authorize by getting user’s memberOf (default of groups_from_user_attribute) groups


(Simone Scarduzio) #14

Hmm got it.
Hey @metsaviha, I created a new build for you with more logs. It will basically print more about what has returned from the LDAP server and how are we going to extract it.


#15

Hi @sscarduzio

Thank you very much and sorry for the delay
In the end I’ve settled with the default LDAP groups handling, which is working ok.

But for the sake of science, let’s get down to the root of the original issue.

So it looks much clearer now.

First of all, RoR gets only one memberOf line instead of 3 sent by LDAP (compare with the tcpdump above)
Secondly, it doesn’t resolve even that one into a group name:

[2018-06-10T16:58:19,084][DEBUG][t.b.r.a.d.l.u.UnboundidGroupsProviderLdapClient] getGroupsFromUser got 1 responses.
[2018-06-10T16:58:19,086][DEBUG][t.b.r.a.d.l.u.UnboundidGroupsProviderLdapClient] getGroupsFromUser responses:
sg=CN=My User,OU=Users,OU=Floor,OU=Place,OU=This and That,DC=organization,DC=com, attrs={ memberOf=CN=GROUP-1,OU=Floor,OU=Place,OU=My Groups,OU=XXX & YYYY,DC=organization,DC=com }
[2018-06-10T16:58:19,086][DEBUG][t.b.r.a.d.l.u.UnboundidGroupsProviderLdapClient] Will now get attribute values of memberOf
[2018-06-10T16:58:19,104][DEBUG][t.b.r.a.d.l.l.GroupsProviderLdapClientLoggingDecorator] LDAP [ldap1] returned for user [my.user] following groups: []