Need to connect elasticsearch with Active Directory for authentication - Need help

Ok , here the methodes that I use on my side, and it is used with a LDAP user :

curl.exe --cacert D:\certs\rootca_cert.pem --cert-type PEM -H "Authorization: Basic bGRhcHRlc3Q6bXlwYXNzd29yZA==" -XGET "https://10.10.10.87:9200/toto/_search?size=10000&pretty=true"

here bGRhcHRlc3Q6bXlwYXNzd29yZA== is base64 of ldap user as ldaptest:mypassword

and it works

Now I gonna try now with your approach with clear username password

here a second approach which works :

curl.exe --cacert D:\certs\rootca_cert.pem --cert-type PEM -XGET "https://ldaptest:mypassword@10.10.10.87:9200/toto/_search?size=10000&pretty=true"

this one works too on my side

and :

curl.exe --cacert D:\certs\rootca_cert.pem --cert-type PEM -v --basic -u ldaptest:mypassword -XGET "https://10.10.10.87:9200/toto/_search?size=10000&pretty=true"

works too

To clarify, as you see i never write the domain name

Also, I have no clue how to set the domain name, if the situation would request it (2 same logon name, different domain).

@sscarduzio, do you already have an hint ? ( I did not look for that yet)

Well, @sim, how it goes ?

`curl.exe --cacert D:\certs\rootca_cert.pem --cert-type PEM -XGET "https://ldaptest:mypassword@10.10.10.87:9200/toto/_search?size=10000&pretty=true"`

I cannot use this as I have special character in my password

Tried this and still getting unauthorized error.

curl.exe --cacert D:\certs\rootca_cert.pem --cert-type PEM -v --basic -u ldaptest:mypassword -XGET "https://10.10.10.87:9200/toto/_search?size=10000&pretty=true"

@sscarduzio I get this error as well but i don’t see any other LDAP debug logs

[2018-11-12T16:32:51,591][DEBUG][i.n.u.NetUtil            ] Failed to get SOMAXCONN from sysctl and file /proc/sys/net/core/somaxconn. Default: 128
java.security.AccessControlException: access denied ("java.io.FilePermission" "/proc/sys/net/core/somaxconn" "read")

Could this be the reason?

This is just a netty optimisation, it’s totally harmless.

There must be something else above that.

BTW That stack trace is not pathologic, it’s just shown by ROR in debug mode in case no block has matched (I know, it’s just ugly).

Consider my experiment:

curl -k -ucartman:user2 'https://localhost:9200'
[2018-11-13T13:12:36,906][DEBUG][t.b.r.a.b.r.i.LdapAuthenticationAsyncRule] Attempting Login as: cartman rc: { ID:589822737-529412023#84, TYP:MainRequest, CGR:N/A, USR:cartman(?), BRS:true, KDX:null, ACT:cluster:monitor/main, OA:127.0.0.1, DA:0:0:0:0:0:0:0:1, IDX:<N/A>, MET:GET, PTH:/, CNT:<N/A>, HDR:{Accept=*/*, Authorization=Basic Y2FydG1hbjp1c2VyMg==, content-length=0, Host=localhost:9200, User-Agent=curl/7.54.0}, HIS:[Kibana->[auth_key->false]] }
[2018-11-13T13:12:36,906][DEBUG][t.b.r.a.d.l.l.AuthenticationLdapClientLoggingDecorator] Trying to authenticate user [cartman] with LDAP [ldap1]
[2018-11-13T13:12:36,908][DEBUG][t.b.r.a.d.l.l.AuthenticationLdapClientLoggingDecorator] User [cartman]  authenticated by LDAP [ldap1]
[2018-11-13T13:12:36,909][DEBUG][t.b.r.a.d.l.l.AuthenticationLdapClientLoggingDecorator] Trying to fetch user with identifier [cartman] from LDAP [ldap1]
[2018-11-13T13:12:36,911][DEBUG][t.b.r.a.d.l.l.AuthenticationLdapClientLoggingDecorator] User with identifier [cartman] found [dn = cn=Eric Cartman,ou=People,dc=example,dc=com]
[2018-11-13T13:12:36,912][DEBUG][t.b.r.a.d.l.l.GroupsProviderLdapClientLoggingDecorator] Trying to fetch user [id=cartman, dncn=Eric Cartman,ou=People,dc=example,dc=com] groups from LDAP [ldap1]
[2018-11-13T13:12:36,914][DEBUG][t.b.r.a.d.l.l.GroupsProviderLdapClientLoggingDecorator] LDAP [ldap1] returned for user [cartman] following groups: [group3, groupAll, group1]
[2018-11-13T13:12:36,916][DEBUG][t.b.r.a.b.Block          ] matched { name: '::g1::', policy: ALLOW, rules: [kibana_index, ldap_auth]}
[2018-11-13T13:12:36,926][INFO ][t.b.r.a.ACL              ] ALLOWED by { name: '::g1::', policy: ALLOW, rules: [kibana_index, ldap_auth]} req={ ID:589822737-529412023#84, TYP:MainRequest, CGR:N/A, USR:cartman, BRS:true, KDX:.kibana_group1, ACT:cluster:monitor/main, OA:127.0.0.1, DA:0:0:0:0:0:0:0:1, IDX:<N/A>, MET:GET, PTH:/, CNT:<N/A>, HDR:{Accept=*/*, Authorization=Basic Y2FydG1hbjp1c2VyMg==, content-length=0, Host=localhost:9200, User-Agent=curl/7.54.0}, HIS:[Kibana->[auth_key->false]], [::g1::->[ldap_authorization->true, kibana_index->true]] } 

Can you see how much information I get from the LDAP connector?

User with identifier [cartman] found [dn = cn=Eric Cartman,ou=People,dc=example,dc=com]
...
 Trying to fetch user [id=cartman, dncn=Eric Cartman,ou=People,dc=example,dc=com] groups from LDAP [ldap1]
...
 LDAP [ldap1] returned for user [cartman] following groups: [group3, groupAll, group1]
...

This stuff is very helpful

@sscarduzio

Sorry i missed those logs, why is not able to find the cn? and i have a special character in my password, does that make any difference?

[2018-11-13T16:23:59,023][DEBUG][t.b.r.a.b.r.i.LdapAuthenticationAsyncRule] Attempting Login as: elkadmin@appdev.local rc: { ID:1809111936-1186442698#12779, TYP:MainRequest, CGR:N/A, USR:elkadmin@appdev.local(?), BRS:true, KDX:null, ACT:cluster:monitor/main, OA:172.********, DA:172.********, IDX:<N/A>, MET:GET, PTH:/, CNT:<N/A>, HDR:{Accept=*/*, Authorization=Basic ZWxrYWRtaW5AYXBwZGV2LmxvY2FsOnNvY0AyMDE4, content-length=0, Host=readlogs.appdev.local:9200, User-Agent=curl/7.58.0}, HIS:[kibana server->[auth_key->false]], [::LOGSTASH::->[auth_key->false]] }
[2018-11-13T16:23:59,023][DEBUG][t.b.r.a.d.l.l.AuthenticationLdapClientLoggingDecorator] Trying to authenticate user [elkadmin@appdev.local] with LDAP [appdev]
[2018-11-13T16:23:59,028][DEBUG][t.b.r.a.d.l.u.UnboundidAuthenticationLdapClient] LDAP getting user CN returned no entries
[2018-11-13T16:23:59,028][DEBUG][t.b.r.a.d.l.l.AuthenticationLdapClientLoggingDecorator] User [elkadmin@appdev.local] not authenticated by LDAP [appdev]
[2018-11-13T16:23:59,028][DEBUG][t.b.r.a.b.Block          ] [Accept requests from users in group team] the request matches no rules in this block: { ID:1809111936-1186442698#12779, TYP:MainRequest, CGR:N/A, USR:elkadmin@appdev.local(?), BRS:true, KDX:null, ACT:cluster:monitor/main, OA:172.*******, DA:172.*********, IDX:<N/A>, MET:GET, PTH:/, CNT:<N/A>, HDR:{Accept=*/*, Authorization=Basic ZWxrYWRtaW5AYXBwZGV2LmxvY2FsOnNvY0AyMDE4, content-length=0, Host=readlogs.appdev.local:9200, User-Agent=curl/7.58.0}, HIS:[kibana server->[auth_key->false]], [::LOGSTASH::->[auth_key->false]], [Accept requests from users in group team->[ldap_authentication->false]] }

I don’t think it matters the special char. The search parameters are off though.
Can you show us your successful ldapsearch command?
Also, please try to specify a user_id_attribute like they do here

successful ldapsearch command


ldapsearch
-x -h 172.**********
-D “elkadmin@appdev.local”
-W
-b “ou=elk,dc=appdev,dc=local”


I am not sure what is user_id_attribute?

By default, users in search_user_base_DN should contain a uid LDAP attribute referring to a unique ID for the user within the base DN. An alternative attribute name can be specified via the optional user_id_attribute configuration item.

From the docs

Alright, make sense!

And it worked now, in the documentation it says, it’s optional to use the “user_id_attribute”, but looks like it is a mandatory field because i didn’t make any other changes and it started working after adding “user_id_attribute”.

Thank you so much @sscarduzio for all your help. I really appreciate it. I will play with a little more and if any other issue comes up, I will let you know. :slight_smile:

1 Like

Great news @sim, sorry this took a while actually. The problem with LDAP is that every LDAP server and its configuration is a snowflake of its own. So we can help, but to a certain degree.

Thanks to @ld57 too btw! :raised_hands:

@sscarduzio I totally understand, but really appreciate your patience with me here. Thank you so much.

Now, I will start working on the kibana end, need to setup the AD login on Kibana now. Going through the documentation again.

And yes thank you @ld57 :slight_smile:

1 Like

Hello @sscarduzio

If i want to join my elasticsearch with 2 different ADs, what needs to be done?

The keystore.jks which we create is using the root cert from AD but in case of 2 ADs how is it going to be? could you assist and share a sample configuration of readonlyrest.yml

Hi @sim,

Do you need to set up two LDAP for high availability, or they contain different users data?

@sscarduzio they contain different users data

And for monitoring plugin to work, do we have to get the readonlyrest pro/enterprise?