We’re trying to use Kibana - ES without Shield or x-pack. Kibana is being fronted by an NGINX based SO authentication.
The question we have is how do we pass in the Groups returned by NGINX all the way to elastic search which has ROR plugin installed so we can control access to specific indexes? Also - will ROR Pro help in anyway to achieve this functionality?
Secondly, can we use the same UserGroups to control specific Apps and Widgets in Kibana?
Hi @Daedelus, you should use a combination of the proxy_auth rule and headers rule.
NB: I just created the
headersrule! it will be part of ROR 1.16.20-pre9. Tell me what ES version you use, and I will provide a build to you.
Example
readonlyrest:
access_control_rules:
- name: "::NGINX-READONLY-GROUP::"
indices: [".kibana_@{user}", "@{user}_logstash-*"]
headers: ["x-foobar-group:readonly"]
kibana_access: "ro"
proxy_auth:
proxy_auth_config: "proxy1"
users: ["*"]
- name: "::NGINX-READWRITE-GROUP::"
indices: [".kibana_@{user}", "@{user}_logstash-*"]
headers: [x-foobar-group:readwrite"]
kibana_access: "rw"
proxy_auth:
proxy_auth_config: "proxy1"
users: ["*"]
proxy_auth_configs:
- name: "proxy1"
user_id_header: "X-Forwarded-User"
Thank you. This could be exactly what I need.
I am on ES 6.2.4 the current latest release.
Very good, the pre release is being built in CI as we speak. Will link it to you as soon as it’s ready 
perfect i’ll check it out today. just to be clear - this is an ES level configuration with no requirement of ROR PRO/ENT on kibana correct?
yes this is part of the GPLv3 project (ROR Free)
Hey - I configured the rules exactly as indicated. Set my NGINX conf as below but I get the following error for readwrite group attempt ! Still comes out as FORBIDDEN when I attempt a POST. Any idea what I’m missing?
[2018-05-23T16:13:03,671][INFO ][t.b.r.a.ACL ] FORBIDDEN by default req={ ID:1821961603-1825235502#476, TYP:IndexRequest, CGR:N/A, USR:[no basic auth header], BRS:false, KDX:null, ACT:indices:data/write/index, OA:127.0.0.1, DA:0.0.0.0, IDX:converse, MET:POST, PTH:/converse/foo, CNT:<OMITTED, LENGTH=16>, HDR:{Connection=close, content-length=16, content-type=application/json, Host=localhost:9200, x-forwarded-for=127.0.0.1, x-forwarded-host=localhost:5100, x-forwarded-port=52312, x-forwarded-proto=http}, HIS:[::KIBANA-SRV 1::->[auth_key->false]], [::KIBANA-SRV 2::->[kibana_access->false]], [::ADMIN::->[auth_key->false]], [::RO::->[auth_key->false]], [::RW::->[auth_key->false]], [Global Write/Admin Access->[auth_key->false]], [Route53 Access->[actions->false, x_forwarded_for->true]], [::NGINX-READONLY-GROUP::->[proxy_auth->false]], [::NGINX-READWRITE-GROUP::->[proxy_auth->false]] }
NGINX CONF
http {
include mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
sendfile on;
keepalive_timeout 65;
server {
listen 5100;
server_name localhost;
location / {
rewrite ^/(.*) /$1 break;
proxy_ignore_client_abort on;
proxy_pass http://localhost:5601;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-User ABCD1234;
proxy_set_header X-FooBar-Group readwrite;
}
}
}
IT feels like althought NGINX is passing the headers to Kibana - Kibana is not able to forward it to Elastic.
Ok so it turns out that Kibana had to WHITELIST the headers coming in from NGINX. I am able to see the headers in ES. However, it looks like for any request - the ROR is returning true for Kibana access yet I’m seeing a white page (but the left navigation) is showing all the panels “Discovery, Dev Tools, Management etc”
FORBIDDEN by default req={ ID:1699004145–1587876091#1527, TYP:SearchRequest, CGR:N/A, USR:ABC851, BRS:false, KDX:null, ACT:indices:data/read/search, OA:127.0.0.1, DA:127.0.0.1, IDX:.kibana, MET:POST, PTH:/.kibana/_search?size=10000&from=0, CNT:<OMITTED, LENGTH=80>, HDR:{Connection=keep-alive, Content-Length=80, content-type=application/json, Host=localhost:9200, x-foobar-group=readwrite, x-forwarded-user=ABC851}, HIS:[::KIBANA-SRV 1::->[auth_key->false]], [::ADMIN::->[auth_key->false]], [::RO::->[auth_key->false]], [::RW::->[auth_key->false]], [Global Write/Admin Access->[auth_key->false]], [Route53 Access->[x_forwarded_for->false]], [::NGINX-READONLY-GROUP::->[headers->false, kibana_access->true, proxy_auth->true]], [::NGINX-READWRITE-GROUP::->[kibana_access->true, headers->true, indices->false, proxy_auth->true]]
OK this is possible. Try using this kibana setting “elasticsearch.requestHeadersWhitelist”
Looks like we photofinished on the reply 
That’s exactly what I had used. I’ve managed to pass the headers to ES but Kibana is showing a white page ( but the left navigation page is loading - just clicking on them does naught).
This is my Kibana YML
server.host: localhost
elasticsearch.url: http://localhost:9200
elasticsearch.requestHeadersWhitelist: [ authorization, X-Forwarded-User, X-FooBar-Group ]
elasticsearch.customHeaders: {}
logging.verbose: true
I removed these entries (assuming its the SSO User and Group that would either show or remove kibana apps). Things became worse - I now get an “Authorization Exception” on the main Kibana Page.
elasticsearch.username: “kibana”
elasticsearch.password: “kibana”
FORBIDDEN by default req={ ID:1010666346-1949853709#67, TYP:GetRequest, CGR:N/A, USR:ABC851, BRS:false, KDX:null, ACT:indices:data/read/get, OA:127.0.0.1, DA:127.0.0.1, IDX:.kibana, MET:GET, PTH:/.kibana/doc/config%3A6.2.4, CNT:<N/A>, HDR:{Connection=keep-alive, Content-Length=0, Host=localhost:9200, x-foobar-group=readwrite, x-forwarded-user=ABC851}, HIS:[::RO::->[auth_key->false]], [::RW::->[auth_key->false]], [Global Write/Admin Access->[auth_key->false]], [Route53 Access->[x_forwarded_for->false]], [::NGINX-READONLY-GROUP::->[headers->false, kibana_access->true, proxy_auth->true]], [::NGINX-READWRITE-GROUP::->[kibana_access->true, headers->true, indices->false, proxy_auth->true]] }
Do you have “FORBIDDEN” in the ES logs?
That’s correct. I’ve shared my Kibana YML Conf (notice, I’ve removed the elastisearch.username and elasticsearch.password from it as well - I assume for SSO to work, removing these is necessary?). Pasted the new ElasticLogs with the Forbidden status as well.
This is my Kibana YML
server.host: localhost
elasticsearch.url: http://localhost:9200
elasticsearch.requestHeadersWhitelist: [ authorization, X-Forwarded-User, X-FooBar-Group ]
elasticsearch.customHeaders: {}
logging.verbose: true
I removed these entries (assuming its the SSO User and Group that would either show or remove kibana apps). Things became worse - I now get an “Authorization Exception” on the main Kibana Page.
elasticsearch.username: “kibana”
elasticsearch.password: “kibana”
FORBIDDEN by default req={ ID:1010666346-1949853709#67, TYP:GetRequest, CGR:N/A, USR:ABC851, BRS:false, KDX:null, ACT:indices:data/read/get, OA:127.0.0.1, DA:127.0.0.1, IDX:.kibana, MET:GET, PTH:/.kibana/doc/config%3A6.2.4, CNT:<N/A>, HDR:{Connection=keep-alive, Content-Length=0, Host=localhost:9200, x-foobar-group=readwrite, x-forwarded-user=ABC851}, HIS:[::RO::->[auth_key->false]], [::RW::->[auth_key->false]], [Global Write/Admin Access->[auth_key->false]], [Route53 Access->[x_forwarded_for->false]], [::NGINX-READONLY-GROUP::->[headers->false, kibana_access->true, proxy_auth->true]], [::NGINX-READWRITE-GROUP::->[kibana_access->true, headers->true, indices->false, proxy_auth->true]] }
you still need these, and the corresponding ACL block. This is used by the Kibana server for communication with Elasticsearch, independently of any active Kibana sessions.
Can you share your latest readonlyrest.yml ?
Sure - thanks this worked. and I was able to use the proxy auth to login successfully
Only one item is left. I’m using the group RESTRICTED( NGINX-RESTRICTED-GROUP see below) to restrict access to certain apps - however, the apps are still returning as visible! ES Logs seem ok though. Solving this should get me over the finish line. Do I need ROR Kibana (PRO/ENT) for hiding apps?
readonlyrest:
prompt_for_basic_auth: falseaccess_control_rules: - name: "::KIBANA-SRV 1::" kibana_access: admin auth_key: kibana:kibana verbosity: error - name: "Route53 Access" type: allow x_forwarded_for: ["0.0.0.0/0"] actions: ["cluster:monitor/*", "indices:admin/get", "indices:admin/aliases", "indices:admin/aliases/*", "indices:admin/analyze", "indices:monitor/*", "indices:data/read/*"] verbosity: error - name: "::NGINX-RESTRICTED-GROUP::" indices: [".kibana", ".kibana_@{user}", "@{user}_logstash-*"] headers: ["x-foobar-group:restricted"] kibana_access: "ro" kibana_hide_apps: ["readonlyrest_kbn", "timelion", "kibana:dev_tools", "kibana:management"] proxy_auth: proxy_auth_config: "proxy1" users: ["*"] verbosity: info - name: "::NGINX-READONLY-GROUP::" indices: [".kibana", ".kibana_@{user}", "@{user}_logstash-*"] headers: ["x-foobar-group:readonly"] kibana_access: "ro" proxy_auth: proxy_auth_config: "proxy1" users: ["*"] verbosity: info - name: "::NGINX-READWRITE-GROUP::" indices: [".kibana", ".kibana_@{user}", "@{user}_logstash-*"] headers: ["x-foobar-group:readwrite"] kibana_access: admin proxy_auth: proxy_auth_config: "proxy1" users: ["*"] verbosity: info proxy_auth_configs: - name: "proxy1" user_id_header: "x-forwarded-user"
[2018-05-24T11:43:44,710][INFO ][t.b.r.a.ACL ] ALLOWED by { name: '::NGINX-RESTRICTED-GROUP::', policy: ALLOW} req={ ID:1274693229-1142794160#291, TYP:SearchRequest, CGR:N/A, USR:ABC851, BRS:false, KDX:null, ACT:indices:data/read/search, OA:127.0.0.1, DA:127.0.0.1, IDX:.kibana, MET:POST, PTH:/.kibana/_search?size=10000&from=0, CNT:<OMITTED, LENGTH=80>, HDR:{Connection=keep-alive, Content-Length=80, content-type=application/json, Host=localhost:9200, x-foobar-group=restricted, x-forwarded-user=ABC851}, HIS:[::KIBANA-SRV 1::->[auth_key->false]], [::RO::->[auth_key->false]], [::RW::->[auth_key->false]], [Global Write/Admin Access->[auth_key->false]], [Route53 Access->[x_forwarded_for->false]], [::NGINX-RESTRICTED-GROUP::->[kibana_access->true, headers->true, indices->true, proxy_auth->true, kibana_hide_apps->true]] }
Yes, that is correct. Hiding apps is a bunch of code on the Kibana side, you will need the Kibana plugin.
Perfect. Thanks so much @sscarduzio . I believe there is a colleague who is evaluating a 14 day trial version already in our enterprise. I had a follow up question on by passing login screen will create another thread for it.
No problem @Daedelus!
You might be interested in the proxy_auth support and custom logout link features that we provide as part of the PRO and Enterprise Kibana plugin.