NGINX (SSO) to Kibana to ElasticSearch?


(Roger Seth) #1

We’re trying to use Kibana - ES without Shield or x-pack. Kibana is being fronted by an NGINX based SO authentication.

The question we have is how do we pass in the Groups returned by NGINX all the way to elastic search which has ROR plugin installed so we can control access to specific indexes? Also - will ROR Pro help in anyway to achieve this functionality?

Secondly, can we use the same UserGroups to control specific Apps and Widgets in Kibana?


(Simone Scarduzio) #2

Hi @Daedelus, you should use a combination of the proxy_auth rule and headers rule.

NB: I just created the headers rule! it will be part of ROR 1.16.20-pre9. Tell me what ES version you use, and I will provide a build to you.

  1. Nginx authenticates
  2. Nginx injects x-forwarded-user header
  3. Nginx injects x-foobar-group header
  4. ES/ROR configured with proxy_auth will trust Nginx header as a claim of user identity
  5. ES/ROR configured with headers rule with the specifi group name.

Example

readonlyrest:    

    access_control_rules:

    - name: "::NGINX-READONLY-GROUP::"
      indices: ["[email protected]{user}", "@{user}_logstash-*"]
      headers: ["x-foobar-group:readonly"]
      kibana_access: "ro"
      proxy_auth:
        proxy_auth_config: "proxy1"
        users: ["*"]

    - name: "::NGINX-READWRITE-GROUP::"
      indices: ["[email protected]{user}", "@{user}_logstash-*"]
      headers: [x-foobar-group:readwrite"]
      kibana_access: "rw"
      proxy_auth:
        proxy_auth_config: "proxy1"
        users: ["*"]

    proxy_auth_configs:
    - name: "proxy1"
      user_id_header: "X-Forwarded-User"
    


(Roger Seth) #3

Thank you. This could be exactly what I need.

I am on ES 6.2.4 the current latest release.


(Simone Scarduzio) #4

Very good, the pre release is being built in CI as we speak. Will link it to you as soon as it’s ready :raised_hands:


(Simone Scarduzio) #5

Here you go: https://readonlyrest-data.s3-eu-west-1.amazonaws.com/build/1.16.20_pre9/readonlyrest-1.16.20_pre9_es6.2.4.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJEKIPNTOTIVGQ4EQ/20180523/eu-west-1/s3/aws4_request&X-Amz-Date=20180523T111359Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=92ed4772334e62346ba70f4d239ce8ccb9ec792c61b03fa8194d4ef676684350


(Roger Seth) #6

perfect i’ll check it out today. just to be clear - this is an ES level configuration with no requirement of ROR PRO/ENT on kibana correct?


(Simone Scarduzio) #7

yes this is part of the GPLv3 project (ROR Free)


(Roger Seth) #8

Hey - I configured the rules exactly as indicated. Set my NGINX conf as below but I get the following error for readwrite group attempt ! Still comes out as FORBIDDEN when I attempt a POST. Any idea what I’m missing?

[2018-05-23T16:13:03,671][INFO ][t.b.r.a.ACL ] FORBIDDEN by default req={ ID:1821961603-1825235502#476, TYP:IndexRequest, CGR:N/A, USR:[no basic auth header], BRS:false, KDX:null, ACT:indices:data/write/index, OA:127.0.0.1, DA:0.0.0.0, IDX:converse, MET:POST, PTH:/converse/foo, CNT:<OMITTED, LENGTH=16>, HDR:{Connection=close, content-length=16, content-type=application/json, Host=localhost:9200, x-forwarded-for=127.0.0.1, x-forwarded-host=localhost:5100, x-forwarded-port=52312, x-forwarded-proto=http}, HIS:[::KIBANA-SRV 1::->[auth_key->false]], [::KIBANA-SRV 2::->[kibana_access->false]], [::ADMIN::->[auth_key->false]], [::RO::->[auth_key->false]], [::RW::->[auth_key->false]], [Global Write/Admin Access->[auth_key->false]], [Route53 Access->[actions->false, x_forwarded_for->true]], [::NGINX-READONLY-GROUP::->[proxy_auth->false]], [::NGINX-READWRITE-GROUP::->[proxy_auth->false]] }

NGINX CONF

http {
    include       mime.types;
    default_type  application/octet-stream;
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    sendfile        on;
    keepalive_timeout  65;

	server {
		listen 5100;
		server_name localhost;

		location / {
			rewrite ^/(.*) /$1 break;
			proxy_ignore_client_abort on;
			proxy_pass http://localhost:5601;
			proxy_set_header  X-Real-IP  $remote_addr;
			proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
			proxy_set_header  Host $http_host;
			proxy_set_header X-Forwarded-User ABCD1234;
			proxy_set_header X-FooBar-Group readwrite;
		}
	}
}

(Simone Scarduzio) #9

Hi @Daedelus, as you can see, there is no x-forwarded-user header coming to ES. Not sure why tho.


(Roger Seth) #10

IT feels like althought NGINX is passing the headers to Kibana - Kibana is not able to forward it to Elastic.


(Roger Seth) #11

Ok so it turns out that Kibana had to WHITELIST the headers coming in from NGINX. I am able to see the headers in ES. However, it looks like for any request - the ROR is returning true for Kibana access yet I’m seeing a white page (but the left navigation) is showing all the panels “Discovery, Dev Tools, Management etc”

FORBIDDEN by default req={ ID:1699004145–1587876091#1527, TYP:SearchRequest, CGR:N/A, USR:ABC851, BRS:false, KDX:null, ACT:indices:data/read/search, OA:127.0.0.1, DA:127.0.0.1, IDX:.kibana, MET:POST, PTH:/.kibana/_search?size=10000&from=0, CNT:<OMITTED, LENGTH=80>, HDR:{Connection=keep-alive, Content-Length=80, content-type=application/json, Host=localhost:9200, x-foobar-group=readwrite, x-forwarded-user=ABC851}, HIS:[::KIBANA-SRV 1::->[auth_key->false]], [::ADMIN::->[auth_key->false]], [::RO::->[auth_key->false]], [::RW::->[auth_key->false]], [Global Write/Admin Access->[auth_key->false]], [Route53 Access->[x_forwarded_for->false]], [::NGINX-READONLY-GROUP::->[headers->false, kibana_access->true, proxy_auth->true]], [::NGINX-READWRITE-GROUP::->[kibana_access->true, headers->true, indices->false, proxy_auth->true]]


(Simone Scarduzio) #12

OK this is possible. Try using this kibana setting “elasticsearch.requestHeadersWhitelist”


(Roger Seth) #13

Looks like we photofinished on the reply :slight_smile: That’s exactly what I had used. I’ve managed to pass the headers to ES but Kibana is showing a white page ( but the left navigation page is loading - just clicking on them does naught).

This is my Kibana YML

server.host: localhost
elasticsearch.url: http://localhost:9200
elasticsearch.requestHeadersWhitelist: [ authorization, X-Forwarded-User, X-FooBar-Group ]
elasticsearch.customHeaders: {}
logging.verbose: true

I removed these entries (assuming its the SSO User and Group that would either show or remove kibana apps). Things became worse - I now get an “Authorization Exception” on the main Kibana Page.

elasticsearch.username: "kibana"
elasticsearch.password: “kibana”

FORBIDDEN by default req={ ID:1010666346-1949853709#67, TYP:GetRequest, CGR:N/A, USR:ABC851, BRS:false, KDX:null, ACT:indices:data/read/get, OA:127.0.0.1, DA:127.0.0.1, IDX:.kibana, MET:GET, PTH:/.kibana/doc/config%3A6.2.4, CNT:<N/A>, HDR:{Connection=keep-alive, Content-Length=0, Host=localhost:9200, x-foobar-group=readwrite, x-forwarded-user=ABC851}, HIS:[::RO::->[auth_key->false]], [::RW::->[auth_key->false]], [Global Write/Admin Access->[auth_key->false]], [Route53 Access->[x_forwarded_for->false]], [::NGINX-READONLY-GROUP::->[headers->false, kibana_access->true, proxy_auth->true]], [::NGINX-READWRITE-GROUP::->[kibana_access->true, headers->true, indices->false, proxy_auth->true]] }


(Simone Scarduzio) #14

Do you have “FORBIDDEN” in the ES logs?


(Roger Seth) #15

That’s correct. I’ve shared my Kibana YML Conf (notice, I’ve removed the elastisearch.username and elasticsearch.password from it as well - I assume for SSO to work, removing these is necessary?). Pasted the new ElasticLogs with the Forbidden status as well.

This is my Kibana YML

server.host: localhost
elasticsearch.url: http://localhost:9200
elasticsearch.requestHeadersWhitelist: [ authorization, X-Forwarded-User, X-FooBar-Group ]
elasticsearch.customHeaders: {}
logging.verbose: true

I removed these entries (assuming its the SSO User and Group that would either show or remove kibana apps). Things became worse - I now get an “Authorization Exception” on the main Kibana Page.

elasticsearch.username: "kibana"
elasticsearch.password: “kibana”

FORBIDDEN by default req={ ID:1010666346-1949853709#67, TYP:GetRequest, CGR:N/A, USR:ABC851, BRS:false, KDX:null, ACT:indices:data/read/get, OA:127.0.0.1, DA:127.0.0.1, IDX:.kibana, MET:GET, PTH:/.kibana/doc/config%3A6.2.4, CNT:<N/A>, HDR:{Connection=keep-alive, Content-Length=0, Host=localhost:9200, x-foobar-group=readwrite, x-forwarded-user=ABC851}, HIS:[::RO::->[auth_key->false]], [::RW::->[auth_key->false]], [Global Write/Admin Access->[auth_key->false]], [Route53 Access->[x_forwarded_for->false]], [::NGINX-READONLY-GROUP::->[headers->false, kibana_access->true, proxy_auth->true]], [::NGINX-READWRITE-GROUP::->[kibana_access->true, headers->true, indices->false, proxy_auth->true]] }


(Simone Scarduzio) #16

you still need these, and the corresponding ACL block. This is used by the Kibana server for communication with Elasticsearch, independently of any active Kibana sessions.

Can you share your latest readonlyrest.yml ?


(Roger Seth) #17

Sure - thanks this worked. and I was able to use the proxy auth to login successfully :slight_smile:

Only one item is left. I’m using the group RESTRICTED( NGINX-RESTRICTED-GROUP see below) to restrict access to certain apps - however, the apps are still returning as visible! ES Logs seem ok though. Solving this should get me over the finish line. Do I need ROR Kibana (PRO/ENT) for hiding apps?

readonlyrest:
prompt_for_basic_auth: false

access_control_rules:
- name: "::KIBANA-SRV 1::"
  kibana_access: admin
  auth_key: kibana:kibana
  verbosity: error
- name: "Route53 Access"
  type: allow
  x_forwarded_for: ["0.0.0.0/0"]
  actions: ["cluster:monitor/*", "indices:admin/get", "indices:admin/aliases", "indices:admin/aliases/*", "indices:admin/analyze", "indices:monitor/*", "indices:data/read/*"]
  verbosity: error

- name: "::NGINX-RESTRICTED-GROUP::"
  indices: [".kibana", "[email protected]{user}", "@{user}_logstash-*"]
  headers: ["x-foobar-group:restricted"]
  kibana_access: "ro"
  kibana_hide_apps: ["readonlyrest_kbn", "timelion", "kibana:dev_tools", "kibana:management"]
  proxy_auth:
    proxy_auth_config: "proxy1"
    users: ["*"]
  verbosity: info

- name: "::NGINX-READONLY-GROUP::"
  indices: [".kibana", "[email protected]{user}", "@{user}_logstash-*"]
  headers: ["x-foobar-group:readonly"]
  kibana_access: "ro"
  proxy_auth:
    proxy_auth_config: "proxy1"
    users: ["*"]
  verbosity: info

- name: "::NGINX-READWRITE-GROUP::"
  indices: [".kibana", "[email protected]{user}", "@{user}_logstash-*"]
  headers: ["x-foobar-group:readwrite"]
  kibana_access: admin
  proxy_auth:
    proxy_auth_config: "proxy1"
    users: ["*"]
  verbosity: info

proxy_auth_configs:
- name: "proxy1"
  user_id_header: "x-forwarded-user"

[2018-05-24T11:43:44,710][INFO ][t.b.r.a.ACL ] ALLOWED by { name: '::NGINX-RESTRICTED-GROUP::', policy: ALLOW} req={ ID:1274693229-1142794160#291, TYP:SearchRequest, CGR:N/A, USR:ABC851, BRS:false, KDX:null, ACT:indices:data/read/search, OA:127.0.0.1, DA:127.0.0.1, IDX:.kibana, MET:POST, PTH:/.kibana/_search?size=10000&from=0, CNT:<OMITTED, LENGTH=80>, HDR:{Connection=keep-alive, Content-Length=80, content-type=application/json, Host=localhost:9200, x-foobar-group=restricted, x-forwarded-user=ABC851}, HIS:[::KIBANA-SRV 1::->[auth_key->false]], [::RO::->[auth_key->false]], [::RW::->[auth_key->false]], [Global Write/Admin Access->[auth_key->false]], [Route53 Access->[x_forwarded_for->false]], [::NGINX-RESTRICTED-GROUP::->[kibana_access->true, headers->true, indices->true, proxy_auth->true, kibana_hide_apps->true]] }


(Simone Scarduzio) #18

Yes, that is correct. Hiding apps is a bunch of code on the Kibana side, you will need the Kibana plugin.


(Roger Seth) #19

Perfect. Thanks so much @sscarduzio . I believe there is a colleague who is evaluating a 14 day trial version already in our enterprise. I had a follow up question on by passing login screen will create another thread for it.


(Simone Scarduzio) #20

No problem @Daedelus!
You might be interested in the proxy_auth support and custom logout link features that we provide as part of the PRO and Enterprise Kibana plugin.