[GUIDE] ROR, Own Home and NGINX

#ROR, Own Home and NGINX

  • In this guide, I’m sharing how we were able to get ROR, Own Home and NGINX working together to provide multitenancy with rights management by ROR
  • This is how I got it working for me, no guarantees it’ll work exactly right for you! Hopefully this can serve as a rough map, though!
  • In this scenario, there are four users. User1 has access to both indices starting with “research” and “sales”. User2 has access only to “sales”, user3 only “research” and “administrator” is an administrator with access to everything.

Software versions

  • ROR 1.16.7 for ES 5.4.0
  • Own Home 5.4.0
  • Kibana 5.4.0
  • Elasticsearch 5.4.0
  • NGINX - 1.10.3

Instructions

  1. Install and configure your elasticsearch cluster.
  2. Install ROR plugin in Elasticsearch
  3. Install and configure your Kibana instance.
  4. Install own_home plugin in Kibana

Configure own_home plugin in kibana.yml as below

own_home.proxy_user_header: x-forwarded-user
own_home.session.secretkey: "mysecretkey"
own_home.session.isSecure: false
own_home.local.groups: [ sandbox ]
elasticsearch.url: "http://localhost:19200"
elasticsearch.username: "kibana"
elasticsearch.password: "kibana"
elasticsearch.requestHeadersWhitelist: [ authorization, x-forwarded-user ]

Configure elasticsearch.yml as below (also configure any other elasticsearch.yml options are needed, of course, this is ROR support parameters only.

     readonlyrest:
         enable: true
         response_if_req_forbidden: Forbidden by ReadonlyREST ES plugin
         access_control_rules:
           - name: "user1_kibana"
             proxy_auth: ["user1"]
             indices: [".kibana_user1"]
             actions: ["*"]

           - name: "user2_kibana"
             proxy_auth: ["user2"]
             indices: [".kibana_user2"]
             actions: ["*"]

           - name: "user3_kibana"
             proxy_auth: ["user3"]
             indices: [".kibana_user3"]
             actions: ["*"]

           - name: "proxy auth - user1"
             proxy_auth: ["user1"]
             actions: ["indices:data/read/*","indices:admin/mappings/*"]
             indices: ["sales_*","research_*"]
             kibana_access: rw

           - name: "proxy auth - user2"
             proxy_auth: ["user2"]
             actions: ["indices:data/read/*","indices:admin/mappings/*"]
             indices: ["sales_*"]
             kibana_access: rw

           - name: "proxy auth - user3 only"
             proxy_auth: ["user3"]
             actions: ["indices:data/read/*","indices:admin/mappings/*"]
             indices: ["research_*"]
             kibana_access: rw

           - name: "::KIBANA-SRV::"
             auth_key: kibana:kibana

           - name: "Administrator access"
             proxy_auth: ["administrator"]
             kibana_access: admin
             actions: ["*"]

Configure /etc/nginx/default.d/kibana.conf like the following:

location ~ (/app/kibana|/bundles|/kibana|/status|/plugins|/ui|/api|/es_admin|/elasticsearch|/app/own_home|/app/timelion) {
  proxy_pass  https://localhost:5601;
  proxy_redirect off;

  proxy_http_version 1.1;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection 'upgrade';
  proxy_set_header Host $host;
  proxy_cache_bypass $http_upgrade;

  rewrite ^/kibana(.*)$ /$1 break;

  proxy_set_header  X-Real-IP         $remote_addr; # pass on real client's IP
  proxy_set_header  X-Forwarded-For   $proxy_add_x_forwarded_for;
  proxy_set_header  X-Forwarded-Proto $scheme;
  auth_basic "Basic Auth";
  auth_basic_user_file /etc/nginx/conf.d/.htpasswd;
  proxy_set_header x-forwarded-user $remote_user;
  proxy_set_header Authorization "";
  access_log /var/log/nginx/kibana.access.log;
}

Configure /etc/nginx/conf.d/.htpasswd accounts that correspond to user1, user2, user3 and administrator.

  1. Start elasticsearch, kibana and nginx services
  2. Navigate to your nginx host (HTTP/HTTPS as you configure in /etc/nginx/nginx.conf)
  3. You’ll be prompted for credentials as defined in your .htpasswd file
  4. The validated username is passed on to Kibana through nginx on the x-forwarded-user header
  5. own_home plugin proxies an appropriate index and does its magic to point kibana towards that personalized index
  6. Kibana passes the x-forwarded-user header on to Elasticsearch, and proxy-auth, ROR uses this to grant appropriate access

Notes on improvements:

  • kibana_index: “.kibana_@{x-forwarded-user}” parameter doesn’t seem to work - this might allow us to avoid the need for the userX_kibana ACL blocks. Without write access to the individual own_home created .kibana_userX index, kibana fails. I also tried specifying explicit “.kibana_user1” values on the kibana_index option, to no avail.
  • However, if we wanted to provide user1, user2 and user3 write permissions to the indexes, we could probably collapse the .kibana_userX index into the block with the other permissions.
  • We have to specify “indices:admin/mappings/*” permissions, otherwise users are unable to create their own mappings in their own kibana indexes. I can see why these are split out, but reading the mappings and field properties seems generally innocent.
  • I can’t have the ROR Kibana plugin installed when I am running proxy_auth blocks. I successfully pass through the proxy auth header, and am redirected to the login screen. Some ability to get around this, as is discussed in Skip Login page for kibana would be useful.

#Thoughts
I’ve seen that Kibana multitenancy is on the roadmap for ROR - perhaps a partnership with own_home would be an efficient path forward?

2 Likes

Great job @hijakk! Thanks for sharing this.

Improvements

This is possible using a Kibana setting.
Quoting from https://www.elastic.co/guide/en/kibana/current/settings.html

elasticsearch.requestHeadersWhitelist:

Default: [ ‘authorization’ ] List of Kibana client-side headers to send to Elasticsearch. To send no client-side headers, set this value to (an empty list)


Can you share an example of request that got blocked if you didn’t have the actions rule?
Mappings to what indices? Their own Kibana indices? What action in particular?


Hmmm this needs the support in the Kibana plugin: when a request comes with no credentials, but has x-forwarded-user, try to see if ES likes it, if yes, cache the user and allow it thereafter.

I have the x-forwarded-user header passing through Kibana to ES to do the proxy auth using this setting, so in theory it should be made available to the variable in kibana_index, correct?
I also found that explicitly defining the kibana_index parameter (ie, .kibana_user1) in the main ACL block didn’t seem to have the effect I was hoping for. Is the functionality of this option supposed to explicitly set the kibana index as far as ROR is concerned, so that it can apply the kibana rights appropriately? The workaround was the extra block granting that proxy auth user explicit write access to their own kibana index.


I found that if I don’t have
indices:admin/mappings/*
On the block where I assign the proxy_auth user rights to the sales and/or research indices, then when I attempt to create an index pattern on an index the user has access to, it is unable to get a field listing. This means it can’t pick a time field, so it gets completely stuck. I can see if I can produce an exact error message when I’m back at work.
It got really interesting when I defined an index pattern when the user had access, removed access, and refreshed the pattern - everything other than _source, _id, _type and _index disappeared!

That said, I didn’t test to see if I could do it with just one of
indices:admin/mappings/fields/get
indices:admin/mappings/get

I’d guess if it succeeds with just one, it would be indices:admin/mappings/fields/get - I can test this out when I get back into work on Monday.


Proxy auth fallback logic in the kibana plugin would fit the bill nicely I think!

As of today, in the kibana_access macro-rule we allow indices:admin/mappings/fields/get* for RO and indices:admin/mappings/put for RW.

https://github.com/sscarduzio/elasticsearch-readonlyrest-plugin/blob/master/core/src/main/java/org/elasticsearch/plugin/readonlyrest/acl/blocks/rules/impl/KibanaAccessSyncRule.java#L40

Also, for RO we allow write ops for some exceptional paths:
https://github.com/sscarduzio/elasticsearch-readonlyrest-plugin/blob/master/core/src/main/java/org/elasticsearch/plugin/readonlyrest/acl/blocks/rules/impl/KibanaAccessSyncRule.java#L126

Would be nice to pinpoint these mappings things so the UX is satisfactory.

Is it that the RO/RW kibana mappings described in KibanaAccessSyncRule apply to the .kibana index specifically, and not indexes in general? I can’t quite tell if that’s the case in a very quick look through of the code.
If it is, the problem is that the “sales” or “research” index is the one that needs the access to the fields and/or mappings granted - not the kibana index.

Yes the logic I referred to in my previous post referred to the .kibana or any other index passed as kibana_index rule.

The rest of the indices are expected to be accessed by a kibana account in read only.
So even if you are kibana_access: rw or even kibana_access: admin you still will be allowed to operate any other index within the set of actions listed in the Java code under RO. That is:

     "indices:admin/exists",
      "indices:admin/mappings/fields/get*",
      "indices:admin/validate/query",
      "indices:data/read/field_stats",
      "indices:data/read/search",
      "indices:data/read/msearch",
      "indices:admin/get",
      "indices:admin/refresh*",
      "indices:data/read/*"

Why not to compress these to:

           - name: "users_kibana"
             proxy_auth: ["user1", user2", user3"]
             indices: [".kibana_@{user}"]
             actions: ["*"]

or even:

           - name: "users_kibana"
             proxy_auth: ["user1", user2", user3"]
             kibana_index: [".kibana_@{user}"]
             kibana_access: rw

I will give that a try, thanks @sscarduzio

This works and I will use it going forward, much easier to maintain one block than many!

This ends up making everything available because there aren’t any indices defined.

Regarding the “indices:admin/mappings/*” rule, in my testing I had to have both of the following defined, having just one enabled didn’t do it.
indices:admin/mappings/fields/get
indices:admin/mappings/get
I got these same set of errors when either single rule was missing:

[2017-07-17T15:22:23,429][INFO ][o.e.p.r.a.ACL            ] FORBIDDEN by default req={ ID:1330938648-1579963182#2187, TYP:GetFieldMappingsRequest, USR:user2, BRS:false, ACT:indices:admin/mappings/fields/get, OA:127.0.0.1, IDX:research_*, MET:GET, PTH:/research_*/_mapping/field/*?ignore_unavailable=false&allow_no_indices=false&include_defaults=true, CNT:<OMITTED, LENGTH=0>, HDR:connection,content-length,host,x-forwarded-for,x-forwarded-port,x-forwarded-proto,x-forwarded-user, HIS:[users_kibana->[indices->false, proxy_auth->true]] }
[2017-07-17T15:22:23,430][INFO ][o.e.p.r.a.ACL            ] FORBIDDEN by default req={ ID:1330938648-1579963182#2187, TYP:GetFieldMappingsRequest, USR:[no basic auth header], BRS:false, ACT:indices:admin/mappings/fields/get, OA:127.0.0.1, IDX:research_*, MET:GET, PTH:/research_*/_mapping/field/*?ignore_unavailable=false&allow_no_indices=false&include_defaults=true, CNT:<OMITTED, LENGTH=0>, HDR:connection,content-length,host,x-forwarded-for,x-forwarded-port,x-forwarded-proto,x-forwarded-user, HIS:[users_kibana->[indices->false, proxy_auth->true]], [proxy auth - user1 both->[proxy_auth->false]] }
[2017-07-17T15:22:23,430][INFO ][o.e.p.r.a.ACL            ] FORBIDDEN by default req={ ID:1330938648-1579963182#2187, TYP:GetFieldMappingsRequest, USR:user2, BRS:false, ACT:indices:admin/mappings/fields/get, OA:127.0.0.1, IDX:research_*, MET:GET, PTH:/research_*/_mapping/field/*?ignore_unavailable=false&allow_no_indices=false&include_defaults=true, CNT:<OMITTED, LENGTH=0>, HDR:connection,content-length,host,x-forwarded-for,x-forwarded-port,x-forwarded-proto,x-forwarded-user, HIS:[users_kibana->[indices->false, proxy_auth->true]], [proxy auth - user1->[proxy_auth->false]], [proxy auth - user2 only->[indices->true, proxy_auth->true, actions->false]] }
[2017-07-17T15:22:23,430][INFO ][o.e.p.r.a.ACL            ] FORBIDDEN by default req={ ID:1330938648-1579963182#2187, TYP:GetFieldMappingsRequest, USR:[no basic auth header], BRS:false, ACT:indices:admin/mappings/fields/get, OA:127.0.0.1, IDX:research_*, MET:GET, PTH:/research_*/_mapping/field/*?ignore_unavailable=false&allow_no_indices=false&include_defaults=true, CNT:<OMITTED, LENGTH=0>, HDR:connection,content-length,host,x-forwarded-for,x-forwarded-port,x-forwarded-proto,x-forwarded-user, HIS:[users_kibana->[indices->false, proxy_auth->true]], [proxy auth - user1->[proxy_auth->false]], [proxy auth - user2 only->[indices->true, proxy_auth->true, actions->false]], [proxy auth - user3 only->[proxy_auth->false]] }
[2017-07-17T15:22:23,430][INFO ][o.e.p.r.a.ACL            ] FORBIDDEN by default req={ ID:1330938648-1579963182#2187, TYP:GetFieldMappingsRequest, USR:[no basic auth header], BRS:false, ACT:indices:admin/mappings/fields/get, OA:127.0.0.1, IDX:research_*, MET:GET, PTH:/research_*/_mapping/field/*?ignore_unavailable=false&allow_no_indices=false&include_defaults=true, CNT:<OMITTED, LENGTH=0>, HDR:connection,content-length,host,x-forwarded-for,x-forwarded-port,x-forwarded-proto,x-forwarded-user, HIS:[users_kibana->[indices->false, proxy_auth->true]], [proxy auth - user1->[proxy_auth->false]], [proxy auth - user2 only->[indices->true, proxy_auth->true, actions->false]], [proxy auth - user3 only->[proxy_auth->false]], [::KIBANA-SRV::->[auth_key->false]] }
[2017-07-17T15:22:23,430][INFO ][o.e.p.r.a.ACL            ] FORBIDDEN by default req={ ID:1330938648-1579963182#2187, TYP:GetFieldMappingsRequest, USR:[no basic auth header], BRS:false, ACT:indices:admin/mappings/fields/get, OA:127.0.0.1, IDX:research_*, MET:GET, PTH:/research_*/_mapping/field/*?ignore_unavailable=false&allow_no_indices=false&include_defaults=true, CNT:<OMITTED, LENGTH=0>, HDR:connection,content-length,host,x-forwarded-for,x-forwarded-port,x-forwarded-proto,x-forwarded-user, HIS:[users_kibana->[indices->false, proxy_auth->true]], [proxy auth - user1>[proxy_auth->false]], [proxy auth - user2 only->[indices->true, proxy_auth->true, actions->false]], [proxy auth - user3 only->[proxy_auth->false]], [Administrator access->[proxy_auth->false]], [::KIBANA-SRV::->[auth_key->false]] }
[2017-07-17T15:22:23,471][INFO ][o.e.p.r.a.ACL            ] FORBIDDEN by default req={ ID:1397692488-922310578#2188, TYP:FieldStatsRequest, USR:user2, BRS:false, ACT:indices:data/read/field_stats, OA:127.0.0.1, IDX:research_*, MET:POST, PTH:/research_*/_field_stats?fields=*&allow_no_indices=false, CNT:<OMITTED, LENGTH=0>, HDR:connection,Content-Length,host,x-forwarded-for,x-forwarded-port,x-forwarded-proto,x-forwarded-user, HIS:[users_kibana->[indices->false, proxy_auth->true]] }
[2017-07-17T15:22:23,471][INFO ][o.e.p.r.a.ACL            ] FORBIDDEN by default req={ ID:1397692488-922310578#2188, TYP:FieldStatsRequest, USR:[no basic auth header], BRS:false, ACT:indices:data/read/field_stats, OA:127.0.0.1, IDX:research_*, MET:POST, PTH:/research_*/_field_stats?fields=*&allow_no_indices=false, CNT:<OMITTED, LENGTH=0>, HDR:connection,Content-Length,host,x-forwarded-for,x-forwarded-port,x-forwarded-proto,x-forwarded-user, HIS:[proxy auth - user1 both->[proxy_auth->false]], [users_kibana->[indices->false, proxy_auth->true]] }

This resulted in the behavior I described previously, where I was unable to add an index pattern for research_* and attempting to refresh an already established index pattern for research_* resulted in a failure where I could only see fields like _index, _source, _id, _type, etc.

I also tried rolling

             kibana_index: [".kibana_@{user}"]
             kibana_access: rw

back directly into the user level definition block again, ie

      - name: "proxy auth - user2"
        proxy_auth: ["user2"]
        actions: ["indices:data/read/*","indices:admin/mappings/*"]
        indices: ["research_*"]
        kibana_index: [".kibana_@{user}"]
        kibana_access: rw

And it had no kibana access at all - no pages would load in kibana.

Hi, I follow the instruction, but there are some problems.
elasticsearch log:

[2018-06-25T11:04:40,628][INFO ][t.b.r.a.ACL              ] FORBIDDEN by default req={ ID:1527080106-2015191641#46, TYP:MainRequest, CGR:N/A, USR:kibana(?), BRS:false, KDX:null, ACT:cluster:monitor/main, OA:127.0.0.1, DA:127.0.0.1, IDX:<N/A>, MET:HEAD, PTH:/, CNT:<N/A>, HDR:{authorization=Basic a2liYW5hOmtpYmFuYQ==, Authorization=<OMITTED>, content-length=0, x-forwarded-proto=http, host=localhost:19200, x-forwarded-port=45938, connection=keep-alive, x-forwarded-for=127.0.0.1}, HIS:[nginx_kibana->[proxy_auth->false]] }
[2018-06-25T11:04:40,636][INFO ][t.b.r.a.ACL              ] FORBIDDEN by default req={ ID:825025810-1414881638#47, TYP:NodesInfoRequest, CGR:N/A, USR:kibana(?), BRS:false, KDX:null, ACT:cluster:monitor/nodes/info, OA:127.0.0.1, DA:127.0.0.1, IDX:<N/A>, MET:GET, PTH:/_nodes?filter_path=nodes.*.version%2Cnodes.*.http.publish_address%2Cnodes.*.ip, CNT:<N/A>, HDR:{authorization=Basic a2liYW5hOmtpYmFuYQ==, Authorization=<OMITTED>, content-length=0, x-forwarded-proto=http, host=localhost:19200, x-forwarded-port=45948, connection=keep-alive, x-forwarded-for=127.0.0.1}, HIS:[nginx_kibana->[proxy_auth->false]] }
[2018-06-25T11:04:40,641][INFO ][t.b.r.a.ACL              ] FORBIDDEN by default req={ ID:1767789554-278767995#48, TYP:NodesInfoRequest, CGR:N/A, USR:kibana(?), BRS:false, KDX:null, ACT:cluster:monitor/nodes/info, OA:127.0.0.1, DA:127.0.0.1, IDX:<N/A>, MET:GET, PTH:/_nodes/_local?filter_path=nodes.*.settings.tribe, CNT:<N/A>, HDR:{authorization=Basic a2liYW5hOmtpYmFuYQ==, Authorization=<OMITTED>, content-length=0, x-forwarded-proto=http, host=localhost:19200, x-forwarded-port=45948, connection=keep-alive, x-forwarded-for=127.0.0.1}, HIS:[nginx_kibana->[proxy_auth->false]] }
[2018-06-25T11:04:40,686][INFO ][t.b.r.a.ACL              ] FORBIDDEN by default req={ ID:370530990-684177691#49, TYP:GetMappingsRequest, CGR:N/A, USR:kibana(?), BRS:false, KDX:null, ACT:indices:admin/mappings/get, OA:127.0.0.1, DA:127.0.0.1, IDX:.kibana, MET:GET, PTH:/.kibana/_mappings, CNT:<N/A>, HDR:{authorization=Basic a2liYW5hOmtpYmFuYQ==, Authorization=<OMITTED>, content-length=0, x-forwarded-proto=http, host=localhost:19200, x-forwarded-port=45948, connection=keep-alive, x-forwarded-for=127.0.0.1}, HIS:[nginx_kibana->[proxy_auth->false]] }

when I open the kibana on the browser, it shows me the error.

kibana log:

[root@localhost bin]# sh kibana
  log   [03:01:10.439] [info][status][plugin:kibana@6.2.2] Status changed from uninitialized to green - Ready
  log   [03:01:10.471] [info][status][plugin:elasticsearch@6.2.2] Status changed from uninitialized to yellow - Waiting for Elasticsearch
  log   [03:01:10.554] [info][status][plugin:timelion@6.2.2] Status changed from uninitialized to green - Ready
  log   [03:01:10.635] [info][status][plugin:own_home@6.2.2] Status changed from uninitialized to green - Ready
  log   [03:01:10.640] [info][status][plugin:console@6.2.2] Status changed from uninitialized to green - Ready
  log   [03:01:10.643] [info][status][plugin:metrics@6.2.2] Status changed from uninitialized to green - Ready
  log   [03:01:10.654] [info][plugin:own-home] Proxy server started at http://localhost:19200
  log   [03:01:10.655] [info][listening] Server running at http://localhost:5601
  log   [03:01:10.784] [error][status][plugin:elasticsearch@6.2.2] Status changed from yellow to red - Cannot convert undefined or null to object
error  [03:05:38.374]  TypeError: Cannot read property 'updated_at' of undefined
at /run/media/es3/80967063-20eb-4c15-82d1-56e01f888254/kibana-6.2.2-linux-x86_64/src/server/saved_objects/client/saved_objects_client.js:442:41
at next (native)
at step (/run/media/es3/80967063-20eb-4c15-82d1-56e01f888254/kibana-6.2.2-linux-x86_64/src/server/saved_objects/client/saved_objects_client.js:20:191)
at /run/media/es3/80967063-20eb-4c15-82d1-56e01f888254/kibana-6.2.2-linux-x86_64/src/server/saved_objects/client/saved_objects_client.js:20:361

thank you in advance!