Mult-tenancy issue with ROR Kibana

Hi,

I’m trying to get ROR working in the context of multi-tenancy in order to grant access to two groups of users to different ES indices and therefore I’d like to have a bunch of index patterns for one group and another one for the other group.

The issue I experiment is the following, whenever I’d like to set a default index pattern for a specific group (or a specific .kibana- to be precise), it is updated in the relevant template and as soon as I log out and in again, the template is updated with the value of the default index-pattern found in the “normal” .kibana index, as this index pattern doesn’t exist in the current context it is directly overwritten with “null” value making the default disappear.

Any help will be greatly appreciated (I can provide logs if needed as well as the readonlyrest.yml configuration file)

Kibana/ES version used: 6.5.3
ROR plugin version: 1.17.2

Thank you in advance.

Hi Nicko,

Thanks for resubmitting this issue. Reading your description I get confused when you talk about “template”. Are you using tenancy templating? If not, can you expand on what you mean? Ideally make a bullet point of a sequence of actions we can do in order to reproduce the issue.

Mmmmh, wasn’t speaking about the “tenancy templating” as you mentioned (wasn’t aware about that, maybe I should give it a try), but more about kibana_index_template which are mentioned as a result of curl localhost:9200/_cat/templates

1 Like

If it can help, here is my readonlyrest.yml config:

readonlyrest:
  audit_collector: true

  access_control_rules:

  - name: "::AD ADMIN::"
    ldap_auth:
       name: "activedirectory"
       groups: ["admins"]
    type: allow
    verbosity: error

  - name: "::AD ONE GROUP::"
    ldap_auth:
       name: "activedirectory"
       groups: ["onegroup"]
    kibana_access: ro
    indices: [".onegroup", <bunch of other indices>]
    kibana_index: ".onegroup"
    verbosity: error

  - name: "::AD OTHER GROUP::"
    ldap_auth:
       name: "activedirectory"
       groups: ["othergroup"]
    kibana_access: ro
    indices: [".othergroup", <bunch of other indices>]
    kibana_index: ".othergroup"
    verbosity: error

  - name: "::ONE GROUP ADMIN::"
    auth_key: onegroup:onegroup
    kibana_access: rw
    indices: [".onegroup", <bunch of other indices>]
    kibana_index: ".onegroup"
    verbosity: error

  - name: "::OTHER GROUP ADMIN::"
    auth_key: othergroup:othergroup
    kibana_access: rw
    indices: [".othergroup", <bunch of other indices>]
    kibana_index: ".othergroup"
    verbosity: error

  - name: "::KIBANA-SRV::"
    auth_key: kibana:xxxxxx
    type: allow

And the relevant logs once the default index pattern is modified:

[2019-03-25T15:44:28,715][INFO ][t.b.r.e.IndexLevelActionFilter] [my-es-host] Settings observer refreshing...
[2019-03-25T15:44:28,719][INFO ][t.b.r.r.SerializationTool] [my-es-host] no .onegrouptom audit log serialisers found, proceeding with default.
[2019-03-25T15:44:28,751][INFO ][t.b.r.a.ACL              ] [my-es-host] ADDING BLOCK:   { name: '::AD ADMIN::', policy: ALLOW, rules: [ldap_auth]}
[2019-03-25T15:44:28,751][INFO ][t.b.r.a.ACL              ] [my-es-host] ADDING BLOCK:   { name: '::AD ONE GROUP::', policy: ALLOW, rules: [ldap_auth, kibana_access, indices, kibana_index]}
[2019-03-25T15:44:28,751][INFO ][t.b.r.a.ACL              ] [my-es-host] ADDING BLOCK:   { name: '::AD OTHER GROUP::', policy: ALLOW, rules: [ldap_auth, kibana_access, indices, kibana_index]}
[2019-03-25T15:44:28,751][INFO ][t.b.r.a.ACL              ] [my-es-host] ADDING BLOCK:   { name: '::ONE GROUP ADMIN::', policy: ALLOW, rules: [auth_key, kibana_access, indices, kibana_index]}
[2019-03-25T15:44:28,751][INFO ][t.b.r.a.ACL              ] [my-es-host] ADDING BLOCK:   { name: '::OTHER GROUP ADMIN::', policy: ALLOW, rules: [auth_key, kibana_access, indices, kibana_index]}
[2019-03-25T15:44:28,752][INFO ][t.b.r.a.ACL              ] [my-es-host] ADDING BLOCK:   { name: '::KIBANA-SRV::', policy: ALLOW, rules: [auth_key]}
[2019-03-25T15:44:28,752][INFO ][t.b.r.e.IndexLevelActionFilter] [my-es-host] Configuration reloaded - ReadonlyREST enabled
[2019-03-25T15:44:28,773][INFO ][t.b.r.e.SettingsObservableImpl] [my-es-host] all ok, written settings
[2019-03-25T15:44:39,959][INFO ][t.b.r.a.ACL              ] [my-es-host] ESC[36mALLOWED by { name: '::KIBANA-SRV::', policy: ALLOW, rules: [auth_key]} req={ ID:1962194496-1122009013#24162273, TYP:CreateIndexRequest, CGR:N/A, USR:kibana, BRS:false, KDX:null, ACT:indices:admin/create, OA:<kibana ip>, DA:<es ip>, IDX:.onegroup, MET:PUT, PTH:/.onegroup, CNT:<OMITTED, LENGTH=3024>, HDR:{Authorization=<OMITTED>, Connection=keep-alive, content-type=application/json, Host=my-es-host.example.com:9200, Content-Length=3024}, HIS:[::AD ADMIN::->[ldap_authentication->false]], [::AD ONE GROUP::->[ldap_authentication->false]], [::AD OTHER GROUP::->[ldap_authentication->false]], [::OTHER GROUP ADMIN::->[auth_key->false]], [::ONE GROUP ADMIN::->[auth_key->false]], [::KIBANA-SRV::->[auth_key->true]] } ESC[0m
[2019-03-25T15:44:39,982][INFO ][t.b.r.a.ACL              ] [my-es-host] ESC[36mALLOWED by { name: '::KIBANA-SRV::', policy: ALLOW, rules: [auth_key]} req={ ID:2131611748-1949012630#24162275, TYP:ReindexRequest, CGR:N/A, USR:kibana, BRS:false, KDX:null, ACT:indices:data/write/reindex, OA:<kibana ip>, DA:<es ip>, IDX:, MET:POST, PTH:/_reindex?refresh=true&wait_for_active_shards=all, CNT:<OMITTED, LENGTH=138>, HDR:{Authorization=<OMITTED>, Connection=keep-alive, content-type=application/json, Host=my-es-host.example.com:9200, Content-Length=138}, HIS:[::AD ADMIN::->[ldap_authentication->false]], [::AD ONE GROUP::->[ldap_authentication->false]], [::AD OTHER GROUP::->[ldap_authentication->false]], [::OTHER GROUP ADMIN::->[auth_key->false]], [::ONE GROUP ADMIN::->[auth_key->false]], [::KIBANA-SRV::->[auth_key->true]] } ESC[0m
[2019-03-25T15:44:41,782][INFO ][t.b.r.a.ACL              ] [my-es-host] ESC[36mALLOWED by { name: '::KIBANA-SRV::', policy: ALLOW, rules: [auth_key]} req={ ID:1306909829-1591218567#24162328, TYP:PutIndexTemplateRequest, CGR:N/A, USR:kibana, BRS:false, KDX:null, ACT:indices:admin/template/put, OA:<kibana ip>, DA:<es ip>, IDX:.onegroup, MET:PUT, PTH:/_template/kibana_index_template%3A.onegroup, CNT:<OMITTED, LENGTH=3024>, HDR:{Authorization=<OMITTED>, Connection=keep-alive, content-type=application/json, Host=my-es-host.example.com:9200, Content-Length=3024}, HIS:[::AD ADMIN::->[ldap_authentication->false]], [::AD ONE GROUP::->[ldap_authentication->false]], [::AD OTHER GROUP::->[ldap_authentication->false]], [::OTHER GROUP ADMIN::->[auth_key->false]], [::ONE GROUP ADMIN::->[auth_key->false]], [::KIBANA-SRV::->[auth_key->true]] } ESC[0m
[2019-03-25T15:44:41,790][INFO ][o.e.c.m.MetaDataIndexTemplateService] [my-es-host] adding template [kibana_index_template:.onegroup] for index patterns [.onegroup]
[2019-03-25T15:44:51,950][INFO ][t.b.r.a.ACL              ] [my-es-host] ESC[36mALLOWED by { name: '::KIBANA-SRV::', policy: ALLOW, rules: [auth_key]} req={ ID:104823682-1993644643#24162836, TYP:PutIndexTemplateRequest, CGR:N/A, USR:kibana, BRS:false, KDX:null, ACT:indices:admin/template/put, OA:<kibana ip>, DA:<es ip>, IDX:.onegroup, MET:PUT, PTH:/_template/kibana_index_template%3A.onegroup, CNT:<OMITTED, LENGTH=3024>, HDR:{Authorization=<OMITTED>, Connection=keep-alive, content-type=application/json, Host=my-es-host.example.com:9200, Content-Length=3024}, HIS:[::AD ADMIN::->[ldap_authentication->false]], [::AD ONE GROUP::->[ldap_authentication->false]], [::AD OTHER GROUP::->[ldap_authentication->false]], [::OTHER GROUP ADMIN::->[auth_key->false]], [::ONE GROUP ADMIN::->[auth_key->false]], [::KIBANA-SRV::->[auth_key->true]] } ESC[0m
[2019-03-25T15:44:51,957][INFO ][o.e.c.m.MetaDataIndexTemplateService] [my-es-host] adding template [kibana_index_template:.onegroup] for index patterns [.onegroup]
[2019-03-25T15:45:03,082][INFO ][t.b.r.a.ACL              ] [my-es-host] ESC[36mALLOWED by { name: '::KIBANA-SRV::', policy: ALLOW, rules: [auth_key]} req={ ID:797550172-1338405730#24163385, TYP:CreateIndexRequest, CGR:N/A, USR:kibana, BRS:false, KDX:null, ACT:indices:admin/create, OA:<kibana ip>, DA:<es ip>, IDX:.onegroup, MET:PUT, PTH:/.onegroup, CNT:<OMITTED, LENGTH=3024>, HDR:{Authorization=<OMITTED>, Connection=keep-alive, content-type=application/json, Host=my-es-host.example.com:9200, Content-Length=3024}, HIS:[::AD ADMIN::->[ldap_authentication->false]], [::AD ONE GROUP::->[ldap_authentication->false]], [::AD OTHER GROUP::->[ldap_authentication->false]], [::OTHER GROUP ADMIN::->[auth_key->false]], [::ONE GROUP ADMIN::->[auth_key->false]], [::KIBANA-SRV::->[auth_key->true]] } ESC[0m
[2019-03-25T15:45:03,104][INFO ][t.b.r.a.ACL              ] [my-es-host] ESC[36mALLOWED by { name: '::KIBANA-SRV::', policy: ALLOW, rules: [auth_key]} req={ ID:877396540-751578736#24163387, TYP:ReindexRequest, CGR:N/A, USR:kibana, BRS:false, KDX:null, ACT:indices:data/write/reindex, OA:<kibana ip>, DA:<es ip>, IDX:, MET:POST, PTH:/_reindex?refresh=true&wait_for_active_shards=all, CNT:<OMITTED, LENGTH=138>, HDR:{Authorization=<OMITTED>, Connection=keep-alive, content-type=application/json, Host=my-es-host.example.com:9200, Content-Length=138}, HIS:[::AD ADMIN::->[ldap_authentication->false]], [::AD ONE GROUP::->[ldap_authentication->false]], [::AD OTHER GROUP::->[ldap_authentication->false]], [::OTHER GROUP ADMIN::->[auth_key->false]], [::ONE GROUP ADMIN::->[auth_key->false]], [::KIBANA-SRV::->[auth_key->true]] } ESC[0m
[2019-03-25T15:45:04,474][INFO ][t.b.r.a.ACL              ] [my-es-host] ESC[36mALLOWED by { name: '::KIBANA-SRV::', policy: ALLOW, rules: [auth_key]} req={ ID:1171508911-1101117123#24163519, TYP:PutIndexTemplateRequest, CGR:N/A, USR:kibana, BRS:false, KDX:null, ACT:indices:admin/template/put, OA:<kibana ip>, DA:<es ip>, IDX:.onegroup, MET:PUT, PTH:/_template/kibana_index_template%3A.onegroup, CNT:<OMITTED, LENGTH=3024>, HDR:{Authorization=<OMITTED>, Connection=keep-alive, content-type=application/json, Host=my-es-host.example.com:9200, Content-Length=3024}, HIS:[::AD ADMIN::->[ldap_authentication->false]], [::AD ONE GROUP::->[ldap_authentication->false]], [::AD OTHER GROUP::->[ldap_authentication->false]], [::OTHER GROUP ADMIN::->[auth_key->false]], [::ONE GROUP ADMIN::->[auth_key->false]], [::KIBANA-SRV::->[auth_key->true]] } ESC[0m

For what I understood, it looks like the user kibana is messing with my .onegroup index even if this should be the responsibility of the onegroup user (with which I logged in to create the index patterns and select the default)

In order to reproduce the issue, one can try to do the following:

  • Create a specific user (let say basic_user) with dedicated kibana_index (let say .mykibana)
  • Grant RO access on a few indices and on .mykibana index, hide administration app
  • Create another user (let say admin_user) with the same kibana_index
  • Grant this user the same indices but this time RW and allow administration app
  • Log in with admin_user, create a few index_patterns, select one to be the default
  • Log in with basic_user and go to “Discover”

The readonlyrest.yml would look like that:

readonlyrest:
  audit_collector: true

  access_control_rules:

  - name: "::basic_user::"
    auth_key: basic_user:password
    kibana_hide_apps: ["readonlyrest_kbn", "kibana:management", "whatever_other_application"]
    kibana_access: ro
    indices: [".mykibana", "index1", "index2", "etc"]
    kibana_index: ".mykibana"
    verbosity: error

  - name: "::admin_user::"
    auth_key: admin_user:strong_password
    kibana_hide_apps: ["readonlyrest_kbn", "whatever_other_application"]
    kibana_access: rw
    indices: [".mykibana", "index1", "index2", "etc"]
    kibana_index: ".mykibana"
    verbosity: error

  - name: "::Kibana server::"
    auth_key: admin:admin
    type: allow
    verbosity: error

Expected result:
basic_user sees the list of index_patterns and can make queries with filters and so on

Noticed result:
“In order to visualize and explore data in Kibana, you’ll need to create an index pattern to retrieve data from Elasticsearch.” because no default index pattern is set

1 Like

Thank you for the great bug description@zonArt, will have a look at this tomorrow.

@zonArt I ran into a similar problem a while ago, which was related to Spaces. I ended up disabling them in kibana.yml to get around it. Might be worth looking at…

2 Likes

You’re the man !! Thank you so much

Cool, glad it helped!

@sscarduzio I think I sent some logs in at the time, RoR was copying the config doc (containing the default index) from the default .kibana index when spaces were enabled.

oh yeah I remember, that logic was largely rewritten in 6.6.0. Will need to port it to pre_6.6.0 branch.