How/when does .kibana_{user} get created?

1

NOTE: The below is without the ror kbn plugin installed? So maybe the .kibana index translations is done in the client and not the es server?

I’m trying to automate the creation of the index patterns (and other kibana objects). But, if I understand correctly, those objects need to be in the user-index. Just using a user with a kibana_index does not seem to create the index, eg

        - name: plant-admin--kibana-rw
          headers:
            - x-vi-plant:*
          auth_key: "plantadmin:{{ vault_ror_admin_password }}"
          kibana_index: .kibana_@{x-vi-plant}

curl --user 'plantadmin:plantadmin \
-X POST "http://localhost:5601/api/saved_objects/index-pattern/test_1" \
-H 'kbn-xsrf: true' -H 'Content-Type: application/json' \
-H 'x-vi-plant: male' \
-d'
{
  "attributes": {
    "title": "kibana_sample_*",
    "timeFieldName": "order_date"
  }
}
'

The pattern does get created, but in the .kibana index not .kibana_male. The logs shows KDX being kibana_male but the the PTH still uses .kibana. Am I missing something?

[2018-12-25T21:32:38,735][INFO ][tech.beshu.ror.acl.ACL   ] [s-ror-es-1] ALLOWED by { name: 'plant-admin', policy: ALLOW, rules: [auth_key, headers, kibana_index]} req={ ID:87760583-106528550#3896, TYP:IndexRequest, CGR:N/A, USR:plantadmin, BRS:false, KDX:.kibana_male, ACT:indices:data/write/index, OA:10.11.136.165, DA:10.11.136.165, IDX:.kibana, MET:POST, PTH:/.kibana/doc/index-pattern%3Atest_1/_create?refresh=wait_for, CNT:<OMITTED, LENGTH=182>, HDR:{authorization=<OMITTED>, x-vi-plant=male, Connection=keep-alive, content-type=application/json, Host=10.11.136.165:9200, Content-Length=182}, HIS:[svc-consul->[auth_key->false]], [svc-kibana->[auth_key->false]], [plant-admin->[headers->true, auth_key->true, kibana_index->true]] }
2

I fired up another kibana with ror plugin installed. In this kibana, it forwards the request to es but does not include the headers.

Is the ror plugin not handling these routes /api/... correctly or at all?

kibana.yml (partial)

elasticsearch.requestHeadersWhitelist:
  - authorization
  - x-forwarded-for
  - x-forwarded-user
  - x-vi-plant

kibana log:

{"type":"response","@timestamp":"2018-12-26T02:08:17Z","tags":[],"pid":1,"method":"post","statusCode":403,"req":{"url":"/api/saved_objects/index-pattern/test_1?overwrite=false","method":"post","headers":{"host":"localhost:5601","user-agent":"curl/7.54.0","accept":"*/*","kbn-xsrf":"true","content-type":"application/json","x-vi-plant":"male","content-length":"93"},"remoteAddress":"192.168.0.1","userAgent":"192.168.0.1"},"res":{"statusCode":403,"responseTime":880,"contentLength":9},"message":"POST /api/saved_objects/index-pattern/test_1?overwrite=false 403 880ms - 9.0B"}

ror audit log:

[2018-12-26T02:08:17,615][INFO ][tech.beshu.ror.acl.ACL   ] [s-ror-es-1] FORBIDDEN by default req={ ID:1957800223-448931582#402, TYP:IndexRequest, CGR:N/A, USR:plantadmin(?), BRS:false, KDX:null, ACT:indices:data/write/index, OA:10.0.192.247, DA:10.11.136.165, IDX:.kibana, MET:POST, PTH:/.kibana/doc/index-pattern%3Atest_1/_create?refresh=wait_for, CNT:<OMITTED, LENGTH=182>, HDR:{authorization=<OMITTED>, Connection=keep-alive, content-type=application/json, Host=s-ror-es-1.use1.systeminsights.com:9200, Content-Length=182}, HIS:[svc-consul->[auth_key->false]], [svc-kibana->[auth_key->false]], [plant-admin--kibana-rw->[headers->false, kibana_access->true, auth_key->true]], [plant-admin--kibana-r0->[kibana_access->true, headers->false, auth_key->true]], [plant-user--kibana-rw->[proxy_auth->false]], [plant-user--kibana-ro->[proxy_auth->false]], [plant-user--data->[proxy_auth->false]] }
3

Now I’m not so sure my testing is consistent. I am trying to reproduce my first post, where I create a pattern via kibana without ror plugin installed.

I get the es log

{"@version":1,"source_host":"s-ror-es-1","message":"adding template [kibana_index_template:.kibana] for index patterns [.kibana]","thread_name":"elasticsearch[s-ror-es-1][masterService#updateTask][T#1]","@timestamp":"2018-12-26T02:16:21.849+00:00","level":"INFO","logger_name":"org.elasticsearch.cluster.metadata.MetaDataIndexTemplateService"}

No ror audit log, and the pattern is created.

So clearly I do not have the correct model of how these kibana /api routes are being processed.

4

@memelet if the kibana index does not exist, it will be initialised by our Kibana plugin at first login.
Index initialisation includes the creation of the index, and the mappings. It does not include any Kibana objects: i.e. index patterns.

If you want more headers to be passed to ES to requests coming through Kibana, use elasticsearch.requestHeadersWhitelist: [...your headers...] like explained in Elastic’s Kibana documentation.

And of course, If you are delegating Kibana authentication to a reverse proxy, you have to enable proxy auth passthrough in our Kibana plugin, like explained in our documentation.

5

Wait!!! Sorry, the ror plugin is NOT installed. Sigh, too many experiments going at once. I’ll update the below once that bit is installed.

I went from using kibana via apt to docker. Discovered that kibana ignores the /api routes if kibana.server is not defined. But to figure that out I had to comment out most things. Not all got put back.

(Waiting for kibana to optimize the plugin for 5 minutes during docker build, and then again during docker run is fraying my nerves)

6

Ok, I’m back to where I started. When kibana with ror plugin installed receives request on the /api routes it seems to eat them.

It gets the request and kibana logs it

ex 1

curl --user 'plantadmin:plantadmin' \  
-X POST "http://localhost:5601/api/saved_objects/index-pattern/test_1" \
-H 'kbn-xsrf: true' -H 'Content-Type: application/json' \
-H 'x-vi-plant: male' \
-d'
{
  "attributes": {
    "title": "kibana_sample_*",
    "timeFieldName": "order_date"
  }
}
'

kibana log

{"type":"response","@timestamp":"2018-12-26T04:14:53Z","tags":[],"pid":1,"method":"post","statusCode":302,"req":{"url":"/api/saved_objects/index-pattern/test_1","method":"post","headers":{"host":"localhost:5601","user-agent":"curl/7.54.0","accept":"*/*","kbn-xsrf":"true","content-type":"application/json","x-vi-plant":"male","content-length":"93"},"remoteAddress":"192.168.16.1","userAgent":"192.168.16.1"},"res":{"statusCode":302,"responseTime":1,"contentLength":9},"message":"POST /api/saved_objects/index-pattern/test_1 302 1ms - 9.0B"}

ex 2

curl --user 'plantadmin:plantadmin' \ 
-X GET "http://localhost:5601/api/saved_objects/index-pattern/ff959d40-b880-11e8-a6d9-e546fe2bba5f" \
-H 'kbn-xsrf: true' -H 'Content-Type: application/json' \
-H 'x-vi-plant: male'

kibana log

{"type":"response","@timestamp":"2018-12-26T04:16:59Z","tags":[],"pid":1,"method":"get","statusCode":302,"req":{"url":"/api/saved_objects/index-pattern/ff959d40-b880-11e8-a6d9-e546fe2bba5f","method":"get","headers":{"host":"localhost:5601","user-agent":"curl/7.54.0","accept":"*/*","kbn-xsrf":"true","content-type":"application/json","x-vi-plant":"male"},"remoteAddress":"192.168.16.1","userAgent":"192.168.16.1"},"res":{"statusCode":302,"responseTime":3,"contentLength":9},"message":"GET /api/saved_objects/index-pattern/ff959d40-b880-11e8-a6d9-e546fe2bba5f 302 3ms - 9.0B"}

Nothing in the es logs, and nothing in the ror audit logs.

(running now with logging.verbose…)

7

Not much from the verbose logging other than

{"type":"log","@timestamp":"2018-12-26T04:33:19Z","tags":["trace","legacy","service"],"pid":1,"message":"Request will be handled by proxy GET:/api/saved_objects/index-pattern/ff959d40-b880-11e8-a6d9-e546fe2bba5f."}

Not sure if that is the ror proxy or some kibana proxy.

8

@sscarduzio Should I expect that ror_kbn will drop /api requests?

9

@memelet not sure why you want to create the kibana object without a valid cookie, but we are about to release a feature that will ease your tasks (if I understood correctly).

Basically you will be able to configure a template kibana index i.e. “.kibana_template” containing all the default Kibana objects (index patterns, dashboards, visualizations, Kibana settings) that will be copied into a newly created tenancy (kibana index).

10

@sscarduzio Because the object is being created by automation when provisioning a new tenant – not by a human via kibana.

Was the answer to my question “yes” then? Because there is no cookie when using kibana’s /api via code?

11

Yes, that is exactly what I am trying to do. But not kibana “default” values, I need to configure each tenant with objects specific to our applications. (Maybe that is what you meant.)

12

You mean you need each tenant to have some tenant-specific values preconfigured? Or just setting up common defaults?

13

Not tenant-specific. Each tenant would be configured with the same dashboards, etc.

Although it might make sense to assign different index patterns on per tenant basis for those indices that split out by tenant if that would be more efficient.

14

Yeah I was thinking about copying over the documents customised on a per-user basis. But not now. Now we need to release ASAP.