cookiePass and kibanaIndexTemplate settings do not work when given as CLI arguments

Kibana ReadOnlyRest Version: 1.16.33
Elasticsearch ReadOnlyRest Version: 1.16.33
Kibana & Elasticsearch version: 6.5.4

We are running Kibana in a docker container and giving compatible options (kibana-docker/kibana-docker at master · elastic/kibana-docker · GitHub) as ENV vars, while rest of the configuration as CLI arguments.

The following are the CLI arguments we give:

[
  "/usr/local/bin/kibana-docker",
  "--readonlyrest_kbn.proxy_auth_passthrough=true", 
  "--readonlyrest_kbn.kibanaIndexTemplate=.kibana_template",
  "--readonlyrest_kbn.cookiePass=cookiepassthathasthirtytwocharactersormore",
  "--elasticsearch.requestHeadersWhitelist[0]=x-forwarded-user",
  "--elasticsearch.requestHeadersWhitelist[1]=x-tenant",
  "--elasticsearch.requestHeadersWhitelist[2]=authorization",
  "--xpack.spaces.enabled=false"
]

The rest of the configuration such as Elasticsearch URL, password, etc are given as environment variables. This docker container runs inside a Nomad Cluster.
When running just one instance of Kibana with installed ROR, we find that the application works seamlessly.

We are using proxy auth, so it can be said that the setting --readonlyrest_kbn.proxy_auth_passthrough=true worked. So, we also added the cookiePass and the kibanaIndexTemplate settings as CLI arguments. However, we find that:

  1. When a new user logs in, the .kibana_template is not copied to the .kibana_{tenant} index.
  2. When we run multiple instances of kibana behind the proxy, the auth sessions is not retained and the user gets logged out abruptly.

Is it that these configurations cannot be passed as CLI arguments? Or we are doing something wrong here?

Hi @anishm,

readonlyrest_kbn.cookiePass only works in old versions of Kibana as stated in our documentation. Please use sticky sessions in your load balancer and try again.

This will cause a huge amount of havoc in our system. All kibana’s run in the nomad/k8 cluster. They can be restarted/moved at any time. Sticky balancing will not help.

Indeed this would require us to run a kibana-per-tenant, which makes the RoR kibana plugin not needed. Am I misunderstanding?

We don’t see this behavior with xpack. We can login, then move across kibana instances. (@anishm , is this really correct?)

mmmm now that you made me think about it, I think I found the problem. You will have a test build tomorrow. Ok?

Yes, the setting in xpack setting XPACK_SECURITY_ENCRYPTIONKEY allows us to share a common “cookiePass” across x-pack Kibana instances.
Can you also please let me know why is the configuration for kibanaIndexTemplate not passing through? We have that setting enabled in command line, but the indices .kibana_{tenant} for new tenants don’t have the data that the template did.

Not sure why it didn’t pass, or simply something did not work. Maybe when you test the new build I’m giving you, try again after deleting the .kibana_tenant index after commenting out some “verbosity: error” in readonlyrest.yml so we shall see if the reindex request (copying the documents from template kibana index) comes in or not, and if fails.

Sent you the build in private.

Hey @anishm, how did it go?

I tested the build you sent, and the cookiePass setting worked. I ran multiple kibana indices and the cookie was able to maintain the session and did not get logged out.
However, the kibana_template is not working as of now. I tried with a new tenant, and an older one.
I ran with verbosity info, but did not see reindexing starting up.

Is this just an issue with passing the configuration key via CLI? I mean, if you repeat the experiment with a tweaked kibana.yml it works?

Let’s try to narrow the circle…