Kibana app hiding - Kibana did not load properly

[gulycka]

Hi,

after upgrading to ROR from 1.17.0 to 1.17.2, I experienced problem when logging in with my regular user.
If I enter correct credentials on login screen, I’ll get error “Kibana did not load properly. Check the server output for more information.”

I found some articles pointing out that I should check the css file (touch optimize/bundles/readonlyrest_kbn.style.css), but is in place.

I played with ROR configuration little bit and found out that following line is causing the problem:

kibana_hide_apps: [ “readonlyrest_kbn”, “timelion”, “kibana:dev_tools”, “kibana:management”, “apm”, “monitoring”, “canvas”, “infra:home”, “infra:logs” ]

When I ommited kibana_hide_apps from configuration, I was able to log in without error.

Two configurations:

1. With problem

- name: default
  type: allow
  groups: [ "default" ]
  indices: [ ".kibana", "default" ]
  kibana_hide_apps: [ "readonlyrest_kbn", "timelion", "kibana:dev_tools", "kibana:management", "apm", "monitoring", "canvas", "infra:home", "infra:logs" ]
  kibana_access: rw

- username: tester
  groups: [ "default" ]
  ldap_authentication:
    name: "ldap_connector"

2. Without problem

- name: default
  type: allow
  groups: [ "default" ]
  indices: [ ".kibana", "default" ]
  kibana_access: rw

- username: tester
  groups: [ "default" ]
  ldap_authentication:
    name: "ldap_connector"

I don’t find any suspicious logs either in kibana.log or in elasticsearch.log file.

Some version info:

elasticsearch - 6.6.1
kibana - 6.6.1
ROR Elasticsearch plugin - readonlyrest-1.17.2_es6.6.1.zip
ROR Kibana plugin - readonlyrest_kbn_pro-1.17.2_es6.6.1.zip

Kibana is behind reverse proxy (Apache).

I’ll be glad if you can take a look at it.

[sscarduzio (Simone Scarduzio)]

Hi @gulycka, will test today, thanks for reporting.

[gulycka]

Hi @sscarduzio,

thank you, just let me know if I can share you anything else.

Thanks.

[gulycka]

Hi @sscarduzio,

did you have time to test that behavior?
Is there any workaround for that?

I want to perform upgrade of our production cluster and need to decide, if to go with 1.17.0 or 1.17.2 (with some workaround for that issue).

Thanks.

[sscarduzio (Simone Scarduzio)]

1.17.0 is definitely not the case. It’s been the first release after changing so much code, it was kinda unstable. Actually, I think we’re going to release 1.17.3 as soon as next week after the bugs @susannamartinelli recently discovered (thank you Susanna ) and are now fixed.

So @gulycka, I could not reproduce the bug in my environment unfortunately. Could you have look at the chrome javascript console, if there’s some error messages? Or even in ES logs!

[gulycka]

Hi @sscarduzio,

es.log is free of errors.

But I found something in Chrome console (I just removed hostname):

<kibana_url>/:1
Refused to apply style from ‘https://<kibana_url>/’ because its MIME type (‘text/html’) is not a supported stylesheet MIME type, and strict MIME checking is enabled.

manifest.json:1
Manifest: Line: 1, column: 1, Unexpected token.

<kibana_url>/:1
Refused to apply style from ‘https://<kibana_url>/’ because its MIME type (‘text/html’) is not a supported stylesheet MIME type, and strict MIME checking is enabled.

space_selector.bundle.js:2
Uncaught TypeError: Cannot read property ‘getAttribute’ of null
at Object.1880 (space_selector.bundle.js:2)
at webpack_require (space_selector.bundle.js:2)
at checkDeferredModules (space_selector.bundle.js:2)
at space_selector.bundle.js:2
at space_selector.bundle.js:2

I checked again kibana.log and found something interesting, hope it’ll help.

Right after login:

{“type”:“log”,“@timestamp”:“2019-03-15T15:24:36Z”,“tags”:[“info”,“readonlyrest_kbn”],“pid”:23659,“message”:“try extract credentials from JSON”}
{“type”:“log”,“@timestamp”:“2019-03-15T15:24:36Z”,“tags”:[“info”,“readonlyrest_kbn”],“pid”:23659,“message”:“try extract credentials from JSON”}
{“type”:“log”,“@timestamp”:“2019-03-15T15:24:36Z”,“tags”:[“info”,“readonlyrest_kbn”],“pid”:23659,“message”:" received identity payload: {"x-ror-current-group":"default","x-ror-username":"tester","x-ror-kibana-hidden-apps":["kibana:dev_tools","timelion","kibana:management","canvas","infra:home","infra:logs","readonlyrest_kbn","monitoring","apm"],"x-ror-kibana_access":"rw","x-ror-available-groups":["all","default"]}“}
{“type”:“log”,”@timestamp":“2019-03-15T15:24:36Z”,“tags”:[“info”,“readonlyrest_kbn”],“pid”:23659,“message”:“ON_IDENTITY no kibana index from headers, setting kibana index to default configured .kibana”}

Then some access logs with 200 response status codes.
Then few interesting logs between lines.

{“type”:“log”,“@timestamp”:“2019-03-15T15:24:36Z”,“tags”:[“ror”,“warning”],“pid”:23659,“message”:“Attempted navigation towards forbidden app timelion usign path /plugins/timelion/index.css”}
{“type”:“log”,“@timestamp”:“2019-03-15T15:24:37Z”,“tags”:[“ror”,“warning”],“pid”:23659,“message”:“Attempted navigation towards forbidden app canvas usign path /plugins/canvas/style/index.css”}
{“type”:“log”,“@timestamp”:“2019-03-15T15:24:37Z”,“tags”:[“ror”,“warning”],“pid”:23659,“message”:“Attempted navigation towards forbidden app monitoring usign path /plugins/monitoring/index.css”}

Have a nice weekend.

[sscarduzio (Simone Scarduzio)]

In what page are you navigating to when you get this error? ROR is apparently detecting an attempt to access Timelion’s CSS.

[gulycka]

I’m getting this error right after login. No action is performed. I just privde my credentials and get this error.

App hiding feature probably do some “backend” request to those hidden apps, causing error.

I tried to use following rule:

kibana_hide_apps: [ “readonlyrest_kbn”, “kibana:dev_tools”, “kibana:management”, “apm”, “infra:home”, “infra:logs” ]

I was able to log in, without problems, kibana.log shows me only:

{“type”:“log”,“@timestamp”:“2019-03-18T07:12:23Z”,“tags”:[“ror”,“warning”],“pid”:23659,“message”:“Attempted navigation towards forbidden app readonlyrest_kbn usign path /plugins/readonlyrest_kbn/img/favicon.ico”}

When I used either one of the following, I ended up with error “Kibana did not load properly. Check the server output for more information.”, not able to log:

kibana_hide_apps: [ “readonlyrest_kbn”, “kibana:dev_tools”, “kibana:management”, “apm”, “infra:home”, “infra:logs”, “monitoring” ]
kibana_hide_apps: [ “readonlyrest_kbn”, “kibana:dev_tools”, “kibana:management”, “apm”, “infra:home”, “infra:logs”, “canvas” ]
kibana_hide_apps: [ “readonlyrest_kbn”, “kibana:dev_tools”, “kibana:management”, “apm”, “infra:home”, “infra:logs”, “timelion” ]

For each rule I got corresponging error, one at the time:

{“type”:“log”,“@timestamp”:“2019-03-18T07:13:16Z”,“tags”:[“ror”,“warning”],“pid”:23659,“message”:“Attempted navigation towards forbidden app canvas usign path /plugins/canvas/style/index.css”}
{“type”:“log”,“@timestamp”:“2019-03-18T07:14:05Z”,“tags”:[“ror”,“warning”],“pid”:23659,“message”:“Attempted navigation towards forbidden app monitoring usign path /plugins/monitoring/index.css”}
{“type”:“log”,“@timestamp”:“2019-03-18T07:14:50Z”,“tags”:[“ror”,“warning”],“pid”:23659,“message”:“Attempted navigation towards forbidden app timelion usign path /plugins/timelion/index.css”}

[sscarduzio (Simone Scarduzio)]

Oh I see, now I get it. Thanks for the analysis, will fix and provide new build

[gulycka]

No problem, I’m looking forward to test fixed version.
Just let me know in this ticket, that there is new build with fixed issue.

Thanks.