Basic XPack reporting is not working

gulycka · November 12, 2018, 3:57pm

Hi,

I just configured ReadonlyREST PRO on our ELK cluster.
Trying the new ES6.x features such as XPack reporting - Generate CSV from Discovery tab.
I am using both Readonlyrest plugin installed on ES and installed on Kibana.

After clicking “Generate CSV” I’m getting “Queued report for Search, Track its progress in Management”.
After few seconds I’m getting error - TypeError: Cannot read property ‘isAuthenticated’ of undefined

I tried the same without ReadonlyREST plugin and it worked well. Report was generated and I downloaded it.

I tried the same with only ReadonlyREST plugin installed for ES and I get different error - Reporting error, Can’t reach the server. Please try again."
When turning on debug logging in ES I see that the request is not authenticated - Basic auth header or auth key not present! FORBIDDEN by default.

As I saw in other posts, XPack reporting (just a basic one, without XPack license) should be supported by ROR plugin. Could you take a look at it?

Versions:
Elasticsearch: 6.4.2
Kibana: 6.4.2
ROR: 1.16.28

Kibana is running on 127.0.0.1:5601, in front of Kibana is Apache as reverse proxy running on local_ip:443.

Thanks.

sscarduzio · November 12, 2018, 11:44pm

what is the whole log line? Path, method, action, etc.

gulycka · November 14, 2018, 2:11pm

Hi sccarduzio,

thanks for your reply, below you can see the whole request processing flow from debug logs.
I think that it was related to report processing requests.
I just changed IP addresses, as this is public forum.
Hope it helps.

[2018-11-12T16:53:04,749][DEBUG][t.b.r.a.ACL ] checking request:1033665454-1697438696#2268
[2018-11-12T16:53:04,749][DEBUG][t.b.r.a.b.r.i.AuthKeySyncRule] Basic auth header or auth key not present!
[2018-11-12T16:53:04,749][DEBUG][t.b.r.e.RequestInfo ] Discovered indices: .kibana
[2018-11-12T16:53:04,749][DEBUG][t.b.r.a.b.Block ] [Kibana] the request matches no rules in this block: { ID:1033665454-1697438696#2268, TYP:GetRequest, CGR:N/A, USR:[no basic auth header], BRS:false, KDX:null, ACT:indices:data/read/get, OA:1.1.1.2, DA:1.1.1.1, IDX:.kibana, MET:GET, PTH:/.kibana/doc/index-pattern%3Afafbef80-e433-11e8-b508-b7f3b0014377, CNT:<N/A>, HDR:{Connection=keep-alive, Content-Length=0, Host=1.1.1.1:9200}, HIS:[Kibana->[auth_key->false]] }
[2018-11-12T16:53:04,749][DEBUG][t.b.r.a.b.r.i.AuthKeySyncRule] Basic auth header or auth key not present!
[2018-11-12T16:53:04,749][DEBUG][t.b.r.a.b.Block ] [Logstash] the request matches no rules in this block: { ID:1033665454-1697438696#2268, TYP:GetRequest, CGR:N/A, USR:[no basic auth header], BRS:false, KDX:null, ACT:indices:data/read/get, OA:1.1.1.2, DA:1.1.1.1, IDX:.kibana, MET:GET, PTH:/.kibana/doc/index-pattern%3Afafbef80-e433-11e8-b508-b7f3b0014377, CNT:<N/A>, HDR:{Connection=keep-alive, Content-Length=0, Host=1.1.1.1:9200}, HIS:[Kibana->[auth_key->false]], [Logstash->[auth_key->false]] }
[2018-11-12T16:53:04,749][DEBUG][t.b.r.a.b.r.i.LdapAuthenticationAsyncRule] Basic auth header not present!
[2018-11-12T16:53:04,749][DEBUG][t.b.r.a.b.Block ] [users] the request matches no rules in this block: { ID:1033665454-1697438696#2268, TYP:GetRequest, CGR:N/A, USR:[no basic auth header], BRS:false, KDX:null, ACT:indices:data/read/get, OA:1.1.1.2, DA:1.1.1.1, IDX:.kibana, MET:GET, PTH:/.kibana/doc/index-pattern%3Afafbef80-e433-11e8-b508-b7f3b0014377, CNT:<N/A>, HDR:{Connection=keep-alive, Content-Length=0, Host=1.1.1.1:9200}, HIS:[Kibana->[auth_key->false]], [Logstash->[auth_key->false]], [users->[ldap_authentication->false]] }
[2018-11-12T16:53:04,750][DEBUG][r.suppressed ] path: /.kibana/doc/index-pattern%3Afafbef80-e433-11e8-b508-b7f3b0014377, params: {index=.kibana, id=index-pattern:fafbef80-e433-11e8-b508-b7f3b0014377, type=doc}
[2018-11-12T16:53:04,753][INFO ][t.b.r.a.ACL ] FORBIDDEN by default req={ ID:1033665454-1697438696#2268, TYP:GetRequest, CGR:N/A, USR:[no basic auth header], BRS:false, KDX:null, ACT:indices:data/read/get, OA:1.1.1.2, DA:1.1.1.1, IDX:.kibana, MET:GET, PTH:/.kibana/doc/index-pattern%3Afafbef80-e433-11e8-b508-b7f3b0014377, CNT:<N/A>, HDR:{Connection=keep-alive, Content-Length=0, Host=1.1.1.1:9200}, HIS:[Kibana->[auth_key->false]], [Logstash->[auth_key->false]], [users->[ldap_authentication->false]] }

Regards.

gulycka · November 28, 2018, 3:12pm

Hi sccarduzio,

any update please?

Thanks.

ld57 · November 29, 2018, 10:26am

Hi @gulycka,

logs is a good start.

it seems your readonlyrest.yml does not have a block for kibana service.
see setup here https://github.com/beshu-tech/readonlyrest-docs/blob/master/multiuser_guide.md

could you share your readonlyrest.yml ?

gulycka · November 29, 2018, 11:02am

Hi @ld57,

thanks for your reply.

I have following readonlyrest.yml configuration, it is managed from Web GUI via Readonlyrest button:
readonlyrest:

prompt_for_basic_auth: false

access_control_rules:

###### System accounts ######

- name: Kibana
  type: allow
  auth_key_sha256: ***

- name: Logstash
  type: allow
  auth_key_sha256: ***

- name: Admin
  type: allow
  auth_key_sha256: ***

- name: Curator
  type: allow
  auth_key_sha256: ***

Then there continue definitions of Groups and Users, which I think is irrelevant for the debugging so I cut it out.
But as you can see, there is Kibana user, with name and password.
I put the same name and password to /etc/kibana/kibana.yml as:
elasticsearch.username: ***
elasticsearch.password: ***

Do you see anything that can cause the problems?

Thanks.

ld57 · November 29, 2018, 10:53pm

hi @gulycka

well , the unknown USR is abnormal.

I need to check back on my own kibana.yml and readonlyrest config to write here my settings, you may see a difference or something which would help you.

sscarduzio · December 2, 2018, 11:08am

So to sum up the problem is that in 6.4.3, when you use certain XPack specific feature in Kibana, no credentials are sent to ES. Right?

gulycka · December 3, 2018, 12:44pm

Hi @sscarduzio & @ld57,

you are right, it looks like credentials are not sent to ES.
I am using 6.4.2 at the moment.

Thanks.

ld57 · December 5, 2018, 12:51pm

I do not meet the issue on 6.2.1 unfortunately I am late to migrate, ( also I am late to everything )

it works fine, the report is generated to management section reporting.

here are activities when I created and saved a search , then used reporting/generate csv

December 5th 2018, 13:44:38.000	indices:data/read/msearch	/_msearch
	December 5th 2018, 13:44:37.000	indices:data/read/mget	/.kibana_all/_mget
	December 5th 2018, 13:44:37.000	indices:data/read/search	/.kibana_all/_search?size=10000&from=0&_source=index-pattern.title%2Ctype%2Ctitle
	December 5th 2018, 13:44:15.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:15.000	indices:data/read/scroll/clear	/_search/scroll
	December 5th 2018, 13:44:14.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:14.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:13.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:13.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:12.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:12.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:12.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:11.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:11.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:10.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:10.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:10.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:09.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:09.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:08.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:08.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:07.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:07.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:07.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:06.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:06.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:05.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:05.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:05.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:04.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:04.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:03.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:03.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:02.000	indices:data/read/scroll	/_search/scroll?scroll=30s
	December 5th 2018, 13:44:01.000	indices:data/read/get	/.kibana_all/doc/config%3A6.2.1
	December 5th 2018, 13:44:01.000	indices:data/read/get	/.kibana_all/doc/config%3A6.2.1
	December 5th 2018, 13:44:01.000	indices:data/read/search	/log_lu_ec_*/_search?scroll=30s&size=500
	December 5th 2018, 13:44:01.000	indices:data/read/get	/.kibana_all/doc/config%3A6.2.1
	December 5th 2018, 13:43:56.000	indices:data/read/get	/.kibana_all/doc/index-pattern%3A960dc170-ab99-11e8-9fa1-e5f0bbae96d8
	December 5th 2018, 13:43:48.000	indices:data/read/msearch	/_msearch
	December 5th 2018, 13:43:47.000	indices:data/read/mget	/.kibana_all/_mget
	December 5th 2018, 13:43:47.000	indices:data/read/search	/.kibana_all/_search?size=10000&from=0&_source=index-pattern.title%2Ctype%2Ctitle
	December 5th 2018, 13:43:46.000	indices:data/write/index	/.kibana_all/doc/search%3A677c44a0-f88b-11e8-a384-e90084dffa5c?refresh=wait_for
	December 5th 2018, 13:43:45.000	indices:data/read/search	/.kibana_all/_search?size=10&from=0&_source=search.title%2Ctype%2Ctitle
	December 5th 2018, 13:43:35.000	indices:data/read/search	/.kibana_all/_search?size=1000&from=0
	December 5th 2018, 13:43:25.000	indices:data/read/msearch	/_msearch
	December 5th 2018, 13:43:24.000	indices:data/read/mget	/.kibana_all/_mget
	December 5th 2018, 13:43:23.000	indices:data/read/search	/.kibana_all/_search?size=10000&from=0&_source=index-pattern.title%2Ctype%2Ctitle
	December 5th 2018, 13:43:21.000	indices:data/read/search	/.kibana_all/_search?size=10000&from=0
	December 5th 2018, 13:43:18.000	cluster:monitor/nodes/info	/_nodes/_local
	December 5th 2018, 13:43:18.000	indices:data/read/get	/.kibana_all/doc/config%3A6.2.1

ld57 · December 5, 2018, 12:52pm

I ll write down here rules my user belong to.

sscarduzio · December 5, 2018, 1:31pm

6.2.1 is unaffected, makes sense. The saved object API changed in 6.3.x.

ld57 · December 5, 2018, 9:56pm

ah bleh.

good to know.

now i understand the headhache sentinl team had
ok then for now I can not help more at my level.

I should have more time next year to follow up and test latest version.

Kr

Ld

gulycka · December 6, 2018, 5:24pm

Hi @sscarduzio,

would it be possible to check the behaviour in 6.4.x or 6.5.x and fix it?
Reporting is quite nice feature and it will be great to have it there. At the moment we aren’t able to use it even with fully privileged user.

Thank you.

sscarduzio · December 27, 2018, 4:58am

Hi guys, this is resolved now. Every user has their own list of reports. Will be available in 1.16.32, soon to be released. Apologies for the wait!

gulycka · January 4, 2019, 10:37am

Hi @sscarduzio,

that’s great news!
I’ve just tested it and it works perfect.

Thank you very much for fixing the issue, reporting is very nice feature that comes with ES out of the box.

Have a nice day.

gulycka · January 8, 2019, 1:50pm

Hi @sscarduzio,

could you tell me more about own list of reports for each user?
Is it working only in newest version of Elasticsearch / Kibana, or is it matter of ROR plugin?

I am running on ES 6.4.2 with ROR plugin 1.16.32, but I can’t find the user specific report tab.

Thanks.

sscarduzio · January 8, 2019, 6:24pm

Tomorrow I will release a new version with better support for older version.

gulycka · January 17, 2019, 2:25pm

Hi,
is it Enterprise only feature, or does it also work in PRO version (according to the release notes Enhancement - (Enterprise) Now tenants save their CSV exports in their own reporting index)?
Is it implemented in 1.16.33?

Thanks.

sscarduzio · January 17, 2019, 6:54pm

@gulycka the issue was with multiple tenancies, so the solution and the problem was Enterprise only.
Why do you ask?
Are you using PRO and having problems with the latest version?