Kibana Status red after Installing Readonly REST

pk125xl · May 31, 2018, 8:25am

Installed readonlyrest-1.16.19_es6.2.2 on a local machine with EL and Kibana 6.2.2. Kibana Status is red. What I’m doing wrong ?
Thanks for help this newbie

readonlyrest.yaml :
readonlyrest:
enable: true
response_if_req_forbidden: Forbidden by ReadonlyREST ES plugin

ssl:
  enable: false
  keystore_file: "/elasticsearch/plugins/readonlyrest/elasticsearch.jks"
  keystore_pass: PASS_JKS
  key_pass: PASS_JKS

access_control_rules:
- name: "Block 1 - Allowing kibana to access"
  type: allow
  auth_key: kibana:PASS_KIBANA
  verbosity: error
  actions: ["cluster:monitor/main", "cluster:monitor/nodes/info", "indices:admin/mappings/get"]

- name: "Block 2 - read only for other indices"
  type: allow
  auth_key: ui:PASS_UI
  kibana_access: ro
  actions: ["indices:data/read/*", "indices:admin/get"]

- name: "Block 3 - read write access to indices for application"
  type: allow
  auth_key: appl:PASS_APPL
  verbosity: error

- name: "Block 4 - admin user for emergency"
  type: allow
  auth_key: admin:PASS_ADMIN
  kibana_access: admin

- name: "Block 5 - monitoring User"
  type: allow
  auth_key: mon:PASS_MON
  actions: ["cluster:monitor/main", "cluster:monitor/health", "cluster:monitor/nodes/stats", "cluster:monitor/state", "cluster:monitor/stats", "indices:monitor/stats"]

Elasticsearch Log File:

C:\pers\kibana-6.2.2-windows-x86_64\kibana-6.2.2-windows-x86_64\bin>.\kibana.bat
log [08:17:07.323] [info][status][plugin:kibana@6.2.2] Status changed from uninitialized to green - Ready
log [08:17:07.407] [info][status][plugin:elasticsearch@6.2.2] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [08:17:08.549] [info][status][plugin:timelion@6.2.2] Status changed from uninitialized to green - Ready
log [08:17:08.559] [info][status][plugin:console@6.2.2] Status changed from uninitialized to green - Ready
log [08:17:08.565] [info][status][plugin:metrics@6.2.2] Status changed from uninitialized to green - Ready
log [08:17:08.618] [info][listening] Server running at http://localhost:5601
log [08:17:08.836] [error][status][plugin:elasticsearch@6.2.2] Status changed from yellow to red - Authentication Exception

[2018-05-31T10:20:06,716][INFO ][t.b.r.a.ACL ] e[35mFORBIDDEN by default req={ ID:589574002-970298994#48, TYP:MainRequest, CGR:N/A, USR:[no basic auth header], BRS:false, KDX:null, ACT:cluster:monitor/main, OA:127.0.0.1, DA:127.0.0.1, IDX:<N/A>, MET:HEAD, PTH:/, CNT:<N/A>, HDR:{Connection=keep-alive, Content-Length=0, Host=localhost:9200}, HIS:[Block 1 - Allowing kibana to access->[auth_key->false]], [Block 2 - read only for other indices->[auth_key->false]], [Block 3 - read write access to indices for application->[auth_key->false]], [Block 4 - admin user for emergency->[auth_key->false]], [Block 5 - monitoring User->[auth_key->false]] } e[0m
[2018-05-31T10:20:06,716][INFO ][t.b.r.a.ACL ] e[35mFORBIDDEN by default req={ ID:1859738829-2026337532#47, TYP:MainRequest, CGR:N/A, USR:[no basic auth header], BRS:false, KDX:null, ACT:cluster:monitor/main, OA:127.0.0.1, DA:127.0.0.1, IDX:<N/A>, MET:HEAD, PTH:/, CNT:<N/A>, HDR:{Connection=keep-alive, Content-Length=0, Host=localhost:9200}, HIS:[Block 1 - Allowing kibana to access->[auth_key->false]], [Block 2 - read only for other indices->[auth_key->false]], [Block 3 - read write access to indices for application->[auth_key->false]], [Block 4 - admin user for emergency->[auth_key->false]], [Block 5 - monitoring User->[auth_key->false]] } e[0m
[2018-05-31T10:20:09,222][INFO ][t.b.r.a.ACL ] e[35mFORBIDDEN by default req={ ID:1235125248-118906047#49, TYP:MainRequest, CGR:N/A, USR:[no basic auth header], BRS:false, KDX:null, ACT:cluster:monitor/main, OA:127.0.0.1, DA:127.0.0.1, IDX:<N/A>, MET:HEAD, PTH:/, CNT:<N/A>, HDR:{Connection=keep-alive, Content-Length=0, Host=localhost:9200}, HIS:[Block 1 - Allowing kibana to access->[auth_key->false]], [Block 2 - read only for other indices->[auth_key->false]], [Block 3 - read write access to indices for application->[auth_key->false]], [Block 4 - admin user for emergency->[auth_key->false]], [Block 5 - monitoring User->[auth_key->false]] } e[0m
[2018-05-31T10:20:11,732][INFO ][t.b.r.a.ACL ] e[35mFORBIDDEN by default req={ ID:1124847677-1459140648#52, TYP:MainRequest, CGR:N/A, USR:[no basic auth header], BRS:false, KDX:null, ACT:cluster:monitor/main, OA:127.0.0.1, DA:127.0.0.1, IDX:<N/A>, MET:HEAD, PTH:/, CNT:<N/A>, HDR:{Connection=keep-alive, Content-Length=0, Host=localhost:9200}, HIS:[Block 1 - Allowing kibana to access->[auth_key->false]], [Block 2 - read only for other indices->[auth_key->false]], [Block 3 - read write access to indices for application->[auth_key->false]], [Block 4 - admin user for emergency->[auth_key->false]], [Block 5 - monitoring User->[auth_key->false]] } e[0m
[2018-05-31T10:20:14,238][INFO ][t.b.r.a.ACL ] e[35mFORBIDDEN by default req={ ID:118155470-629632947#53, TYP:MainRequest, CGR:N/A, USR:[no basic auth header], BRS:false, KDX:null, ACT:cluster:monitor/main, OA:127.0.0.1, DA:127.0.0.1, IDX:<N/A>, MET:HEAD, PTH:/, CNT:<N/A>, HDR:{Connection=keep-alive, Content-Length=0, Host=localhost:9200}, HIS:[Block 1 - Allowing kibana to access->[auth_key->false]], [Block 2 - read only for other indices->[auth_key->false]], [Block 3 - read write access to indices for application->[auth_key->false]], [Block 4 - admin user for emergency->[auth_key->false]], [Block 5 - monitoring User->[auth_key->false]] } e[0m
[2018-05-31T10:20:16,741][INFO ][t.b.r.a.ACL ] e[35mFORBIDDEN by default req={ ID:707967796-1523730112#56, TYP:MainRequest, CGR:N/A, USR:[no basic auth header], BRS:false, KDX:null, ACT:cluster:monitor/main, OA:127.0.0.1, DA:127.0.0.1, IDX:<N/A>, MET:HEAD, PTH:/, CNT:<N/A>, HDR:{Connection=keep-alive, Content-Length=0, Host=localhost:9200}, HIS:[Block 1 - Allowing kibana to access->[auth_key->false]], [Block 2 - read only for other indices->[auth_key->false]], [Block 3 - read write access to indices for application->[auth_key->false]], [Block 4 - admin user for emergency->[auth_key->false]], [Block 5 - monitoring User->[auth_key->false]] } e[0m
[2018-05-31T10:20:19,256][INFO ][t.b.r.a.ACL ] e[35mFORBIDDEN by default req={ ID:411324807-759305612#57, TYP:MainRequest, CGR:N/A, USR:[no basic auth header], BRS:false, KDX:null, ACT:cluster:monitor/main, OA:127.0.0.1, DA:127.0.0.1, IDX:<N/A>, MET:HEAD, PTH:/, CNT:<N/A>, HDR:{Connection=keep-alive, Content-Length=0, Host=localhost:9200}, HIS:[Block 1 - Allowing kibana to access->[auth_key->false]], [Block 2 - read only for other indices->[auth_key->false]], [Block 3 - read write access to indices for application->[auth_key->false]], [Block 4 - admin user for emergency->[auth_key->false]], [Block 5 - monitoring User->[auth_key->false]] } e[0m
[2018-05-31T10:20:21,762][INFO ][t.b.r.a.ACL ] e[35mFORBIDDEN by default req={ ID:1195765663-1416574631#60, TYP:MainRequest, CGR:N/A, USR:[no basic auth header], BRS:false, KDX:null, ACT:cluster:monitor/main, OA:127.0.0.1, DA:127.0.0.1, IDX:<N/A>, MET:HEAD, PTH:/, CNT:<N/A>, HDR:{Connection=keep-alive, Content-Length=0, Host=localhost:9200}, HIS:[Block 1 - Allowing kibana to access->[auth_key->false]], [Block 2 - read only for other indices->[auth_key->false]], [Block 3 - read write access to indices for application->[auth_key->false]], [Block 4 - admin user for emergency->[auth_key->false]], [Block 5 - monitoring User->[auth_key->false]] } e[0m

sscarduzio · May 31, 2018, 9:26am

I think you forgot to change the elasticsearch username and password inside the kibana.yml

pk125xl · May 31, 2018, 12:26pm

Thank You so much, that solved most of the problems. One still remained. When I try to create an Index Pattern I get following error:
[object Object]: [undefined] Forbidden by ReadonlyREST ES plugin

I tried both with the kiban User and with the Admin User too.

name: “Block 4 - admin user for emergency”
type: allow
auth_key: admin:PASS_ADMIN
kibana_access: admin

Sorry

sscarduzio · May 31, 2018, 12:30pm

Hi @pk125xl,

To understand why it’s failing, we need to see the “FORBIDDEN” line in the ES logs and the readonlyrest.yml settings (if they changed from above). Especially the “HST” (history) block where all the ACL execution logs are shown.

pk125xl · May 31, 2018, 1:04pm

These are the entries showing forbidden in the log after browsing in Kibana - 4 entries:
[2018-05-31T14:55:22,052][INFO ][t.b.r.a.ACL ] e[35mFORBIDDEN by default req={ ID:1699455688-1170500175#995, TYP:MainRequest, CGR:N/A, USR:[no basic auth header], BRS:true, KDX:null, ACT:cluster:monitor/main, OA:127.0.0.1, DA:0:0:0:0:0:0:0:1, IDX:<N/A>, MET:GET, PTH:/, CNT:<N/A>, HDR:{Accept=text/html, application/xhtml+xml, image/jxr, /, Accept-Encoding=gzip, deflate, peerdist, Accept-Language=de-CH,de;q=0.9,en-US;q=0.8,en;q=0.6,it-CH;q=0.5,it;q=0.4,fr-CH;q=0.3,fr;q=0.1, Connection=Keep-Alive, content-length=0, DNT=1, Host=localhost:9200, User-Agent=Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; Touch; rv:11.0) like Gecko, X-P2P-PeerDist=Version=1.1, X-P2P-PeerDistEx=MinContentInformation=1.0, MaxContentInformation=2.0}, HIS:[Block 1 - Allowing kibana to access->[auth_key->false]], [Block 2 - read only for other indices->[auth_key->false]], [Block 3 - read write access to indices for application->[auth_key->false]], [Block 4 - admin user for emergency->[auth_key->false]], [Block 5 - monitoring User->[auth_key->false]] } e[0m
[2018-05-31T14:55:59,917][INFO ][t.b.r.a.ACL ] e[35mFORBIDDEN by default req={ ID:2091776504-1824209315#1109, TYP:GetRequest, CGR:N/A, USR:[no basic auth header], BRS:false, KDX:null, ACT:indices:data/read/get, OA:127.0.0.1, DA:127.0.0.1, IDX:.kibana, MET:GET, PTH:/.kibana/doc/config%3A6.2.2, CNT:<N/A>, HDR:{Connection=keep-alive, Content-Length=0, Host=localhost:9200}, HIS:[Block 1 - Allowing kibana to access->[auth_key->false]], [Block 2 - read only for other indices->[auth_key->false]], [Block 3 - read write access to indices for application->[auth_key->false]], [Block 4 - admin user for emergency->[auth_key->false]], [Block 5 - monitoring User->[auth_key->false]] } e[0m
[2018-05-31T14:55:59,942][INFO ][t.b.r.a.ACL ] e[35mFORBIDDEN by default req={ ID:1615024545-1932662376#1110, TYP:GetRequest, CGR:N/A, USR:kibana(?), BRS:false, KDX:null, ACT:indices:data/read/get, OA:127.0.0.1, DA:127.0.0.1, IDX:.kibana, MET:GET, PTH:/.kibana/doc/config%3A6.2.2, CNT:<N/A>, HDR:{authorization=Basic a2liYW5hOlBBU1NfS0lCQU5B, Connection=keep-alive, Authorization=, Host=localhost:9200, Content-Length=0}, HIS:[Block 1 - Allowing kibana to access->[auth_key->true, actions->false]], [Block 2 - read only for other indices->[auth_key->false]], [Block 3 - read write access to indices for application->[auth_key->false]], [Block 4 - admin user for emergency->[auth_key->false]], [Block 5 - monitoring User->[auth_key->false]] } e[0m
[2018-05-31T14:56:56,730][INFO ][t.b.r.a.ACL ] e[35mFORBIDDEN by default req={ ID:1367089437–1403020886#1283, TYP:SearchRequest, CGR:N/A, USR:[no basic auth header], BRS:false, KDX:null, ACT:indices:data/read/search, OA:127.0.0.1, DA:127.0.0.1, IDX:.kibana, MET:POST, PTH:/.kibana/_search?size=10000&from=0, CNT:<OMITTED, LENGTH=80>, HDR:{Connection=keep-alive, Content-Length=80, content-type=application/json, Host=localhost:9200}, HIS:[Block 1 - Allowing kibana to access->[auth_key->false]], [Block 2 - read only for other indices->[auth_key->false]], [Block 3 - read write access to indices for application->[auth_key->false]], [Block 4 - admin user for emergency->[auth_key->false]], [Block 5 - monitoring User->[auth_key->false]] } e[0m

sscarduzio · May 31, 2018, 10:15pm

Hi @pk125xl

Where did you read this from? Why did you add that action rule? Is there misleading documentation anywhere?

Never mix kibana_access with actions rule!! Remove that actions rule.