Some issues with version 1.16.27


when i work with version 1.16.21 (es & kibana) everything works fine.

i’ve update to 1.16.27 (es & kibana) and the user get :
Could not login: "Forbidden (403) "

in the logging index i can see :
“origin”: kiban_server_ip,
“match”: false,
“final_state”: “FORBIDDEN”,
“destination”: “”,
“task_id”: 7099,
“type”: “RRAdminRequest”,
“req_method”: “GET”,
“path”: “/_readonlyrest/metadata/current_user”,
“indices”: [],
@timestamp”: “…”,
“content_len_kb”: 0,
“error_type”: null,
“processingMillis”: 39,
“action”: “cluster:admin/rradmin/refreshsettings”,
“block”: “default”,
“id”: “…”,
“content_len”: 0,
“user”: “user1”

the user is identified by ldap.

important : i’m working with the SAME readonlyrest.yml in both versions !

when working with version 1.16.21 it works, when upgrading to 1.16.27 it failes.

what am i missing ?

(Simone Scarduzio) #2

Can you show the full FORBIDDEN log line? including the history (“HIS”)


i couldn’t find a HIS/history line in the elastic log file nor in the kibana log file.

the closest thing is the “acl_history” field i found in the index.
it contains the acl’s blocks from my readonlyrest.yml.

(Simone Scarduzio) #4

yes indeed, can you share the content of that field? It basically shows a trace of how the request was evaluated by the ACL, that is, what rules and what blocks matched and in what chronological order.


[WEBSITE SEARCH BOX::->[indices->true, actions->false]],
[allow all for admin->[ldap_authentication->false]],
[for ROR cleanup->[ldap_authentication->false]],
[for monitoring->[ldap_authentication->false]],
[requests from user for indices d*->[ldap_authentication->false]],
[requests from user for indices a*->[ldap_authentication->false]],
[requests from user for indices m1*->[ldap_authentication->false]],
[requests from user for indices p*->[ldap_authentication->false]],
[requests from user for indices m2*->[ldap_authentication->false]],
[requests from user for indices i*->[ldap_authentication->false]]

(Simone Scarduzio) #6

Great, thank you @sdba2!
And what block should normally (in 1.16.21) this request by “user1” match?
Is this a valid LDAP user that ldap_authentication should match?


the block with index d*.
the user is a valid ldap authentication and it uses ldap to authenticate.
(when running : curl -u user1:pw1 … it works)

(Simone Scarduzio) #8

This works?

curl -vvv -X GET 'http://es:9200/_readonlyrest/metadata/current_user' -u user1:password

Can you show me the output?

* about to connect() to  server1 port 9200 (#0)
*   trying connected
* connected to server1 ( port 9200 (#0)
* trying connected
* server auth using basic with user 'user1'
> get /_readonlyrest/metadata/current_user HTTP/1.1
> authorization: basic <some hash value>
> user-agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 nss/3.18 basic ecc zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> host: server1:9200
> accept: */*
< http/1.1 403 forbidden
< content-type: application/json; charset=utf-8
< connect-length: 103
* connection #0 to host server1 left intact
* closing connect #0
{"error":{"root_cause":[{"reason":"rejected by ror !!!"}],"reason":"rejected by ror !!!"},"status":403}

(Simone Scarduzio) #10

Strange, what ES version is this?
Does it reject similar requests for locally defined users too? Or is it just an LDAP thing?


this is ES 6.1.1

when i’m using another user (also ldap user) and his acl looks like :

name: “ldap1”
groups: [“A123”]
indices: ["*"]
action: ["*"]

it works (kiban login and the curl command)

for user1 the acl looks like:

name: “ldap1”
groups: [“A456”]
indices: [“d*”,".kibana*"]
actions: [“indices:admin/*”,“indices:data/write/*”,“indices/data/read/*”,“cluster:monitor/*”,“indices:monitor/*”]
kibana_hide_apps: [“readonlyrest_kbn”,“timelion”]

and it fails to login to kiban and the curl also fails

(Simone Scarduzio) #12

Ok mystery solved.
You are not using kibana_access macro rule, and you are using raw actions. I highly recommend that you remove the actions rule in favour of a kibana_access: rw/ro/ro_strict/admin.

Please read the explanation in the docs about what kibana_access does and why it’s preferable to simple actions when the objective is to allow a Kibana session to a user.


here is what i did:

the original acl was (that didn’t work) :

 - name: "user1 indices d*"
   verbosity: erro
   type: allow
   name: "ldap1"
   groups: ["A456"]
  indices: ["d*",".kibana"]
  actions: [“indices:admin/*”,“indices:data/write/\*”,“indices/data/read/\*”,“cluster:monitor/\*”,“indices:monitor/\*”]
  kibana_hide_apps: [“readonlyrest_kbn”,“timelion”]

so i tried :

  1. add the macro kibana_access: rw
    restart all and still didn’t work (couldn’ connect to kibana)

  2. changed the macro to : kibana_access: admin
    restart all and still didn’t work (couldn’ connect to kibana)

  3. leave the kibana_access: admin and remove the “actions” line.
    the user couldn’t create an index (using the curl command)

  4. finally, i split the block to 2 (each block handle one index) :

   - name: "user1 indices d*"
     verbosity: error
     type: allow
       name: "ldap1"
       groups: ["A456"]
     indices: ["d*"]
     actions: [“indices:admin/\*”,“indices:data/write/\*”,“indices/data/read/\*”,“cluster:monitor/\*”,“indices:monitor/*”]
     kibana_hide_apps: [“readonlyrest_kbn”,“timelion”]

  - name: "user1 indices d* + .kibana"
    verbosity: error
    type: allow
       name: "ldap1"
       groups: ["A456"]
    indices: [".kibana*"]
    kibana_access: rw
    kibana_hide_apps: [“readonlyrest_kbn”,“timelion”]	  

it works. the user can logon to kiban, create visualize/dashboards and create indices.

is this the correct way (2 blocks) ?
if not, how the acl block should look like ?
the user need to read, write, admin, monitor it’s own indeices (starts with d*)
and also connect to kibana to work with the “dev tools” and create some visualize + dashboard

(Simone Scarduzio) #14

Hi @sdba2, glad it work, I have some questions though:

  1. Why the backslash?
  1. Why is it important for you to allow this user to write inside the “d*” indices? Under normal circumstances, we want to prevent Kibana sessions from modifying the data indices.

In fact, normally it’s preferable to create two separated identities for data writers and Kibana users to keep Kibana users from accidentally modifying the data (i.e. going wild with dev tools, etc.).

Finally, last comment is that yes, it’s quite common to split two blocks to express boolean OR logic. Remember: ACL blocks are like OR statements, as either of them can match; instead rules inside an ACL block are evaluated in AND logic, that is: all of them must match in order for the whole block to match.

  1. the backslash was to show the * in the display(
    without it there was no . in the real file there aren’t any “” only “/”, like: indices:admin/

  2. the users using “dev tools” to sometimes index new data.

i hope you could help me with

  1. why the same readonlyrest.yml was working fine the the 21 version and stopped with the 27?
  2. with my requierments how a good acl block should look like ?

(Simone Scarduzio) #16

OK about the forum not displaying the star symbol, that’s because you have to mark things as code! Use the editor, there is a “</>” button.

x*single line code*


About the user indexing via dev tools, fine for me. In that case having the two blocks in OR logic is perfectly fine.

The previous version of ROR was working because in the new version the login attempts from Kibana go to a dedicated endpoint ( /_readonlyrest/metadata/current_user ), while previously we used a simple get to a generic /_nodes/_local. The new endpoint correspond to a different ‘action’ that was not permitted by your custom list of allowed actions. If you had used kibana_access macro, you would have not even realised, as such a request would have been allowed from within the macro

Your ACL blocks look fine, be always mindful of the order you write them, because that would be also the order they are evaluated as. And also, never mix action and kibana_access rules as the former nullifies the latter.

Further reading: