Index restrictions do not work in ACL

Enterprise Support
1

Hi
ROR 1.69.1_es8.19.15 :unicorn:
elasticsearch 8.19.15

config example

- name: “user”
indices: [“*kibana*”, “*:*-index*”, …. very long list]
kibana_access: rw
ldap_authentication:
name: “ldap”
ldap_authorization:
name: “ldap”
groups: [“group”]
kibana_hide_apps: [ “Machine Learning”, “Management”, “ROR Manage Kibana”, “Observability”, “Elasticsearch”, “Stack Management”]

All indexes added to the list in the old version ROR work.
But adding new indexes to the list doesn’t work.
Even if I create a new acl with a small list.

for example:

indices: [“*:asdasd”] - work
indices: [“*:*asdasd”] - not work
indices: [“*:asdasd*”] - not work
indices: [“*:*asdasd*”] - not work

I check this way
host/*:*asdasd/_search

I expect all these exceptions to work as in previous versions of pop,
Changes to the .readonlyrest index are present
ror log

user: NOT_MATCHED (IDX_NOT_FOUND) → RULES:[ldap_authentication->true, ldap_authorization->true, kibana_hide_apps->true, kibana_access->true, indices->false]]

{“customer_id”: “6c4a385b-2ae8-4f02-a9cd-ef24addfb5b3”, “subscription_id”: “32d4073f-dc2f-4056-a868-842727c637cd”}

2

Hi,

Thanks for the report. We will check this

3

@driveirk

I’ve tried to reproduce your use case in our ROR Sandbox: [RORDEV_2109] reproduction by coutoPL · Pull Request #104 · beshu-tech/ror-sandbox · GitHub

I cannot confirm the issue.
The ROR ACL looks like this: ror-sandbox/ror-demo-cluster/conf/es/readonlyrest.yml at c578b9b789bbd38ccc6a70dd53feaa7882414757 · beshu-tech/ror-sandbox · GitHub

I’ve tested it like this:

> curl -ks -u user:test "https://127.0.0.1:19200/*:*asdasd/_search" -H "test:1"
{"took":15,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"_clusters":{"total":1,"successful":1,"skipped":0,"running":0,"partial":0,"failed":0,"details":{"remote1":{"status":"successful","indices":"asdasd","took":10,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0}}}},"hits":{"total":{"value":1,"relation":"eq"},"max_score":1.0,"hits":[{"_index":"remote1:asdasd","_id":"q115Np8BCpKND3ygIhSw","_score":1.0,"_source":{"hello":"world"}}]}}⏎

> curl -ks -u user:test "https://127.0.0.1:19200/*:*asdasd/_search" -H "test:2"
{"took":5,"timed_out":false,"_shards":{"total":2,"successful":2,"skipped":0,"failed":0},"_clusters":{"total":1,"successful":1,"skipped":0,"running":0,"partial":0,"failed":0,"details":{"remote1":{"status":"successful","indices":"asdasd,xasdasd","took":2,"timed_out":false,"_shards":{"total":2,"successful":2,"skipped":0,"failed":0}}}},"hits":{"total":{"value":2,"relation":"eq"},"max_score":1.0,"hits":[{"_index":"remote1:asdasd","_id":"q115Np8BCpKND3ygIhSw","_score":1.0,"_source":{"hello":"world"}},{"_index":"remote1:xasdasd","_id":"rF15Np8BCpKND3ygIxTo","_score":1.0,"_source":{"hello":"world"}}]}}⏎

> curl -ks -u user:test "https://127.0.0.1:19200/*:*asdasd/_search" -H "test:3"
{"took":4,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"_clusters":{"total":1,"successful":1,"skipped":0,"running":0,"partial":0,"failed":0,"details":{"remote1":{"status":"successful","indices":"asdasd","took":1,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0}}}},"hits":{"total":{"value":1,"relation":"eq"},"max_score":1.0,"hits":[{"_index":"remote1:asdasd","_id":"q115Np8BCpKND3ygIhSw","_score":1.0,"_source":{"hello":"world"}}]}}⏎

> curl -ks -u user:test "https://127.0.0.1:19200/*:*asdasd/_search" -H "test:4"
{"took":5,"timed_out":false,"_shards":{"total":2,"successful":2,"skipped":0,"failed":0},"_clusters":{"total":1,"successful":1,"skipped":0,"running":0,"partial":0,"failed":0,"details":{"remote1":{"status":"successful","indices":"asdasd,xasdasd","took":3,"timed_out":false,"_shards":{"total":2,"successful":2,"skipped":0,"failed":0}}}},"hits":{"total":{"value":2,"relation":"eq"},"max_score":1.0,"hits":[{"_index":"remote1:asdasd","_id":"q115Np8BCpKND3ygIhSw","_score":1.0,"_source":{"hello":"world"}},{"_index":"remote1:xasdasd","_id":"rF15Np8BCpKND3ygIxTo","_score":1.0,"_source":{"hello":"world"}}]}}⏎

Logs look like this:

es-ror-1  | {"@timestamp":"2026-07-06T08:09:22.219Z","log.level": "INFO","message":"[184e3a82-ce1e-4b9b-8f0d-41f1bdd7f880-74040044#4596] \u001B[36mALLOWED by { name: 'Test1', policy: ALLOW, rules: [auth_key, headers_and, indices] } req={ ID:184e3a82-ce1e-4b9b-8f0d-41f1bdd7f880-74040044#4596, TYP:SearchRequest, CGR:<N/A>, USR:user, BRS:true, ACT:indices:data/read/search, OA:192.168.65.1/32, XFF:null, DA:172.19.0.2/32, IDX:*:*asdasd, MET:GET, PTH:/*:*asdasd/_search, CNT:<N/A>, HDR:test=1, User-Agent=curl/8.7.1, Authorization=<OMITTED>, Accept=*/*, Host=127.0.0.1:19200, content-length=0, HIS:[KIBANA: NOT_MATCHED (AUTH_FAIL (Username mismatch)) -> RULES:[auth_key->false]], [Test1: MATCHED -> RULES:[auth_key->true, headers_and->true, indices->true] RESOLVED:[user=user;indices=remote1:asdasd]], }\u001B[0m", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"scala-execution-context-global-111","log.logger":"tech.beshu.ror.accesscontrol.logging.AccessControlListLoggingDecorator","elasticsearch.cluster.uuid":"Srimbv2MSsSZzyVbq4cjsA","elasticsearch.node.id":"LMrbntr-R9anGzvmaxpuJQ","elasticsearch.node.name":"es-ror-single","elasticsearch.cluster.name":"${sys:es.logs.cluster_name}"}
es-ror-1  | {"@timestamp":"2026-07-06T08:09:23.763Z","log.level": "INFO","message":"[0f0fbb11-7383-4a51-a2b1-0e5be23e834c-2001414584#4620] \u001B[36mALLOWED by { name: 'Test2', policy: ALLOW, rules: [auth_key, headers_and, indices] } req={ ID:0f0fbb11-7383-4a51-a2b1-0e5be23e834c-2001414584#4620, TYP:SearchRequest, CGR:<N/A>, USR:user, BRS:true, ACT:indices:data/read/search, OA:192.168.65.1/32, XFF:null, DA:172.19.0.2/32, IDX:*:*asdasd, MET:GET, PTH:/*:*asdasd/_search, CNT:<N/A>, HDR:test=2, User-Agent=curl/8.7.1, Authorization=<OMITTED>, Accept=*/*, Host=127.0.0.1:19200, content-length=0, HIS:[KIBANA: NOT_MATCHED (AUTH_FAIL (Username mismatch)) -> RULES:[auth_key->false]], [Test1: NOT_MATCHED (AUTHZ_FAIL) -> RULES:[auth_key->true, headers_and->false]], [Test2: MATCHED -> RULES:[auth_key->true, headers_and->true, indices->true] RESOLVED:[user=user;indices=remote1:xasdasd, remote1:asdasd]], }\u001B[0m", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"scala-execution-context-global-118","log.logger":"tech.beshu.ror.accesscontrol.logging.AccessControlListLoggingDecorator","elasticsearch.cluster.uuid":"Srimbv2MSsSZzyVbq4cjsA","elasticsearch.node.id":"LMrbntr-R9anGzvmaxpuJQ","elasticsearch.node.name":"es-ror-single","elasticsearch.cluster.name":"${sys:es.logs.cluster_name}"}
es-ror-1  | {"@timestamp":"2026-07-06T08:09:25.615Z","log.level": "INFO","message":"[5b910654-26ea-489c-854c-d00ffe78c2f7-773415090#4630] \u001B[36mALLOWED by { name: 'Test3', policy: ALLOW, rules: [auth_key, headers_and, indices] } req={ ID:5b910654-26ea-489c-854c-d00ffe78c2f7-773415090#4630, TYP:SearchRequest, CGR:<N/A>, USR:user, BRS:true, ACT:indices:data/read/search, OA:192.168.65.1/32, XFF:null, DA:172.19.0.2/32, IDX:*:*asdasd, MET:GET, PTH:/*:*asdasd/_search, CNT:<N/A>, HDR:test=3, User-Agent=curl/8.7.1, Authorization=<OMITTED>, Accept=*/*, Host=127.0.0.1:19200, content-length=0, HIS:[KIBANA: NOT_MATCHED (AUTH_FAIL (Username mismatch)) -> RULES:[auth_key->false]], [Test1: NOT_MATCHED (AUTHZ_FAIL) -> RULES:[auth_key->true, headers_and->false]], [Test2: NOT_MATCHED (AUTHZ_FAIL) -> RULES:[auth_key->true, headers_and->false]], [Test3: MATCHED -> RULES:[auth_key->true, headers_and->true, indices->true] RESOLVED:[user=user;indices=remote1:asdasd]], }\u001B[0m", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"scala-execution-context-global-112","log.logger":"tech.beshu.ror.accesscontrol.logging.AccessControlListLoggingDecorator","elasticsearch.cluster.uuid":"Srimbv2MSsSZzyVbq4cjsA","elasticsearch.node.id":"LMrbntr-R9anGzvmaxpuJQ","elasticsearch.node.name":"es-ror-single","elasticsearch.cluster.name":"${sys:es.logs.cluster_name}"}
es-ror-1  | {"@timestamp":"2026-07-06T08:09:27.052Z","log.level": "INFO","message":"[8f877a46-4df0-403f-bed9-23177502a411-1851082387#4644] \u001B[36mALLOWED by { name: 'Test4', policy: ALLOW, rules: [auth_key, headers_and, indices] } req={ ID:8f877a46-4df0-403f-bed9-23177502a411-1851082387#4644, TYP:SearchRequest, CGR:<N/A>, USR:user, BRS:true, ACT:indices:data/read/search, OA:192.168.65.1/32, XFF:null, DA:172.19.0.2/32, IDX:*:*asdasd, MET:GET, PTH:/*:*asdasd/_search, CNT:<N/A>, HDR:test=4, User-Agent=curl/8.7.1, Authorization=<OMITTED>, Accept=*/*, Host=127.0.0.1:19200, content-length=0, HIS:[KIBANA: NOT_MATCHED (AUTH_FAIL (Username mismatch)) -> RULES:[auth_key->false]], [Test1: NOT_MATCHED (AUTHZ_FAIL) -> RULES:[auth_key->true, headers_and->false]], [Test2: NOT_MATCHED (AUTHZ_FAIL) -> RULES:[auth_key->true, headers_and->false]], [Test3: NOT_MATCHED (AUTHZ_FAIL) -> RULES:[auth_key->true, headers_and->false]], [Test4: MATCHED -> RULES:[auth_key->true, headers_and->true, indices->true] RESOLVED:[user=user;indices=remote1:xasdasd, remote1:asdasd]], }\u001B[0m", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"scala-execution-context-global-111","log.logger":"tech.beshu.ror.accesscontrol.logging.AccessControlListLoggingDecorator","elasticsearch.cluster.uuid":"Srimbv2MSsSZzyVbq4cjsA","elasticsearch.node.id":"LMrbntr-R9anGzvmaxpuJQ","elasticsearch.node.name":"es-ror-single","elasticsearch.cluster.name":"${sys:es.logs.cluster_name}"}

When I don’t configure the remote cluster, I get empty results and “IDX_NOT_FOUND” in logs for these queries - which is correct behaviour.

Could you please double-check it on your side?
Are you able to show us the full log, not only part of it?

4

Using a local user doesn’t work for me either. Use an LDAP user.

    - name: "Test2"
      type: allow
      users: ["user"]
      ldap_authentication:
        name: "ldap"
      indices: ["*:*asdasd"]

I’ll send the full log a little later.

5

I will attach the full log as a file so that it is not indexed.
access.log (2.0 KB)

request
get host/*asdasd/_search?pretty

answer:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "forbidden_response",
        "reason" : "There is no access to view these logs from your country. To get access, you need to go to jira => create a ticket => project (ELK) => type (Whitelist non PII index pattern)",
        "due_to" : "FORBIDDEN_BY_BLOCK"
      }
    ],
    "type" : "forbidden_response",
    "reason" : "There is no access to view these logs from your country. To get access, you need to go to jira => create a ticket => project (ELK) => type (Whitelist non PII index pattern)",
    "due_to" : "FORBIDDEN_BY_BLOCK"
  },
  "status" : 403
}

acl:
acl.txt (560 Bytes)

6

Usually there are some errors nearby, but here I didn’t find anything in the logs.

7

Could you please tell me what version of ROR you have in mind? I want to compare 1.69.1 with it.

8

the problems started in
ROR 1.69.1_es8.19.15 :unicorn:
elasticsearch 8.19.15

Before this, the version was ROR 1.68.0 elasticsearch 8.18.3 and there was no problem there.
The rules added in this version still work.
Even if I move them to a separate ACL.
I haven’t tried deleting them, because if they’re deleted and can’t be added, I’ll have problems.

9

It’s working as designed (and has always worked this way), but we agree the behavior can be confusing, so let me walk through it.

When your indices rule uses a wildcard pattern, ReadonlyREST only matches the block if the wildcard actually resolves to at least one real, existing index in the cluster. If no matching index exists, the block is treated as “not matched” (because of IDX_NOT_FOUND) and ROR moves on to the next block. In your ACL, the next block is a forbid block, so that one matches instead, and you get the “forbidden” response.

So the block isn’t failing because your pattern is wrong - it’s failing because there’s no such index in the cluster at that moment.

Example:

Using a rule like indices: ["*asdasd"]

  • Request GET /*asdasd/_search when an index such as asdasd exists → the block matches → you get your data.
  • Request GET /*asdasd/_search when no matching index exists → the block is skipped → the following forbid block matches → you get 403 forbidden.

Same request, same user, the only difference is whether a matching index currently exists.

This usually shows up when you add a new index pattern to the indices rule for an index that hasn’t been created yet.
Until at least one matching index exists, wildcard requests for it will fall through to your forbid block.

This is not a change or a bug in the recent version - it has behaved this way in older versions too (including 1.68.0).

The reason you see a “forbidden” message (rather than a plain “index not found”) is the catch-all forbid block right after your allow block. If that forbid block weren’t there, the same request would return a normal index_not_found (404) instead.

We agree that getting a “forbidden” result for an index you’re actually allowed to access (it just doesn’t exist yet) is not intuitive. We’re going to look into whether we can improve this behavior.

I haven’t tried deleting them, because if they’re deleted and can’t be added, I’ll have problems.

Don’t worry - editing the indices rule (or the list of patterns inside it) is only a configuration change, and ROR rereads your ACL on every request. Nothing gets “stuck”:

One thing to keep in mind, so the earlier confusion doesn’t come back: adding a pattern for an index that does not exist yet won’t make a wildcard search succeed until at least one matching index is created. That’s the behavior described above - it’s about the index existing, not about your config change. For indices that already exist, adding/removing their patterns takes effect immediately and is completely reversible

10

Indeed, you are right. Thanks for the explanation.

1 Like