RoR 1.37.0 indices rule and alias within kibana

Hi,

Elastic and Kibana 7.16.2
RoR Enterprise 1.37.0

We are trying to reorganize and futuru proof our indices layout.
We plannend on using aliases for indices for this.
But I can’t seem to get Kibana to allow access to the alias of a index.

Setup:

  1. Create a data stream
POST logs-random-indexname/_doc
{
  "message": "logged the request",
  "@timestamp": "1591890612"
}

Please note, this example creates a document in 1970 :slight_smile:

  1. Apply an alias
POST _aliases
{
  "actions": [
		{
		  "add": {
			"index": "logs-random-indexname",
			"alias": "roa-logs-random-indexname"
		  }
		}
	]
}
  1. Give permissions via RoR rules:
  - name: "custom kibana ro"
    indices: ["kibana_custom"]
    kibana_access: rw
    kibana_index: "kibana_custom"
    kibana_hide_apps: ["Analytics|Maps", "Analytics|Overview", "Observability", "Security", "readonlyrest_kbn", "Analytics|Canvas", "Management", "Enterprise Search"]
    groups: ["custom_ro"]
    
  - name: "Custom data access rule"
    indices: ["roa-logs-random-*"]
    actions: ["indices:data/read/*", "indices:admin/resolve/index"]
    groups: ["custom_admin", "custom_ro"] 
    
  - name: "custom kibana admin"
    indices: ["kibana_custom"]
    kibana_index: "kibana_custom"
    groups: ["custom_admin"]

  users:
  - username: myloginname
    auth_key_sha256: my_hash
    groups: ["custom_admin"]
  1. Login with user myloginname and open discover for the index pattern:
roa-logs-random-*
  1. No data found :frowning:

I tried several configurations:

    indices: ["roa-logs-random-indexname"]
    indices: ["roa-logs-random-indexname", "logs-random-indexname"]
    indices: ["roa-*", "logs-random-*"]
    indices: ["roa-logs-random-*", "logs-random-indexname"]
    indices: ["roa-logs-random-indexname", "logs-random-indexname"]
    indices: ["roa-*", "random-*"]

Result is always index not found.

I also tried from dev tools doing the query directly

GET /roa-logs-random-indexname/_search

This works.

If you try

GET /roa-logs-random-*/_search

It doesn’t work,
You get

{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 0,
    "successful" : 0,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 0,
      "relation" : "eq"
    },
    "max_score" : 0.0,
    "hits" : [ ]
  }
}

If you make a index pattern for:

roa-logs-random-indexname

It works.

So my initial conclusion was, somehow it is not resolving the data stream alias correctly…
But, if i configure the indices rule as:

    indices: ["roa-logs-random-indexname", "*"]

It suddenly works for index pattern roa-logs-random-* as well.

End conclusion:
Wildcard resolving of data stream aliases in the indices section seems somehow broken?

(Topic was adjusted after more troubleshooting)

Ow to complete the information, the rule resolvement shows something like this:
[Custom data access rule-> RULES:[groups->true
actions->true
indices->false] RESOLVED:[user=myloginname;group=custom_admin;av_groups=custom_admin;indices=roa-logs-random-*]]

In the first place, I’d advise enabling debug logs and tracing the request. Indices rule logic is a bit complicated, but there is much information at debug level in logs which should help to figure out how indices and aliases are being resolved.

1 Like

Oke, we will try enabling debug tomorrow on a test environment and recreate the situation.

Made some adjust to the original post.
I thought it applied for indices and data streams, but it seems it only applies for data streams.
We will check for logs and share those as well.

yeah, maybe a data stream is not properly handled. Waiting for the logs.

Hi @coutoPL, I tried lots of things to try to get more logs than this:
For working index pattern:

{
    "type": "server",
    "timestamp": "2022-01-31T17:14:14,823+01:00",
    "level": "INFO",
    "component": "t.b.r.a.l.AccessControlLoggingDecorator",
    "cluster.name": "clustername",
    "node.name": "puppet-master",
    "message": "\u001b[36mALLOWED by { name: 'Custom data access rule', policy: ALLOW, rules: [groups,actions,indices] req={ ID:496503698--1988695060#12920, TYP:SubmitAsyncSearchRequest, CGR:custom_admin, USR:myloginname, BRS:true, KDX:null, ACT:indices:data/read/async_search/submit, OA:172.28.128.10/32, XFF:172.28.128.10:8001, DA:172.28.128.10/32, IDX:roa-logs-random-indexname, MET:POST, PTH:/roa-logs-random-indexname/_async_search, CNT:<OMITTED, LENGTH=466.0 B> , HDR:Accept-Charset=utf-8, Authorization=<OMITTED>, Host=172.28.128.10:9200, connection=close, content-length=466, content-type=application/json, user-agent=elasticsearch-js/7.16.0-canary.7 (linux 3.10.0-957.12.2.el7.x86_64-x64; Node.js v16.13.0), x-elastic-client-meta=es=7.16.0p,js=16.13.0,t=7.16.0p,hc=16.13.0, x-elastic-product-origin=kibana, x-forwarded-for=172.28.128.10:8001, x-opaque-id=5776e2b6-d357-4a4c-8768-d03e71492a19, x-ror-current-group=custom_admin, x-ror-kibana-request-method=post, x-ror-kibana-request-path=/internal/bsearch, HIS:[testuser-> RULES:[auth_key_sha256->false] RESOLVED:[group=custom_admin;indices=roa-logs-random-indexname]], [Kibana user-> RULES:[auth_key_sha256->false] RESOLVED:[group=custom_admin;indices=roa-logs-random-indexname]], [Curator user-> RULES:[auth_key_sha256->false] RESOLVED:[group=custom_admin;indices=roa-logs-random-indexname]], [username user-> RULES:[auth_key_sha256->false] RESOLVED:[group=custom_admin;indices=roa-logs-random-indexname]], [servername logstash user1-> RULES:[auth_key_sha256->false] RESOLVED:[group=custom_admin;indices=roa-logs-random-indexname]], [servername logstash user2-> RULES:[auth_key_sha256->false] RESOLVED:[group=custom_admin;indices=roa-logs-random-indexname]], [Elasticsearch machines are allowed to monitor-> RULES:[hosts->true, actions->false] RESOLVED:[group=custom_admin;indices=roa-logs-random-indexname]], [custom kibana ro-> RULES:[groups->false] RESOLVED:[group=custom_admin;indices=roa-logs-random-indexname]], [Custom data access rule-> RULES:[groups->true, actions->true, indices->true] RESOLVED:[user=myloginname;group=custom_admin;av_groups=custom_admin;indices=roa-logs-random-indexname]], }\u001b[0m",
    "cluster.uuid": "Y_S7W6KkRs-qU5njZV9kXw",
    "node.id": "oOWS-6L0RLujHWlrl0mqtA"
}
{
    "type": "server",
    "timestamp": "2022-01-31T17:54:56,689+01:00",
    "level": "INFO",
    "component": "t.b.r.a.l.AccessControlLoggingDecorator",
    "cluster.name": "clustername",
    "node.name": "puppet-master",
    "message": "\u001b[35mINDEX NOT FOUND req={ ID:1774894255-1721851344#63457, TYP:SubmitAsyncSearchRequest, CGR:custom_admin, USR:myloginname (attempted), BRS:true, KDX:null, ACT:indices:data/read/async_search/submit, OA:172.28.128.10/32, XFF:172.28.128.10:8001, DA:172.28.128.10/32, IDX:roa-*, MET:POST, PTH:/roa-*/_async_search, CNT:<OMITTED, LENGTH=406.0 B> , HDR:Accept-Charset=utf-8, Authorization=<OMITTED>, Host=172.28.128.10:9200, connection=close, content-length=406, content-type=application/json, user-agent=elasticsearch-js/7.16.0-canary.7 (linux 3.10.0-957.12.2.el7.x86_64-x64; Node.js v16.13.0), x-elastic-client-meta=es=7.16.0p,js=16.13.0,t=7.16.0p,hc=16.13.0, x-elastic-product-origin=kibana, x-forwarded-for=172.28.128.10:8001, x-opaque-id=a60afc8a-2d42-4854-b297-8336e78115a9, x-ror-current-group=custom_admin, x-ror-kibana-request-method=post, x-ror-kibana-request-path=/internal/bsearch, HIS:[testuser-> RULES:[auth_key_sha256->false] RESOLVED:[group=custom_admin;indices=roa-*]], [Kibana user-> RULES:[auth_key_sha256->false] RESOLVED:[group=custom_admin;indices=roa-*]], [Curator user-> RULES:[auth_key_sha256->false] RESOLVED:[group=custom_admin;indices=roa-*]], [username user-> RULES:[auth_key_sha256->false] RESOLVED:[group=custom_admin;indices=roa-*]], [servername logstash user1-> RULES:[auth_key_sha256->false] RESOLVED:[group=custom_admin;indices=roa-*]], [servername logstash user2-> RULES:[auth_key_sha256->false] RESOLVED:[group=custom_admin;indices=roa-*]], [Elasticsearch machines are allowed to monitor-> RULES:[hosts->true, actions->false] RESOLVED:[group=custom_admin;indices=roa-*]], [custom kibana ro-> RULES:[groups->false] RESOLVED:[group=custom_admin;indices=roa-*]], [Custom data access rule-> RULES:[groups->true, actions->true, indices->false] RESOLVED:[user=myloginname;group=custom_admin;av_groups=custom_admin;indices=roa-*]], [custom kibana admin-> RULES:[groups->true, kibana_index->true, indices->false] RESOLVED:[user=myloginname;group=custom_admin;av_groups=custom_admin;indices=roa-*;kibana_idx=kibana_custom]], }\u001b[0m",
    "cluster.uuid": "Y_S7W6KkRs-qU5njZV9kXw",
    "node.id": "oOWS-6L0RLujHWlrl0mqtA"
}

But I don’t think this isn’t more than before I changed the rootlogger to DEBUG level

Do you have any suggestions to get more logging from the readonlyrest classes?

Thanks & regards,
Arjen

Did you configure it in the way it’s described here?

Hi @coutoPL , not in full detail like the log4j example. I just put the rootLogger to debug, expecting all classes to comply with it. Let me try what you describe and I will get back to you.

Thanks & regards,
Arjen

Hi @coutoPL, I managed to get some more logs which I will send to you in a private message.

Regards, Arjen

1 Like

@abuising @ronald.vanboven guys, please try this one:
https://readonlyrest-data.s3.amazonaws.com/build/1.39.0-pre1/readonlyrest-1.39.0-pre1_es7.16.2.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA5SJIWBO54AGBERLX/20220209/eu-west-1/s3/aws4_request&X-Amz-Date=20220209T120122Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=6b45deb0ff9fd2284bc3f03f0ac088e08d6496cfabdb77a6589bcab98aeec494

1 Like

Hi @coutoPL ,

Thank you very much, but as I tried to install it into my 7.16.3 cluster, which I needed to downgrade to 7.16.2 with this plugin, I got the following error starting up Elasticsearch:

"stacktrace": ["org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: cannot downgrade a node from version [7.16.3] to version [7.16.2]",

Can you provide a 7.16.3 prerelease version of 1.39.0?

Since this is the node on which the setup of the provided logging is located.

(besides that, our production cluster is also upgraded to 7.16.3 in the meantime)

Thanks & regards,
Arjen

sure, no problem :slight_smile:
https://readonlyrest-data.s3.amazonaws.com/build/1.39.0-pre1/readonlyrest-1.39.0-pre1_es7.16.3.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA5SJIWBO54AGBERLX/20220209/eu-west-1/s3/aws4_request&X-Amz-Date=20220209T155423Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=ed6617ed99faedc9bf2fce376d2af0a15319891d2d985c1d6968ea46c8d29576

1 Like

Hi @coutoPL,

It works now!

When can we expect the release for this build?

Thanks & regards,
Arjen

Great. I have to write automated integration tests and port the solution to other ES versions.
We release once a month, so I think the release should be within 2 weeks.

2 Likes

Hi @coutoPL, Thank you very much for the quick solution!

Regards, Arjen

1 Like

Hi, the fix is released with ROR 1.39.0. Sorry for the delay

3 Likes