I discovered a strange behavior around indice rules and Index patterns in Kibana, which seems to be unexpected.
The use case :
A cluster with indices: [ “hello_user1.first”, “hello_user1.second”, “hello_user3.first”, “goodbye_user1.first”,“goodbye_user2.first”,“goodbye_user3.first”]
A user with indice rule which allows the following indices : “*.first”
An index pattern in kibana : “hello_user*”
Observed : logged as the user, the index pattern returns no data
What I Expected : The index pattern returns data from hello_user1.first.
From my investigation, this is related to the ZeroKnowledgeIndexFilter.java (more specifically the algorithme between line 82 and 90).
So here comes the questions :
Is this behavior expected?
If not, is there any filtering performed after data gathering in elasticsearch. In other words, what is the goal of the ZeroKnowledgeIndexFilter? Is it the whole guardian of data, or is it here only for performances?
While digging around to generate logs, I discovered that the behavior which I thought was the same on both my cluster is actually different depending on which ReadOnlyRest version is deployed:
On my “old” cluster : the behavior is as I described previously.
On my new cluster, it is not working, but it may be due to a version mismatch between main cluster and remote clusters (admin/resolve/index is not available on Kibana < 7.9)
Was there any major update around indices rules recently ?, If so, what is the first RoR version to have these improvements?
Let’s set aside this issue for now. I’ll first upgrade my whole clusters and test everything once upgraded.
60TB cluster, let’s hope nothing goes wrong.