Advanced Index filtering and index pattern

#1

Hello,

Readonlyrest enterprise user here.

I discovered a strange behavior around indice rules and Index patterns in Kibana, which seems to be unexpected.

The use case :

  • A cluster with indices: [ “hello_user1.first”, “hello_user1.second”, “hello_user3.first”, “goodbye_user1.first”,“goodbye_user2.first”,“goodbye_user3.first”]
  • A user with indice rule which allows the following indices : “*.first”
  • An index pattern in kibana : “hello_user*”

Observed : logged as the user, the index pattern returns no data
What I Expected : The index pattern returns data from hello_user1.first.

From my investigation, this is related to the ZeroKnowledgeIndexFilter.java (more specifically the algorithme between line 82 and 90).

So here comes the questions :

  • Is this behavior expected?
  • If not, is there any filtering performed after data gathering in elasticsearch. In other words, what is the goal of the ZeroKnowledgeIndexFilter? Is it the whole guardian of data, or is it here only for performances?
#2

Hello @pchesneau

Are you able to send us all debug logs related to the request?

#3

Sure! I’ll provide you with this by middle of the week.

1 Like
#4

While digging around to generate logs, I discovered that the behavior which I thought was the same on both my cluster is actually different depending on which ReadOnlyRest version is deployed:

On my “old” cluster : the behavior is as I described previously.
On my new cluster, it is not working, but it may be due to a version mismatch between main cluster and remote clusters (admin/resolve/index is not available on Kibana < 7.9)

Was there any major update around indices rules recently ?, If so, what is the first RoR version to have these improvements?

#5

@pchesneau there were some improvements in indices rule in ROR 1.31.0 (“Enhancement (ES) improved cluster indices handling” Download - ReadonlyREST)

#6

Damned, my old cluster is too old.

Let’s set aside this issue for now. I’ll first upgrade my whole clusters and test everything once upgraded.
60TB cluster, let’s hope nothing goes wrong.

1 Like