Unable to filter documents (DLS)

1

Hi

I am trying to filter so that the only records that are returned are those that match a cardid of aa-bb-cc-dd-ee. If i login and go to the discover tab, and enter in the following in the search field, I only get the record i am expecting.
cardid : aa-bb-cc-dd-ee.

This is my config.

- name: "::TESTING::"
  filter: '{"query_string":{"query":"cardid: aa-bb-cc-dd-ee"}}'
  jwt_auth:
    name: "xxx"

Once i save my config, when i go into Kibana, it takes me straight to the management tab with a message “In order to visualize and explore data in Kibana, you’ll need to create an index pattern to retrieve data from Elasticsearch.”
In there i cant see any of my index patterns, and its asking me to create a new index pattern. In step 1 it tells me it can see 13 indices, although i have 100 (and when i removing the filter it does tell me i have 100 indicies).

I am only using a 1 node cluster.

When i try and use the fields filter, that works - so FLS works, but not DLS.

Any help would be grateful. thanks

2

Hi Paul,

Please note that filter and fields rule will not match if the request is anything different than a search/read request.

In order to make a proper Kibana session work with these rules, you have to duplicate the ACL block.

i.e.


- name: "::TESTING-intercept-searches::"
  filter: '{"query_string":{"query":"cardid: aa-bb-cc-dd-ee"}}'
  jwt_auth:
    name: "xxx"

- name: "::TESTING-intercept-all-the-other-requests::"
  jwt_auth:
    name: "xxx"

By doing so, the search requests will match the first, and the non-search requests will end up matching the second.

3

Hi

I have set it to this, and still getting the same results. Is the query case sensitive? I tried both upper and lower case - still nothing.

- name: "::TESTING::"
  kibana_access: rw
  filter: '{"query_string":{"query":"cardid: aa-bb-cc-dd-ee"}}'
  users: ["xxxxxxxxxxxxxxxxxxxx"]
  jwt_auth:
    name: "jwt_provider_2"

- name: "::TESTING1::"
  users: ["xxxxxxxxxxxxxxxxxxxx"]
  jwt_auth:
    name: "jwt_provider_2"

I also add this rule below it, but removing it still nothing,

- name: "::TESTING Part 2::"
  type: forbid
  users: ["xxxxxxxxxxxxxxxxxxxx"]
  jwt_auth:
    name: "jwt_provider_2"
4

This is an example from our integration tests

  - name: getter_a
    api_keys: ["g"]
    indices: ["testfiltera"]
    filter: "{\"bool\": {\"must\": [{\"term\": {\"title\": {\"value\": \"a1\"}}}]}}"
5

Hi

Still no go, here is what i set it to:

- name: "::TESTING::"
  kibana_access: rw
  filter: "{\"bool\": {\"must\": [{\"term\": {\"cardid\": {\"value\": \"aa-bb-cc-dd-ee\"}}}]}}"
  users: ["xxxxxxxxxxxxxxx"]
  jwt_auth:
    name: "jwt_provider_2"
6

Also your doc doesn’t show it like that (with the \ in front of the "), it shows it like this:

filter: '{"query_string":{"query":"user:@{user}"}}'

When i copy that into my config i get an error ‘Received null ROR settings: null’

Even when i change it to cardid:aa-bb-cc-dd-ee i still get the same error.

The doc also shows it like this:

filter: '{"bool": {"must_not": [{"term": {"access_level": {"value": "classified"}}}]}}'

And i tried to adjust it to this; expecting that it would filter out these records - but it did nothing and showed all records including those with cardid aa-bb-cc-dd.

filter: '{"bool": {"must_not": [{"term": {"cardid": {"value": "aa-bb-cc-dd"}}}]}}'

Also, if i go to the Kibana Dev Tools, i can use this query, and it works as expected:

{
    "query": {
        "query_string" : {
            "default_field" : "cardid",
            "query" : "aa-bb-cc-dd"
        }
    }
}

Can you provide an example that will work? As this is getting very frustrating.

7

Hi Paul, will try to reproduce this now.

EDIT Tested and this works:

Given the following YAML settings:

readonlyrest:
  audit_collector: true
  ssl:
    enable: true
    keystore_file: "keystore.jks"
    keystore_pass: readonlyrest
    key_pass: readonlyrest

  enable: true

  access_control_rules:

  - name: canSeeAll
    auth_key: a:a

  - name: filtered
    auth_key: b:b
    filter: '{"bool": { "must_not": { "match": { "final_state": "ALLOWED" }}}}'

The following can be observed:

$ curl -sqk -ua:a  'https://localhost:9200/readonlyrest_audit*/_search?pretty=true' | jq .hits.total
53
$ curl -sqk -ub:b  'https://localhost:9200/readonlyrest_audit*/_search?pretty=true' | jq .hits.total
2

As you can see the first query (with credentials a:a) gets all the documents, while the second query (using the b:b credentials) gets partial content.

8

Hi

Testing this filter, and it works in the discover page:

    - name: filtered
      auth_key: f:f
      filter: '{"bool": { "must": { "match": { "cardid": "aa-bb-cc-dd" }}}}'

Issue is that it causes the visualize or dashboard pages to stop working. When i go to either of these screens, it throws me across to the Management page, and shows me no indexes. Guessing this has something to do with the filter also filtering the .kibana index.

How do we force the filter to exclude the .kibana index? Or if this isn’t possible, can we do a filter that has an OR type=Index-pattern OR type=dashboard OR type=visualization OR type=search OR type=config?

9

Hi Paul, you have to use the composition of ACL blocks. Leverage the fact that ACL blocks are evaluated sequentially.

Look at this example:

  - name: "for this user, kibana index should never be filtered"
    auth_key: b:b
    indices: [".kibana"]
    
  - name: "for this user, read operations on other indices should be filtered"
    auth_key: b:b
    filter: '{"bool": { "must_not": { "match": { "final_state": "ALLOWED" }}}}'

  - name: "for this user, non-read operations should normally be allowed under the kibana_access: rw policy"
    auth_key: b:b
    kibana_access: rw
10

Hello Simone,

This helped me and i was able to configure and get the authentication and authorization working with the filter which i’m passing.

Wanted to know how to include dynamic user for this scenario, I’m working on a POC where I have to restrict user based on tenant and I have multiple tenants and multiple users per tenant and a few users are created post setup of the cluster.

Now for a single user my configuration is as below.

 - name: "RDW_user accessing all the indices"
    auth_key: rdwadmin:rdwadmin
    indices: [".kibana*","log-*","testindex"]

  - name: "Allow access to only rdw tenant logs for RDW_user"
    auth_key: rdwadmin:rdwadmin
    filter: '{"bool": { "must": { "match": { "TenantId": "rdw" }}}}'

  - name: "Allow non-read operations under the kibana_access for RDW_user "
    auth_key: rdwadmin:rdwadmin
    kibana_access: rw

This requires reboot of cluster every time we add a new user. As mentioned earlier some users are created after the environment is setup and ready to use.

Note : I’m still exploring all the options in the open source version of ROR.

11

Be careful that the read request to the “.kibana” index don’t get filtered by the second block, as they would return no documents. I suggest you add a indices rule that does not include “.kibana” index.

About the need to refresh the settings without restarting the whole ES cluster, you have two ways to achieve this.

  1. Using an external connector like LDAP or any identity provider that emits JWT tokens and use dynamic variables in your readonlyrest.yml.

  2. Buy ReadonlyREST PRO, so you can edit settings, press save, and all nodes will reload the settings with no downtime.