Free ROR 1.24.0 snapshots/get results in 401 authorization error

Attempts to get list of snapshots for repository fail on version 1.24.0 with authorization block defined below. Previous known working version is 1.18.10. Authorization block of person with full admin access can retrieve list of snapshots (i.e. not using any wildcard actions list).

- name: "Test producer"
  type: allow
  ldap_auth:
    name: "ldap2"
    groups: ["ldap_group_a"]
  actions: ["indices:data/*","indices:admin/*","indices:monitor/*","cluster:monitor/*","cluster:admin/snapshot/*"]
  indices: ["infratest*"]
  repositories: ["infratest"]

What ES version do you use? We’ve fixed snapshot API handling in current, not released yet sprint, so I can send you a pre-build to test

We’re using ES 7.9.1. That’d be great, we’d be happy to test fix.

@rodaj please check this one:

https://readonlyrest-data.s3.amazonaws.com/build/1.25.0-pre9/readonlyrest-1.25.0-pre9_es7.9.1.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA5SJIWBO54AGBERLX/20201119/eu-west-1/s3/aws4_request&X-Amz-Date=20201119T143828Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=49dc74f7a8084a56cb0d12308800b051beb6364371a857755aadafe157552a40

patch downloaded, deployment to start shortly, should be able to post results by end of day.

1 Like

No Joy.
Ansible snipet to drive test:

  • name: Get snapshots (success)
    uri:
    url: https://{{ cluster_url }}:{{ cluster_port }}/_snapshot/{{ test_repository }}/_all
    method: GET
    user: “{{ infra_producer_id }}”
    password: “{{ infra_producer_pwd }}”
    force_basic_auth: yes
    validate_certs: no

ROR authorization block using specifically defined actions:
- name: “Test producer”
type: allow
ldap_auth:
name: “ldap2”
groups: [“es_producer”]
actions: [“indices:data/","indices:admin/”,“indices:admin/aliases”,“indices:admin/aliases/exists”,“indices:admin/aliases/get”,“indices:monitor/","cluster:monitor/”,“cluster:admin/snapshot/create”,“cluster:admin/snapshot/delete”,“cluster:admin/snapshot/get”,“cluster:admin/snapshot/restore”,“cluster:admin/snapshot/status”]
indices: [“infratest*”]
repositories: [“infratest”]

Test response:
TASK [Get snapshots (success)] *************************************************************************************************************************************************************************************
fatal: [tester.]: FAILED! => {“changed”: false, “connection”: “close”, “content”: “{“error”:{“root_cause”:[{“reason”:“Forbidden by ReadonlyREST ES plugin”,“due_to”:[“OPERATION_NOT_ALLOWED”]}],“reason”:“Forbidden by ReadonlyREST ES plugin”,“due_to”:[“OPERATION_NOT_ALLOWED”],“status”:401}}”, “content_length”: “205”, “content_type”: “application/json; charset=UTF-8”, “elapsed”: 0, “json”: {“error”: {“due_to”: [“OPERATION_NOT_ALLOWED”], “reason”: “Forbidden by ReadonlyREST ES plugin”, “root_cause”: [{“due_to”: [“OPERATION_NOT_ALLOWED”], “reason”: “Forbidden by ReadonlyREST ES plugin”}], “status”: 401}}, “msg”: “Status code was 401 and not [200]: HTTP Error 401: Unauthorized”, “redirected”: false, “status”: 401, “strict_transport_security”: “max-age=16000000”, “url”: “https://:/_snapshot/infratest/_all”, “www_authenticate”: “Basic”}

ROR authorization block using wildcard actions:
- name: “Test producer”
type: allow
ldap_auth:
name: “ldap2”
groups: [“es_producer”]
actions: [“indices:data/","indices:admin/”,“indices:monitor/","cluster:monitor/”,“cluster:admin/snapshot/"]
indices: ["infratest
”]
repositories: [“infratest”]
Test response:
fatal: [tester.]: FAILED! => {“changed”: false, “connection”: “close”, “content”: “{“error”:{“root_cause”:[{“reason”:“Forbidden by ReadonlyREST ES plugin”,“due_to”:[“OPERATION_NOT_ALLOWED”]}],“reason”:“Forbidden by ReadonlyREST ES plugin”,“due_to”:[“OPERATION_NOT_ALLOWED”],“status”:401}}”, “content_length”: “205”, “content_type”: “application/json; charset=UTF-8”, “elapsed”: 0, “json”: {“error”: {“due_to”: [“OPERATION_NOT_ALLOWED”], “reason”: “Forbidden by ReadonlyREST ES plugin”, “root_cause”: [{“due_to”: [“OPERATION_NOT_ALLOWED”], “reason”: “Forbidden by ReadonlyREST ES plugin”}], “status”: 401}}, “msg”: “Status code was 401 and not [200]: HTTP Error 401: Unauthorized”, “redirected”: false, “status”: 401, “strict_transport_security”: “max-age=16000000”, “url”: “https://:/_snapshot/infratest/_all”, “www_authenticate”: “Basic”}

are you able to show us “FORBIDDEN” log produced by ROR?

Here is couple of log entries with ip/host/port/userid redacted so it loses some of its “value”.

[2020-11-19T13:11:19,466][DEBUG][t.b.r.a.b.Block ] []^[[33m[Audit consumer] the request matches no rules in this block: { ID:1354758091-4610850#133421, TYP:GetRepositoriesRequest, CGR:N/A, USR: (attempted), BRS:true, KDX:null, ACT:cluster:admin/repository/get, OA:/32, XFF:null, DA:/32, IDX:<N/A>, MET:GET, PTH:/_snapshot, CNT:<N/A>, HDR:Accept-Encoding=identity, Authorization=, Connection=close, Host=:, User-Agent=ansible-httpget, content-length=0, HIS:[Audit consumer-> RULES:[ldap_auth->false], RESOLVED:[repositories=_all]] } ^[[0m
[2020-11-19T13:11:19,466][INFO ][t.b.r.a.l.AccessControlLoggingDecorator] []^[[35mFORBIDDEN by default req={ ID:1354758091-4610850#133421, TYP:GetRepositoriesRequest, CGR:N/A, USR: (attempted), BRS:true, KDX:null, ACT:cluster:admin/repository/get, OA:/32, XFF:null, DA:/32, IDX:<N/A>, MET:GET, PTH:/_snapshot, CNT:<N/A>, HDR:Accept-Encoding=identity, Authorization=, Connection=close, Host=:, User-Agent=ansible-httpget, content-length=0, HIS:[DataDog-> RULES:[groups->false], RESOLVED:[repositories=_all]], [cao_etm-> RULES:[ldap_auth->false], RESOLVED:[repositories=_all]], [cognet_consumer-> RULES:[ldap_auth->false], RESOLVED:[repositories=_all]], [cognet_restore_prod_only-> RULES:[ldap_auth->false], RESOLVED:[repositories=_all]], [CEDP_Support-> RULES:[ldap_auth->false], RESOLVED:[repositories=_all]], [CEDP_Admin-> RULES:[ldap_auth->false], RESOLVED:[repositories=_all]], [ICI Producer-> RULES:[ldap_auth->false], RESOLVED:[repositories=_all]], [ICI Consumer-> RULES:[ldap_auth->false], RESOLVED:[repositories=_all]], [Core Ingest-> RULES:[ldap_auth->false], RESOLVED:[repositories=_all]], [Core readers-> RULES:[ldap_auth->false], RESOLVED:[repositories=_all]], [Test producer-> RULES:[ldap_auth->true, repositories->true, actions->false], RESOLVED:[user=;group=es_producer;av_groups=es_producer;repositories=infratest]], [Test consumer-> RULES:[ldap_auth->false], RESOLVED:[repositories=_all]], [Audit producer-> RULES:[ldap_auth->true, actions->false], RESOLVED:[user=;group=es_producer;av_groups=es_producer;repositories=_all]], [Audit consumer-> RULES:[ldap_auth->false], RESOLVED:[repositories=_all]] }^[[0m

Function should match on this block, as the request specifically provide repository name so problem seems to be ROR thinking repository is _all, when in fact it should be for repository “infratest” only:

[Test producer-> RULES:[ldap_auth->true, repositories->true, actions->false], RESOLVED:[user=;group=es_producer;av_groups=es_producer;repositories=infratest]]

@roday if the “Test producer” should be matched, let’s analyse it:

[Test producer-> RULES:[ldap_auth->true, repositories->true, actions->false], RESOLVED:[user=;group=es_producer;av_groups=es_producer;repositories=infratest]]

rules:

  • ldap_auth - was matched (user has been authenticated)
  • repositories - was matched (the rule narrowed request for _all repositories, to “infratest” repository, because according to this rule, the authenticated user has access only to this one)
  • actions - this was MISMATCHED - as you can see action of this request is cluster:admin/repository/get (see log ACT:cluster:admin/repository/get), but your rule allows only ["indices:data/*","indices:admin/*","indices:monitor/*","cluster:monitor/*","cluster:admin/snapshot/*"]

So, IMO everything works fine, but you have to add cluster:admin/repository/* to actions array.

As the rest api requests was _snapshots there is nothing obvious about “repository” rule. That is going to be a source of confusion forever. I am off this week, so I won’t be able to investigate and determine exactly the rules we’ll want with cluster:admin/repository to retain our desired security until next week. Thanks for the help.

yeah, but AFAIK /_snapshot REST endpoint fetches all snapshot repositories, not all snapshots (see: https://www.elastic.co/guide/en/elasticsearch/reference/7.10/snapshots-register-repository.html#snapshots-repository-plugins). To get all snapshots you should use this one: https://www.elastic.co/guide/en/elasticsearch/reference/current/get-snapshot-api.html