ClusterHealthRequest blocked

1

Sorry to be such a nuisance lately. (Posting a lot of questions.)

Can you suggest what might be causing the following in my logs? This occurs when neither Logstash nor Kibana are running (just Elasticsearch, ReadonlyREST, and X-Pack monitoring).

[2017-05-09T10:48:15,768][INFO ][o.e.p.r.a.ACL            ]  no block has matched, forbidding by default: { ID:909308750-1207489306, TYP:ClusterHealthRequest, USR:[no basic auth header], BRS:true, ACT:cluster:monitor/health, OA:127.0.0.1, IDX:logs-2017.01.09, {...trimmed...} logs-2017.01.20, MET:GET, PTH:/_cluster/health, CNT:<OMITTED, LENGTH=0>, HDR:Accept-Encoding,Connection,content-length,Host,User-Agent, HIS:[::RO ACCESS::->[ldap_authentication->false, kibana_access->true, indices->true, verbosity->true]], [::LOCALHOST::->[auth_key->false]], [::KIBANA-SERVER::->[auth_key->false]], [::RW ACCESS::->[ldap_authentication->false, kibana_access->true, indices->true, verbosity->true]], [::LOGSTASH::->[auth_key->false]] }

[2017-05-09T10:48:15,769][INFO ][o.e.p.r.IndexLevelActionFilter] [FBp7aLX] forbidden request: { ID:909308750-1207489306, TYP:ClusterHealthRequest, USR:[no basic auth header], BRS:true, ACT:cluster:monitor/health, OA:127.0.0.1, IDX:logs-2017.01.09, {...trimmed...} logs-2017.01.20, MET:GET, PTH:/_cluster/health, CNT:<OMITTED, LENGTH=0>, HDR:Accept-Encoding,Connection,content-length,Host,User-Agent, HIS:[::RO ACCESS::->[ldap_authentication->false, kibana_access->true, indices->true, verbosity->true]], [::LOCALHOST::->[auth_key->false]], [::KIBANA-SERVER::->[auth_key->false]], [::RW ACCESS::->[ldap_authentication->false, kibana_access->true, indices->true, verbosity->true]], [::LOGSTASH::->[auth_key->false]] } Reason: null (null)

My config is as follows:

readonlyrest:
  enable: true
  response_if_req_forbidden: Forbidden by ReadonlyREST ES plugin

  access_control_rules:
    # allow someone directly logged in to the server to perform any action
    - name: "::LOCALHOST::"
      auth_key: name:password
      hosts: [127.0.0.1]
      type: allow

    # we trust Logstash; access allowed via HTTP auth
    - name: "::LOGSTASH::"
      auth_key: logstash:logstash
      type: allow
      actions: ["cluster:monitor/main", "indices:admin/types/exists", "indices:data/read/*", "indices:data/write/*", "indices:admin/template/*", "indices:admin/create"]
      indices: ["logs-*"]

    # we trust Kibana; access allowed via HTTP auth
    - name: "::KIBANA-SERVER::"
      auth_key: kibana:kibana
      type: allow
      verbosity: error # don't log successful requests

    # this is for read-write user access via LDAP
    - name: "::RW ACCESS::"
      ldap_auth:
        name: "ldap1"
        groups: ["group_does_not_exist"]
      type: allow
      kibana_access: rw
      indices: [".kibana", ".kibana-devnull", "logs-*"]

    # this is for read-only user access via LDAP
    - name: "::RO ACCESS::"
      ldap_auth:
        name: "ldap1"
        groups: ["group_does_not_exist"]
      type: allow
      kibana_access: ro
      indices: [".kibana", ".kibana-devnull", "logs-*"]

  ldaps:
    - name: ldap1
      host: "xxx.xxx.com"
      port: 389
      ssl_enabled: false
      ssl_trust_all_certs: true
      search_user_base_DN: "ou=Users,ou=xxx,dc=xxx,dc=com"
      user_id_attribute: "uid"
      search_groups_base_DN: "ou=ELK,dc=xxx,dc=com"
      unique_member_attribute: "uniqueMember"
      connection_pool_size: 10
      connection_timeout_in_sec: 10
      request_timeout_in_sec: 10
      cache_ttl_in_sec: 60

Thanks for any help you can provide.

Greg

2

If nothing else is running, well, It is very strange that ES calls itself through the REST API. Not sure about what their proprietary code does there, but it’s certainly quite bizarre because they have a native Java client at their disposal in the plugin API.

Until we discover more, for now just add another rule allowing that action coming from localhost. Security-wise this workaround is quite OK.

Still weird though.

3

I’ve spent the last few weeks setting up ELK and its various components, and I think I’ve hit just about all the weird situations :anguished:

Anyway, thanks for the reply and the suggestion. I’m concerned about adding an unrestricted localhost rule (you can see I have one with an “auth_key” set), as I have Logstash and Kibana running on the same machine. When I do this, the localhost rule seems to override the specific ones for Logstash and Kibana, meaning that pretty much anything is allowed. Is there a way to add a localhost rule that will only apply to Elasticsearch itself?

Thoughts? Thanks!

4

you don’t need to use an unrestricted block, how about you pin-hole the ACL with this thing in the beginning

- name: "just that action from localhost"
  type: allow
  actions: ["cluster:monitor/health"]
  hosts: ["127.0.0.1"]

... all the other blocks here...
1 Like
5

Ah, thank you. That makes sense.

6

Please see the latest a bout X-Pack Monitoring compatibility.

TL;DR ReadonlyREST Enterprise/PRO contains a work around to inject the missing authorization headers, and Monitoring works normally.