ECK + ROR | Access issue with internal API usage

Users Feedback
1

Hi,

We are currently using ECK version 8.11.0 with ROR version 1.53.0. While the cluster is up and running without any performance issues, we noticed the following recurrent error on our Elastic nodes -


{"@timestamp":"2024-04-04T12:28:53.651Z", "log.level": "INFO", "message":"\u001B[35mFORBIDDEN by default req={ ID:263549268-1295578936#10068214, TYP:DeleteDesiredNodesAction$Request, CGR:<N/A>, USR:elastic-internal (attempted), BRS:true, KDX:null, ACT:cluster:admin/desired_nodes/delete, OA:10.71.94.23/32, XFF:null, DA:10.71.71.141/32, IDX:<N/A>, MET:DELETE, PTH:/_internal/desired_nodes, CNT:<N/A>, HDR:Accept-Encoding=gzip, Authorization=<OMITTED>, Content-Type=application/json; charset=utf-8, Host=eck-elasticsearch-name-internal-http.k8s-namespace.svc:9200, User-Agent=Go-http-client/1.1, X-Elastic-Product-Origin=cloud, content-length=0, HIS:[::HEALTH::-> RULES:[actions->false]], [::READ::-> RULES:[actions->false]], [::BASIC_AUTH::-> RULES:[auth_key_sha256->false]], }\u001B[0m", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[eck-elasticsearch-name-es-client-0][transport_worker][T#3]","log.logger":"tech.beshu.ror.accesscontrol.logging.AccessControlLoggingDecorator","elasticsearch.cluster.uuid":"Vsm9R5udSxylC4wEfIRskA","elasticsearch.node.id":"yygMhHlhQnO2Y2_YfofAHA","elasticsearch.node.name":"eck-elasticsearch-name-es-client-0","elasticsearch.cluster.name":"eck-elasticsearch-name"}

We looked it up and this delete operation is one of Elastic’s internal APIs(ref Delete desired nodes API | Elasticsearch Guide [8.13] | Elastic) and is carried out by the “elastic-internal” user. Since we are using ROR, I am assuming we would have to add access control rules for this but we are struggling with how to add that.

Our current readonlyrest config looks like this -

      readonlyrest:
        enable: true
        prompt_for_basic_auth: false
        response_if_req_forbidden: Forbidden!
        access_control_rules:
          - name: "::HEALTH::"
            type: allow
            verbosity: error
            actions: ['cluster:monitor/*','indices:monitor/*']
          - name: "::READ::"
            type: allow
            verbosity: error
            actions: ['indices:data/read/*']
          - name: "::BASIC_AUTH::"
            type: allow
            verbosity: error
            auth_key_sha256: AUTH_KEY_TO_BE_REPLACED_AT_RUN_TIME

Any pointers would be appreciated.

Thank you

2

Hi, please take a look at these two files:

I think it should be clear now for you.
If not, I can explain what is going on there.

3

@shweta_c was it helpful?

4

@coutoPL - Thank you for the pointer. Sorry, haven’t been able to try this out yet, will do so and keep you posted.

5

@coutoPL - Thank your for the recommendation, we made the changes and that helped fix the errors we were seeing at the ECK cluster level. Thank you!

1 Like