Setting ROR for Logstash

1

Hello everyone, my task is to install authorization on all servers in the cluster, and not just on the server with kibana.
After installing ROR, Elasticsearch began to give an error

{"error":{"root_cause":[{"type":"circuit_breaking_exception","reason":"[parent] Data too large, data for [<http_request>] would be [30653496160/28.5gb], which is larger than the limit of [30369601945/28.2gb], real usage: [30653496160/28.5gb], new bytes reserved: [0/0b], usages [request=0/0b, fielddata=0/0b, in_flight_requests=21028/20.5kb, model_inference=0/0b, eql_sequence=0/0b, accounting=4136708/3.9mb]","bytes_wanted":30653496160,"bytes_limit":30369601945,"durability":"PERMANENT"}],"type":"circuit_breaking_exception","reason":"[parent] Data too large, data for [<http_request>] would be [30653496160/28.5gb], which is larger than the limit of [30369601945/28.2gb], real usage: [30653496160/28.5gb], new bytes reserved: [0/0b], usages [request=0/0b, fielddata=0/0b, in_flight_requests=21028/20.5kb, model_inference=0/0b, eql_sequence=0/0b, accounting=4136708/3.9mb]","bytes_wanted":30653496160,"bytes_limit":30369601945,"durability":"PERMANENT"},"status":429}

Ror conf

readonlyrest:
    audit_collector: true
    audit_serializer: tech.beshu.ror.requestcontext.QueryAuditLogSerializer
    audit_index_template: "'.readonlyrest-audit'-yyyy-MM-DD"
    prompt_for_basic_auth: false
    response_if_req_forbidden: Wrong password or try clearing your browser cache

    access_control_rules:
    - name: "kibana user"
      auth_key: user:pass
      kibana_access: rw

    - name: "apm_central_config"
      auth_key: user:pass
      kibana_access: rw

    - name: "Internal kibana account"
      type: allow
      auth_key: user:pass

    - name: "IP clusters"
      hosts: ["1.1.1.1", "2.2.2.2"]

    - name: "XCS-access"
      hosts: ["11.11.11.11", "22.22.22.22"]

    - name: "jaeger-access"
      hosts: ["13.13.13.13", "23.23.23.23"]

    - name: "Logstash access"
      hosts: ["14.14.14.14", "24.24.24.24"]

    - name: "LOCALHOST-only access"
      hosts: ["127.0.0.1"]

Maybe you know the best solution so that the connection from Logstash does not waste all the server’s RAM.

2

I disabled logs for logstash

verbosity: error

It seems to be better. Maybe you can still optimize something?

3

I don’t know if there is sth to optimize here. The issue is not related to ROR directly:

I guess that high traffic generated by Logstash and enabled info logs (I assume that you have a default log4j2.properties) for the Logstash block caused more memory consumption than ES without ROR plugin enabled. This is expectable and justified.

Your solution is OK if you don’t need the logs. To reduce memory consumption (and CPU too) you should put the most frequently matched blocks at the top of the ROR ACL (obviously as far as possible)