Kibana task manager failures

1

Hi

I see kibana logs throwing these authentication errors.
Is ROR requesting the task manager update query?

How can I disable this?

"type":"log","@timestamp":"2020-10-21T04:00:21Z","tags":["error","plugins","taskManager","taskManager"],"pid":32307,"message":"Failed to poll for work: [undefined] Waiting for ReadonlyREST start :: {\"path\":\"/.kibana_task_manager/_update_by_query\"

-Thanks

2

this is a transitory error when Kibana tries to talk to Elasticsearch when ROR did not load its configuration just yet. It’s normal.

3

Thanks Simone, may I know what should be the config to allow this ?

4 
{
	"type": "coordserver",
	"timestamp": "2020-10-21T14:13:06,800-04:00",
	"level": "INFO",
	"component": "t.b.r.a.l.AccessControlLoggingDecorator",
	"cluster.name": "elkdr",
	"node.name": "elk01",
	"message": "\u001B[35mFORBIDDEN by default req={  ID:1925865389-1077335052#393,  TYP:GetIndexRequest,  CGR:N/A,  USR:eslnrs (attempted),  BRS:false,  KDX:null,  ACT:indices:admin/get,  OA:127.0.0.1/32,  XFF:null,  DA:127.0.0.1/32,  IDX:.kibana_task_manager,  MET:GET,  PTH:/.kibana_task_manager,  CNT:<N/A>,  HDR:Authorization=<OMITTED>, Connection=keep-alive, Content-Length=0, Host=localhost:9200,  HIS:[Kibana Server-> RULES:[auth_key_sha256->false], RESOLVED:[indices=.kibana_task_manager]], [Logstash Server-> RULES:[auth_key_sha256->false], RESOLVED:[indices=.kibana_task_manager]], [Elasticsearch server-> RULES:[auth_key_sha256->true, actions->false], RESOLVED:[user=eslnrs;indices=.kibana_task_manager]], [Monitoring Server-> RULES:[groups->false], RESOLVED:[indices=.kibana_task_manager]], [Kibana Admin-> RULES:[groups->false], RESOLVED:[indices=.kibana_task_manager]], [ESP Admin-> RULES:[groups->false], RESOLVED:[indices=.kibana_task_manager]], [Kibana Read Write-> RULES:[groups->false], RESOLVED:[indices=.kibana_task_manager]], [ESP read-write-> RULES:[groups->false], RESOLVED:[indices=.kibana_task_manager]], [Kibana Read Only-> RULES:[groups->false], RESOLVED:[indices=.kibana_task_manager]], [ESP read-only-> RULES:[groups->false], RESOLVED:[indices=.kibana_task_manager]], [ESP Cache-> RULES:[groups->false], RESOLVED:[indices=.kibana_task_manager]]  }\u001B[0m",
	"cluster.uuid": "2XirXWf-T5qwnF4gOkvSNw",
	"node.id": "RovpTqtWS6O0AXmwxb1pXA"
}

I keep getting this. Here is the block for Elasticsearch server userid that , Kibana is using.

- name: "Elasticsearch server"
  actions: ["cluster:*","indices:*","internal:*"]
  indices: ["logstash-*",".kibana*",".apm*"]
  verbosity: error

May I know what I am missing? It always says “actions -> fase”

-Thanks
Praveen

5

Is there a way to load the readonlyrest.yml directly into ES ?
I used the API , _readonlyrest/admin/config but it needs json. ROR config is in YAML.

6

PRO/Enterprise subscribers can use Kibana to configure the cluster-wide settings. The API you are using is the internal one (non-public, subject to changes without a deprecation process) , used by our Kibana plugin.

The only other way to feed ROR some configuration is to change the readonlyrest.yml file and restart the node.

7

We use the Enterprise version, my problem is. Lets say I make changes in readyonlyrest.yml and upload it using Kibana UI. For some reason if Kibana cannot connect to ES (may be because I restricted ES user used by Kibana), then I am stuck.

Restarting the ES node, with new ROR.yml changes is not taking effect. It seems like it reads from index only. Only way is to upload the config change using API. And API does not take YAML. Its a vicious cycle.

8

@clutroth spent a lot of work thinking about this very issue recently. And your case is covered:

Force loading ACL from file

Set force_load_from_file in one node to modify config value by kibana plugin.
elasticsearch.yaml

readonlyrest:
  force_load_from_file: true

after saving config, disable force_load_from_file

@clutroth can you update the main documentation to include this?