Error: socket hang up

1

Hello,

When I try to list indices using kibana with RoR I’m getting error:

Jul 30 13:07:59 logs kibana[8538]: Error: socket hang up
Jul 30 13:07:59 logs kibana[8538]:     at Socket.socketCloseListener (node:_http_client:473:25)
Jul 30 13:07:59 logs kibana[8538]:     at Socket.emit (node:events:531:35)
Jul 30 13:07:59 logs kibana[8538]:     at TCP.<anonymous> (node:net:338:12)
Jul 30 13:07:59 logs kibana[8538]:     at TCP.callbackTrampoline (node:internal/async_hooks:130:17)
Jul 30 13:08:04 logs kibana[8538]: [13:08:04:072] [info][plugins][ReadonlyREST][authController][x-ror-correlation-id=5d3bf4cd-8221-40ef-8a88-1d98c92c04f4] Refreshing session against ES
Jul 30 13:08:29 logs kibana[8538]: Error: socket hang up
Jul 30 13:08:29 logs kibana[8538]:     at Socket.socketCloseListener (node:_http_client:473:25)
Jul 30 13:08:29 logs kibana[8538]:     at Socket.emit (node:events:531:35)
Jul 30 13:08:29 logs kibana[8538]:     at TCP.<anonymous> (node:net:338:12)
Jul 30 13:08:29 logs kibana[8538]:     at TCP.callbackTrampoline (node:internal/async_hooks:130:17)
Jul 30 13:08:52 logs kibana[8538]: [13:08:52:766] [info][plugins][ReadonlyREST][authController][x-ror-correlation-id=5d3bf4cd-8221-40ef-8a88-1d98c92c04f4] Refreshing session against ES
Jul 30 13:08:59 logs kibana[8538]: Error: socket hang up
Jul 30 13:08:59 logs kibana[8538]:     at Socket.socketCloseListener (node:_http_client:473:25)
Jul 30 13:08:59 logs kibana[8538]:     at Socket.emit (node:events:531:35)
Jul 30 13:08:59 logs kibana[8538]:     at TCP.<anonymous> (node:net:338:12)
Jul 30 13:08:59 logs kibana[8538]:     at TCP.callbackTrampoline (node:internal/async_hooks:130:17)
Jul 30 13:09:27 logs kibana[8538]: [13:09:27:507] [info][plugins][ReadonlyREST][authController][x-ror-correlation-id=5d3bf4cd-8221-40ef-8a88-1d98c92c04f4] Refreshing session against ES
Jul 30 13:09:28 logs kibana[8538]: Error: socket hang up
Jul 30 13:09:28 logs kibana[8538]:     at Socket.socketOnEnd (node:_http_client:524:23)
Jul 30 13:09:28 logs kibana[8538]:     at Socket.emit (node:events:531:35)
Jul 30 13:09:28 logs kibana[8538]:     at endReadableNT (node:internal/streams/readable:1696:12)
Jul 30 13:09:28 logs kibana[8538]:     at processTicksAndRejections (node:internal/process/task_queues:82:21)
Jul 30 13:09:29 logs kibana[8538]: Error: socket hang up
Jul 30 13:09:29 logs kibana[8538]:     at Socket.socketCloseListener (node:_http_client:473:25)
Jul 30 13:09:29 logs kibana[8538]:     at Socket.emit (node:events:531:35)
Jul 30 13:09:29 logs kibana[8538]:     at TCP.<anonymous> (node:net:338:12)
Jul 30 13:09:29 logs kibana[8538]:     at TCP.callbackTrampoline (node:internal/async_hooks:130:17)

On web ui errors is:

image
image918×735 18.6 KB

Right now I have 4058 indices on one VM. Data is sent using filbeat agent installed on multiple clients.

Elasticsearch version: 8.14.1
Elasticsearch plugin version: readonlyrest-1.59.0-pre1_es8.14.1.zip
Kubana plugin version: readonlyrest_kbn_universal-1.57.3_es8.14.1.zip

VM Resources are:
CPU: 16core
RAM: 32G

Can I find a way to list indices without error? Even using API I am not able to modify data views.

Thanks

2

Could you please try to use ES REST API to list the indices?

curl -v -u USER:PASSWORD http://ES_ADDRESS:9200/_cat/indices

What do you get?
Do you see sth interested (eg. stack trace) in the ES logs?

3

Using curl I can list indices.
Main issue is when I try to update dataview (via API or web ui) with a new index name, at that point I’m getting same error like I’m listing indices from web ui.
No any stacktraces in elastic logs.

Thanks

4

Please remove the ROR KBN plugin and confirm this is strictly an ROR problem.
Because TBH I’m not so sure.

5

With kibana removed and elastic plugin installed I’m getting loop messages like the following:

{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.11.0"},"@timestamp":"2024-07-31T11:19:44.695+00:00","message":"License is not available or does not support security features, re-authentication is not possible (available: true, enabled: false, unavailable reason: undefined).","log":{"level":"ERROR","logger":"plugins.security.authentication"},"process":{"pid":1342573,"uptime":114.344920905},"trace":{"id":"553402153e2803efbbb16833ccdb8b0c"},"transaction":{"id":"b5d19a9a291085e1"}}
6

Oh, right. It seems that you have to disable or remove ROR from the ES too.