Good day,
I’m trying to start new cluster ES 7.13.4 (with native jdk 16) with latest RoR plugin 1.32 (or 1.33) and receiving error in log:
{ "cluster.name": "test-cluster", "cluster.uuid": "dh303uQvQqqs6zZCdHjwjQ", "component": "r.suppressed", "level": "WARN", "message": "path: /_nodes/_local/stats, params: {nodeId=_local}", "node.id": "THOypv93R5aDck0l82dqRg", "node.name": "node.domain.com", "stacktrace": [ "tech.beshu.ror.es.handler.response.RorNotAvailableResponse: ReadonlyREST failed to start", "at tech.beshu.ror.es.handler.response.RorNotAvailableResponse$.createRorStartingFailureResponse(RorNotAvailableResponse.scala:31) ~[readonlyrest-1.33.0_es7.13.4.jar:?]", "at tech.beshu.ror.es.IndexLevelActionFilter.$anonfun$apply$1(IndexLevelActionFilter.scala:117) [readonlyrest-1.33.0_es7.13.4.jar:?]", "at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) [scala-library-2.12.10.jar:?]", "at tech.beshu.ror.utils.AccessControllerHelper$$anon$1.run(AccessControllerHelper.scala:25) [core-1.33.0.jar:?]", "at java.security.AccessController.doPrivileged(AccessController.java:312) [?:?]", "at tech.beshu.ror.utils.AccessControllerHelper$.doPrivileged(AccessControllerHelper.scala:24) [core-1.33.0.jar:?]", "at tech.beshu.ror.es.IndexLevelActionFilter.apply(IndexLevelActionFilter.scala:92) [readonlyrest-1.33.0_es7.13.4.jar:?]", "at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:171) [elasticsearch-7.13.4.jar:7.13.4]", "at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:149) [elasticsearch-7.13.4.jar:7.13.4]", "at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:77) [elasticsearch-7.13.4.jar:7.13.4]", "at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:86) [elasticsearch-7.13.4.jar:7.13.4]", "at org.elasticsearch.rest.action.RestCancellableNodeClient.doExecute(RestCancellableNodeClient.java:80) [elasticsearch-7.13.4.jar:7.13.4]", "at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:402) [elasticsearch-7.13.4.jar:7.13.4]", "at org.elasticsearch.client.support.AbstractClient$ClusterAdmin.execute(AbstractClient.java:695) [elasticsearch-7.13.4.jar:7.13.4]", "at org.elasticsearch.client.support.AbstractClient$ClusterAdmin.nodesStats(AbstractClient.java:791) [elasticsearch-7.13.4.jar:7.13.4]", "at org.elasticsearch.rest.action.admin.cluster.RestNodesStatsAction.lambda$prepareRequest$3(RestNodesStatsAction.java:172) [elasticsearch-7.13.4.jar:7.13.4]", "at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:106) [elasticsearch-7.13.4.jar:7.13.4]", "at tech.beshu.ror.es.ReadonlyRestPlugin.$anonfun$getRestHandlerWrapper$2(ReadonlyRestPlugin.scala:235) [readonlyrest-1.33.0_es7.13.4.jar:?]", "at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:269) [elasticsearch-7.13.4.jar:7.13.4]", "at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:351) [elasticsearch-7.13.4.jar:7.13.4]", "at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:192) [elasticsearch-7.13.4.jar:7.13.4]", "at org.elasticsearch.http.AbstractHttpServerTransport.dispatchRequest(AbstractHttpServerTransport.java:451) [elasticsearch-7.13.4.jar:7.13.4]", "at org.elasticsearch.http.AbstractHttpServerTransport.handleIncomingRequest(AbstractHttpServerTransport.java:516) [elasticsearch-7.13.4.jar:7.13.4]", "at org.elasticsearch.http.AbstractHttpServerTransport.incomingRequest(AbstractHttpServerTransport.java:378) [elasticsearch-7.13.4.jar:7.13.4]", "at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:31) [transport-netty4-client-7.13.4.jar:7.13.4]", "at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:17) [transport-netty4-client-7.13.4.jar:7.13.4]", "at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at org.elasticsearch.http.netty4.Netty4HttpPipeliningHandler.channelRead(Netty4HttpPipeliningHandler.java:47) [transport-netty4-client-7.13.4.jar:7.13.4]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111) [netty-codec-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) [netty-codec-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) [netty-codec-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) [netty-handler-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.49.Final.jar:4.1.49.Final]", "at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.49.Final.jar:4.1.49.Final]", "at java.lang.Thread.run(Thread.java:831) [?:?]" ], "timestamp": "2021-08-11T06:43:20,008-07:00", "type": "server" }
I’ve tried with and without SSL, different sets of rules - no luck. Current RoR config:
readonlyrest: force_load_from_file: true enable: true access_control_rules: - name: "Require HTTP Basic Auth" type: allow auth_key: user:password
Current ES config:
cluster.name: test-cluster node.name: node.domain.com # roles node.master: false node.data: false node.ingest: false # attributes node.attr.chassis: "chassis-xxx-4" # ---- Paths ------------------------------------------------------------------- path.logs: /var/log/elasticsearch path.data: - /srv/elasticsearch_data # ---- Memory ------------------------------------------------------------------ bootstrap.memory_lock: true # ---- Network ----------------------------------------------------------------- network.host: x.x.x.x http.port: 8443 #http.type: ssl_netty4 # ---- Gateway ----------------------------------------------------------------- # Block initial recovery after a full cluster restart until N nodes are started: gateway.recover_after_nodes: 3 # ---- ThreadPools Settings ---------------------------------------------------- thread_pool: write: queue_size: 10000 # ---- X-Pack ------------------------------------------------------------------ xpack.ilm.enabled: true xpack.graph.enabled: false xpack.ml.enabled: false xpack.monitoring.enabled: false xpack.security.enabled: false xpack.watcher.enabled: false xpack.license.self_generated.type: basic # ---- X-Pack: Monitoring ------------------------------------------------------ xpack.monitoring.collection.enabled: false
Please advise if you have any ideas. My thought it is something related with java permissions within plugin, but cannot be sure 100%.