Hi Team,
I have installed ROR on my Elastic cluster having 3 nodes and have below config file for readonlyrest.yml
readonlyrest:
enable: true
access_control_rules:
- name: "Basic Authentication to get started with"
indices: ["*"]
type: allow
auth_key: yyyy:xxxxxxxx
All 3 nodes are up and running but when i hit IP:9200 in browser getting below error,
{“message”:“Waiting for ReadonlyREST start: [undefined] Waiting for ReadonlyREST start”,“statusCode”:503,“error”:“Service Unavailable”}
Error logs,
[2019-11-22T05:34:26,880][INFO ][t.b.r.b.Ror$ ] [cs-es-node-01] [CLUSTERWIDE SETTINGS] Loading ReadonlyREST settings from index …
[2019-11-22T05:34:27,037][DEBUG][o.e.a.ActionModule ] [cs-es-node-01] Using REST wrapper from plugin tech.beshu.ror.es.ReadonlyRestPlugin
[2019-11-22T05:34:27,439][INFO ][o.e.d.DiscoveryModule ] [cs-es-node-01] using discovery type [zen] and seed hosts providers [settings]
[2019-11-22T05:34:28,144][INFO ][o.e.n.Node ] [cs-es-node-01] initialized
[2019-11-22T05:34:28,144][INFO ][o.e.n.Node ] [cs-es-node-01] starting …
[2019-11-22T05:34:28,261][INFO ][o.e.t.TransportService ] [cs-es-node-01] publish_address {10.156.0.156:9300}, bound_addresses {10.156.0.156:9300}
[2019-11-22T05:34:28,268][INFO ][o.e.b.BootstrapChecks ] [cs-es-node-01] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2019-11-22T05:34:28,274][INFO ][o.e.c.c.Coordinator ] [cs-es-node-01] cluster UUID [oakx2Z2wS_2jYK3zt2Kh9w]
[2019-11-22T05:34:28,564][INFO ][o.e.c.c.JoinHelper ] [cs-es-node-01] failed to join {cs-es-node-01}{AXJuaO_XQDCaw2LImf_A-g}{WxpFpG3JTcW0ZWl4lrfdsg}{10.156.0.156}{10.156.0.156:9300}{dim}{ml.machine_memory=33566875648, xpack.installed=true, ml.max_open_jobs=20} with JoinRequest{sourceNode={cs-es-node-01}{AXJuaO_XQDCaw2LImf_A-g}{WxpFpG3JTcW0ZWl4lrfdsg}{10.156.0.156}{10.156.0.156:9300}{dim}{ml.machine_memory=33566875648, xpack.installed=true, ml.max_open_jobs=20}, optionalJoin=Optional[Join{term=9, lastAcceptedTerm=8, lastAcceptedVersion=1234, sourceNode={cs-es-node-01}{AXJuaO_XQDCaw2LImf_A-g}{WxpFpG3JTcW0ZWl4lrfdsg}{10.156.0.156}{10.156.0.156:9300}{dim}{ml.machine_memory=33566875648, xpack.installed=true, ml.max_open_jobs=20}, targetNode={cs-es-node-01}{AXJuaO_XQDCaw2LImf_A-g}{WxpFpG3JTcW0ZWl4lrfdsg}{10.156.0.156}{10.156.0.156:9300}{dim}{ml.machine_memory=33566875648, xpack.installed=true, ml.max_open_jobs=20}}]}
org.elasticsearch.transport.RemoteTransportException: [cs-es-node-01][10.156.0.156:9300][internal:cluster/coordination/join]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: became follower
at org.elasticsearch.cluster.coordination.JoinHelper$CandidateJoinAccumulator.lambda$close$3(JoinHelper.java:477) [elasticsearch-7.3.2.jar:7.3.2]
at java.util.HashMap$Values.forEach(HashMap.java:976) [?:?]
at org.elasticsearch.cluster.coordination.JoinHelper$CandidateJoinAccumulator.close(JoinHelper.java:477) [elasticsearch-7.3.2.jar:7.3.2]
at org.elasticsearch.cluster.coordination.Coordinator.becomeFollower(Coordinator.java:606) [elasticsearch-7.3.2.jar:7.3.2]
at org.elasticsearch.cluster.coordination.Coordinator.onFollowerCheckRequest(Coordinator.java:243) [elasticsearch-7.3.2.jar:7.3.2]
at org.elasticsearch.cluster.coordination.FollowersChecker$2.doRun(FollowersChecker.java:187) [elasticsearch-7.3.2.jar:7.3.2]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:758) [elasticsearch-7.3.2.jar:7.3.2]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.3.2.jar:7.3.2]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:835) [?:?]
[2019-11-22T05:34:28,821][INFO ][o.e.c.s.ClusterApplierService] [cs-es-node-01] master node changed {previous [], current [{cs-es-node-02}{WQc55XBwRaaV2-A_3-nDcg}{NC1h9j5hRLe8v_pZSW4k4w}{10.156.0.157}{10.156.0.157:9300}{dim}{ml.machine_memory=33566867456, ml.max_open_jobs=20, xpack.installed=true}]}, added {{cs-es-node-02}{WQc55XBwRaaV2-A_3-nDcg}{NC1h9j5hRLe8v_pZSW4k4w}{10.156.0.157}{10.156.0.157:9300}{dim}{ml.machine_memory=33566867456, ml.max_open_jobs=20, xpack.installed=true},}, term: 10, version: 1235, reason: ApplyCommitRequest{term=10, version=1235, sourceNode={cs-es-node-02}{WQc55XBwRaaV2-A_3-nDcg}{NC1h9j5hRLe8v_pZSW4k4w}{10.156.0.157}{10.156.0.157:9300}{dim}{ml.machine_memory=33566867456, ml.max_open_jobs=20, xpack.installed=true}}
[2019-11-22T05:34:28,847][INFO ][o.e.h.AbstractHttpServerTransport] [cs-es-node-01] publish_address {10.156.0.156:9200}, bound_addresses {10.156.0.156:9200}
[2019-11-22T05:34:28,848][INFO ][o.e.n.Node ] [cs-es-node-01] started
[2019-11-22T05:34:47,913][INFO ][t.b.r.b.Ror$ ] [cs-es-node-01] Loading ReadonlyREST settings from index failed: cannot find index
[2019-11-22T05:34:47,914][INFO ][t.b.r.b.Ror$ ] [cs-es-node-01] Loading ReadonlyREST settings from file: /etc/elasticsearch/readonlyrest.yml
Please help in getting this sorted out asap.
Br
Pankaj N