Getting java.lang.IllegalArgumentException when initializing ROR in ES 8.10.4

1

Hi! I’m working on deploying Elasticsearch in a Kubernetes cluster using the elasticsearch bitnami chart. I extended the Docker image that bitnami uses to install the readonly rest plugin and use that image in my deployment:

FROM bitnami/elasticsearch:8.10.4-debian-11-r0
USER root
COPY readonlyrest-1.52.0_es8.10.4.zip /tmp/readonlyrest-1.52.0_es8.10.4.zip
RUN /opt/bitnami/elasticsearch/bin/elasticsearch-plugin install --batch file:///tmp/readonlyrest-1.52.0_es8.10.4.zip
RUN /opt/bitnami/java/bin/java -jar /opt/bitnami/elasticsearch/plugins/readonlyrest/ror-tools.jar patch --es-path /opt/bitnami/elasticsearch

I followed the instructions from the official documentation and the image appears to build succesfully:

=> [internal] load build definition from Dockerfile                                                                                     0.0s
 => => transferring dockerfile: 655B                                                                                                     0.0s
 => [internal] load .dockerignore                                                                                                        0.0s
 => => transferring context: 2B                                                                                                          0.0s
 => [internal] load metadata for docker.io/bitnami/elasticsearch:8.10.4-debian-11-r0                                                     0.6s
 => [internal] load build context                                                                                                        0.0s
 => => transferring context: 56B                                                                                                         0.0s
 => [1/4] FROM docker.io/bitnami/elasticsearch:8.10.4-debian-11-r0@sha256:69590312dd227d30b9c0c3d49ade69605166bb7e89e25becf6cdb07d635a5  0.0s
 => CACHED [2/4] COPY readonlyrest-1.52.0_es8.10.4.zip /tmp/readonlyrest-1.52.0_es8.10.4.zip                                             0.0s
 => [3/4] RUN /opt/bitnami/elasticsearch/bin/elasticsearch-plugin install --batch file:///tmp/readonlyrest-1.52.0_es8.10.4.zip           5.4s
 => [4/4] RUN /opt/bitnami/java/bin/java -jar /opt/bitnami/elasticsearch/plugins/readonlyrest/ror-tools.jar patch --es-path /opt/bitnam  1.3s
 => exporting to image                                                                                                                   0.6s 
 => => exporting layers                                                                                                                  0.6s 
 => => writing image sha256:019c1b4ad9429453198f6adf93f1140e72eae674e19aa11ad8cf9898cf423ad7                                             0.0s 
 => => naming to docker.io/elasticsearch/custom:test

However when my pods restart and ES tries to initialize the plugin I get the following error:

elasticsearch [2023-11-09T15:07:31,800][INFO ][t.b.r.b.LogPluginBuildInfoMessage$] [elasticsearch-master-2] Starting ReadonlyREST plugin v1.52.0 on Elasticsearch v8.10.4
elasticsearch [2023-11-09T15:07:31,837][ERROR][o.e.b.Elasticsearch      ] [elasticsearch-master-2] fatal exception while booting Elasticsearch java.lang.IllegalStateException: failed to load plugin class [tech.beshu.ror.es.ReadonlyRestPlugin]
elasticsearch     at org.elasticsearch.server@8.10.4/org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:675)
elasticsearch     at org.elasticsearch.server@8.10.4/org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:518)
elasticsearch     at org.elasticsearch.server@8.10.4/org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:303)
elasticsearch     at org.elasticsearch.server@8.10.4/org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:173)
elasticsearch     at org.elasticsearch.server@8.10.4/org.elasticsearch.plugins.PluginsService.lambda$getPluginsServiceCtor$16(PluginsService.java:702)
elasticsearch     at org.elasticsearch.server@8.10.4/org.elasticsearch.node.Node.<init>(Node.java:431)
elasticsearch     at org.elasticsearch.server@8.10.4/org.elasticsearch.node.Node.<init>(Node.java:338)
elasticsearch     at org.elasticsearch.server@8.10.4/org.elasticsearch.bootstrap.Elasticsearch$2.<init>(Elasticsearch.java:234)
elasticsearch     at org.elasticsearch.server@8.10.4/org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:234)
elasticsearch     at org.elasticsearch.server@8.10.4/org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:72)
elasticsearch Caused by: java.lang.reflect.InvocationTargetException
elasticsearch     at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
elasticsearch     at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
elasticsearch     at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
elasticsearch     at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
elasticsearch     at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
elasticsearch     at org.elasticsearch.server@8.10.4/org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:666)
elasticsearch     ... 9 more
elasticsearch Caused by: java.lang.ExceptionInInitializerError
elasticsearch     at tech.beshu.ror.tools.core.utils.EsDirectory$.<clinit>(EsDirectory.scala:27)
elasticsearch     at tech.beshu.ror.tools.core.patches.EsPatch$.create(EsPatch.scala:35)
elasticsearch     at tech.beshu.ror.es.utils.EsPatchVerifier$.$anonfun$createPatcher$1(EsPatchVerifier.scala:45)
elasticsearch     at scala.util.Try$.apply(Try.scala:210)
elasticsearch     at tech.beshu.ror.es.utils.EsPatchVerifier$.createPatcher(EsPatchVerifier.scala:45)
elasticsearch     at tech.beshu.ror.es.utils.EsPatchVerifier$.$anonfun$verify$2(EsPatchVerifier.scala:31)
elasticsearch     at scala.util.Either.flatMap(Either.scala:352)
elasticsearch     at tech.beshu.ror.es.utils.EsPatchVerifier$.$anonfun$verify$1(EsPatchVerifier.scala:30)
elasticsearch     at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
elasticsearch     at tech.beshu.ror.utils.AccessControllerHelper$$anon$1.run(AccessControllerHelper.scala:27)
elasticsearch     at java.base/java.security.AccessController.doPrivileged(AccessController.java:318)
elasticsearch     at tech.beshu.ror.utils.AccessControllerHelper$.doPrivileged(AccessControllerHelper.scala:26)
elasticsearch     at tech.beshu.ror.es.utils.EsPatchVerifier$.verify(EsPatchVerifier.scala:28)
elasticsearch     at tech.beshu.ror.es.ReadonlyRestPlugin.<init>(ReadonlyRestPlugin.scala:97)
elasticsearch     ... 15 more
elasticsearch Caused by: java.lang.IllegalArgumentException: requirement failed: ? is not an absolute path
elasticsearch     at scala.Predef$.require(Predef.scala:337)
elasticsearch     at os.Path.<init>(Path.scala:446)
elasticsearch     at os.Path$.apply(Path.scala:398)
elasticsearch     at os.package$.<clinit>(package.scala:19)
elasticsearch     ... 29 more
elasticsearch
elasticsearch ERROR: Elasticsearch did not exit normally - check the logs at /opt/bitnami/elasticsearch/logs/elastic.log
elasticsearch
elasticsearch ERROR: Elasticsearch exited unexpectedly, with exit code 1

The readonlyrest.yml I passed is a silly config file to just get started:

readonlyrest:
    enable: true
    access_control_rules:
      - name: "Require HTTP Basic Auth"
        type: allow
        auth_key: <user>:<password>

Could you please help me to figure it out where the error might be coming from? I might be suspecting that it is an issue with permissions since bitnami uses non-root containers (Work With Non-Root Containers for Bitnami Applications) but I’m not sure enough to confirm it. Thanks in advance!

2

Hi, thanks for reporting this. I’ve checked a code (based on the stacktrace you sent), and I see one potential problem. I’ve prepared a build with fix for you. Could you please check it and let us know about the result?

BTW, yes - the patching step for ES 8.10.x has to be done with root privileges.

3

Hi! Thanks for the quick response, I’ll give it a try and I’ll let you know the result. Thanks again!

4

I could try it yesterday and ROR was able to initialize now. I have to figure out the config specific for my deployment but so far everything is working well. Just out of curiosity, what was the cause of the error? Thank you for the help!

5

Great. Thanks for the confirmation. The fix will be a part of the ROR 1.53.0 release.

There was a stupid mistake. The default ES dir path was being evaluated and failed because it doesn’t exist. You use a custom ES dir path, so it should never be evaluated.

2 Likes