ES failing to start after plugin installed

Hitardha · October 22, 2018, 11:29am

Hi,

Installed elasticsearch and kibana opensource (version oss-6.4.1), working fine and then I have downloaded and installed readonlyrest-1.16.27_es6.4.1.zip plugin and created readonlyrest.yml @ /etc/elasticsearch where elasticsearch.yml present. The content of readonlyrest.yml is as below.

[root@server elasticsearch]# cat /etc/elasticsearch/readonlyrest.yml
readonlyrest:
enable: true

access_control_rules:

- name: "Require HTTP Basic Auth"
  type: allow
  auth_key: admin:ABCofPASSWORD

[root@server elasticsearch]#

when I start elasticsearch, I am getting the following error:

[2018-10-22T13:10:07,898][INFO ][o.e.p.PluginsService ] [node-1] loaded module [tribe]
[2018-10-22T13:10:07,898][INFO ][o.e.p.PluginsService ] [node-1] loaded plugin [readonlyrest]
[2018-10-22T13:10:10,278][INFO ][t.b.r.e.SettingsObservableImpl] Could not find settings in /etc/elasticsearch/readonlyrest.yml (/etc/elasticse
arch/readonlyrest.yml)
[2018-10-22T13:10:10,326][ERROR][t.b.r.e.SettingsObservableImpl] Cannot parse YAML: NullPointerException:null
null
java.lang.NullPointerException: null
at java.util.regex.Matcher.getTextLength(Matcher.java:1283) ~[?:1.8.0_171]
at java.util.regex.Matcher.reset(Matcher.java:309) ~[?:1.8.0_171]
at java.util.regex.Matcher.(Matcher.java:229) ~[?:1.8.0_171]
at java.util.regex.Pattern.matcher(Pattern.java:1093) ~[?:1.8.0_171]
at org.yaml.snakeyaml.reader.StreamReader.checkPrintable(StreamReader.java:64) ~[snakeyaml-1.17.jar:?]
at org.yaml.snakeyaml.reader.StreamReader.(StreamReader.java:47) ~[snakeyaml-1.17.jar:?]
at org.yaml.snakeyaml.Yaml.load(Yaml.java:369) ~[snakeyaml-1.17.jar:?]
at tech.beshu.ror.commons.settings.SettingsUtils.lambda$yaml2Map$1(SettingsUtils.java:64) ~[?:?]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_171]
at tech.beshu.ror.commons.settings.SettingsUtils.yaml2Map(SettingsUtils.java:62) ~[?:?]
at tech.beshu.ror.commons.settings.BasicSettings.fromFile(BasicSettings.java:165) ~[?:?]
at tech.beshu.ror.commons.settings.BasicSettings.fromFileObj(BasicSettings.java:149) ~[?:?]
at tech.beshu.ror.es.SettingsObservableImpl.(SettingsObservableImpl.java:60) ~[?:?]
at tech.beshu.ror.es.ReadonlyRestPlugin.lambda$createComponents$0(ReadonlyRestPlugin.java:102) ~[?:?]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_171]
at tech.beshu.ror.es.ReadonlyRestPlugin.createComponents(ReadonlyRestPlugin.java:99) ~[?:?]
at org.elasticsearch.node.Node.lambda$new$8(Node.java:420) ~[elasticsearch-6.4.1.jar:6.4.1]
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:267) [?:1.8.0_171]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) [?:1.8.0_171]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) [elasticsearch-6.4.1.jar:6.4.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:86) [elasticsearch-6.4.1.jar:6.4.1]
[2018-10-22T13:10:10,408][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [node-1] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: tech.beshu.ror.commons.settings.SettingsMalformedException: Could not find required attribute ‘readonlyrest’ in file pidfile:/CHBS/data/ELK/es/elasticsearch.pid,client:{type=node},cluster:{name=DVNC},http:{host=phchbs-sd220166.eu.novartis.net, port=9200},node:{attr={rack=r1}, name=node-1},path:{data=[/CHBS/data/ELK/es/Data], logs=/CHBS/data/ELK/es/Logs, home=/usr/share/elasticsearch}
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:140) ~[elasticsearch-6.4.1.jar:6.4.1]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:127) ~[elasticsearch-6.4.1.jar:6.4.1]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.4.1.jar:6.4.1]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.4.1.jar:6.4.1]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.4.1.jar:6.4.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) ~[elasticsearch-6.4.1.jar:6.4.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:86) ~[elasticsearch-6.4.1.jar:6.4.1]
Caused by: tech.beshu.ror.commons.settings.SettingsMalformedException: Could not find required attribute ‘readonlyrest’ in file pidfile:/CHBS/data/ELK/es/elasticsearch.pid,client:{type=node},cluster:{name=DVNC},http:{host=elkmaster, port=9200},node:{attr={rack=r1}, name=node-1},path:{data=[/CHBS/data/ELK/es/Data], logs=/CHBS/data/ELK/es/Logs, home=/usr/share/elasticsearch}
at tech.beshu.ror.commons.settings.RawSettings.req(RawSettings.java:110) ~[?:?]
at tech.beshu.ror.commons.settings.RawSettings.inner(RawSettings.java:219) ~[?:?]
at tech.beshu.ror.commons.settings.BasicSettings.(BasicSettings.java:88) ~[?:?]
at tech.beshu.ror.commons.settings.BasicSettings.fromFile(BasicSettings.java:172) ~[?:?]
at tech.beshu.ror.commons.settings.BasicSettings.fromFileObj(BasicSettings.java:149) ~[?:?]
at tech.beshu.ror.es.SettingsObservableImpl.(SettingsObservableImpl.java:60) ~[?:?]
at tech.beshu.ror.es.ReadonlyRestPlugin.lambda$createComponents$0(ReadonlyRestPlugin.java:102) ~[?:?]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_171]

sscarduzio · October 22, 2018, 11:40am

This is a parsing error when reading the YAML settings in readonlyrest.yml

Can you please share your settings that generate this error? Please remember when you paste the YAML code to use the code formatting button (the button that looks like this </>) in order to preserve indentation.

Hitardha · October 22, 2018, 11:49am

readonlyrest:
enable: true

access_control_rules:

- name: "Require HTTP Basic Auth"
  type: allow
  auth_key: admin:ARTofDAVINCI

Thank you so much for your quick response. awaits for solution

sscarduzio · October 22, 2018, 11:56am

I think we have an indentation problem. all the lines after readonlyrest:should be indented. Also, wrap every string in quote, just in case.

I.e.

   readonlyrest:
      enable: true

      access_control_rules:

      - name: "Require HTTP Basic Auth"
        type: "allow"
        auth_key: "admin:ARTofDAVINCI"

Hitardha · October 22, 2018, 12:09pm

Hi,

I have just copied your suggestion and tried but still seeing the issue.

[2018-10-22T14:00:55,023][INFO ][t.b.r.e.SettingsObservableImpl] Could not find settings in /etc/elasticsearch/readonlyrest.yml (/etc/elasticsearch/readonlyrest.yml)
[2018-10-22T14:00:55,071][ERROR][t.b.r.e.SettingsObservableImpl] Cannot parse YAML: NullPointerException:null
 null
java.lang.NullPointerException: null

sscarduzio · October 22, 2018, 12:23pm

is your readonlyrest.ymlfile in the same directory with elasticsearch.yml?

Hitardha · October 22, 2018, 12:27pm

YES.

NOTE:- I am using OSS elasticsearch

sscarduzio · October 22, 2018, 1:01pm

so weird, I tested your settings and (obviously) work well. The plugin seems to resolve the correct path for the file, but when it tries to read it, it is a null string.
Are you sure that the user that runs the elasticsearch process has the rights to read that file? i.e.

ls -ltr /etc/elasticsearch

Hitardha · October 22, 2018, 2:54pm

Thank you so much for your help. I am able to fix the issue. It was a silly mistake, when I created the file it created with 600 permissions. It worked when I changed to 660.

Now I am trying to go for complex configuration of one admin access and one readonly access (in kibana) reading the documentation lets see how successful I could be.

Thanks once again

sscarduzio · October 22, 2018, 3:04pm

Happy to help @Hitardha!