[DONE] Move settings to config/raedonlyrest.yml


(Simone Scarduzio) #1

:bulb: Move settings to config/raedonlyrest.yml

Today, credentials (although hashed) are stored alongside with ES settings in ESHOME/config/elasticsearch.yml.
The problem is that they remain visible by ES itself, and can be accessed by other plugins.

One of the biggest instances of this issue is that the whole settings object may be dumped via ES HTTP API if you don’t configure your ACL appropriately.

Solution

So the plan is to move ReadonlyREST settings from:
ES_HOME/conf/elasticsearch.yml

to a separate file:

ESHOME/config/readonlyrest.yml.

:rocket: Let’s do this?

  • 1
  • 2
  • 3
  • 4
  • 5

0 voters


(Daniel Mota Leite) #2

being in a different file may also open the door to reload the config file without restarting the whole elasticsearch


(Simone Scarduzio) #3

I just implemented this in master :tada:
@danielmotaleite how would that work? How would one trigger the reload after changing the file?


(Daniel Mota Leite) #4

the reload could be implemented via a special url, that readonlyrest could intercept and action over or a probe that checks the file for modifications each X minutes.

I like more the url, we can add a curl to salt/ansible/puppet, etc to reload a config on each server and can control exacty when the config is reloaded (prepare the change in all servers, enable it via a curl to each server)

and by the way, many thanks about this! :smiley:


(Simone Scarduzio) #5

We have an API already there where the Kibana plugin sends a refresh request. Currently it watches the only the .readonlyrest index, but I could piggy back also the action to re-read from file if the index is not present.


(Simone Scarduzio) #6

FYI: The config/readonlyrest.yml support is out for download in version 1.16.11-pre5.

The “refresh from file” hook is not there yet, let’s see if I can add it before 1.16.11 full release


(Askids) #7

Can we move all ROR settings to config/readonlyrest.yml file OR some setting like providing the file path has to be still present on elasticsearch.yml file for it to identify that ROR is being used on that node.


(Simone Scarduzio) #8

This is already possible in the pre5 build. The configuration path is already customisable, as it will always be the same with elasticsearch.yml.


(Askids) #9

But I tried this yesterday with the pre5 build. I removed all entries from elasticsearch.yml and created a new file readonlyrest.yml. But ES failed to boot up.

If you see the error, its looking for conf\readonlyrest.yml and not config\readonlyrest.yml. Is this conf folder supposed to be under readonlyrest folder which is under plugins?

[2017-09-20T03:53:49,010][INFO ][o.e.p.r.e.SettingsManagerImpl] Could not find settings in conf\readonlyrest.yml, falling back to elasticsearch.yml (access denied ("java.io.FilePermission" "conf\readonlyrest.yml" "read"))


(Simone Scarduzio) #10

found the bug, it’s fixed in master.


(Askids) #11

Thanks!! This is working fine and settings are getting picked up from config/readonlyrest.yml.


(Kazys Gurskas) #13

I’m not getting this picked up somehow. Tried in ES_HOME, plugin directory and /etc/elasticsearch, none of the locations are picked up. I’m using ubuntu and installed ES from official deb repo. Any ideas?


(Simone Scarduzio) #14

Well, the plugin is expecting readonlyrest.yml to be under $ES_HOME/config or wherever else the main configuration file is found. Is this not working for you?


(Kazys Gurskas) #15

I have /etc/elasticsearch as my path.conf. Getting

[o.e.p.r.e.SettingsManagerImpl] Could not find settings in config/readonlyrest.yml, falling back to elasticsearch.yml (access denied ("java.io.FilePermission" "config/readonlyrest.yml" "read"))

in the logs. I’m using 1.16.11


(Simone Scarduzio) #16

@kazgurs it seems it’s a problem with relative paths, have a look here for workarounds


(Kazys Gurskas) #17

Well, starting ES binary manually is not exactly a workaround. The plugin should be aware of situations when ES is started as a service, and naturally path.conf IS absolute. It’s /etc/elasticsearch/ for both deb and rpm systems.


(Zozo6015) #18

Hello,

I have created a folder in /usr/share/elasticsearch/ named config and inside it I have created a file readonlyrest.yml with the configurations inside it. After I have restarted the elasticsearch I am still getting those errors in the elasticsearch logs which fails to start.

Regards,
Peter


(Askids) #19

@sscarduzio we recently started using Windows 2012. I have the same setup as before (from Win 2008 R2) and have started getting similar error as originally reported when using Win 2008 R2.

[2017-10-23T19:15:10,901][INFO ][o.e.p.r.e.SettingsManagerImpl] Could not find settings in config\readonlyrest.yml, falling back to elasticsearch.yml (access denied ("java.io.FilePermission" "config\readonlyrest.yml" "read"))

To fix this, temporarily, I had to moved the settings back the settings to elasticsearch.yml file. Then it started working - but only the authorization and authentication was working. Enabling https was throwing error that keystore path is incorrect. I think that it is definitely similar error.

[2017-10-23T19:15:16,360][INFO ][o.e.p.r.e.SSLTransportNetty4] SSL: attempting with JKS keystore..
[2017-10-23T19:15:16,361][ERROR][o.e.p.r.e.SSLTransportNetty4] Failed to load SSL certs and keys from JKS Keystore!
[2017-10-23T19:15:16,361][ERROR][o.e.p.r.e.SSLTransportNetty4] Check the JKS Keystore path is correct: config/my_keystore.jks

I originally had the keystore file path as mentioned below in first line (this works in Win 2008 R2). I then changed it to give the absolute path and then the SSL also started working.

#keystore_file: "config/my_keystore.jks"
keystore_file: D:\Apps\Program Files\Elasticsearch-5.5.1\config\my_keystore.jks

Looks like ROR has issue reading relative path and its similar issue in both reading readonlyrest.yml file as well as the keystore file. In my case, we have the config folder under elasticsearch installation folder itself. So we dont have a separate entry for path.conf. I think that ROR is unable to resolve the config path and hence subsequently fails to resolve the paths for readonlyrest.yml and keystore files. When you dont find readonlyrest.yml file, can you please also add a debug message to show to what folder location does the “$ES_HOME/config” resolves to? This might throw some additional light.

Thanks!


(Simone Scarduzio) #20

Hi @askids, thanks for bringing this up. The $ES_HOME detection is largely improved in the latest release we’ll release this week.


(Askids) #21

Great. Is there any pre-release version available that we could test with temporarily?