[DONE] Store configuration in index

(Simone Scarduzio) #1

:bulb: Configuration in index

I’ve been thinking that people operating a cluster might have some issues guaranteeing the ES config file must be copied over to all the nodes + restart every time they add a user or change some ROR conf.

:eyes: Example

  • Have an ES cluster using ROR
  • Need to change a password / add a user / modify anything

Now:

  • Got to edit the elasticsearch.yml
  • Got to scp it over to all nodes
  • Got to restart every node (downtime and shit)

How about:

  • Download the current conf (a YML file coming down from a new endpoint in ES)
  • Send a POST containing the modified YML conf
  • DONE.

NB: Configuration GET/POST can be authenticated
Plus in the future this can easily be a web tool.

:rocket: Let’s do this?

  • 1
  • 2
  • 3
  • 4
  • 5

0 voters

Show results
#2

I’m do that by chef and automatic configuration of all nodes. Much more useful would be using external source of users - ldap/AD etc :slight_smile:

(Simone Scarduzio) #3

Makes sense. Thanks @undefine :slight_smile:

(Daniel Mota Leite) #4

I do it with salt, but the restart part is a pain, as i have to restart one node at time and wait for the system to sync the replicas, so i can reboot the next one. This way i do not have any downtime, but is slow (and annoying when i forgot some permission and have to restart the cluster again!) :slight_smile:
The config as a index would be perfect!

(Simone Scarduzio) #5

@danielmotaleite I know you’re a good tinkerer and would have appreciated this one. The goal with this is:

  • Lower the tweak & test loop
  • Avoid downtime
  • Avoid mismatching configuration files across nodes
  • Remove unencrypted creds and configuration info leaks
  • Nicer UX.

The common understanding though is that this is more of a good to have, than a fundamental feature. Maybe a good candidate for a ReadonlyREST “PRO” version in the future.

(Ld57) #6

Also, chef master server cannot be installed on windows environnment.

here an opportunity to create an “OS” independant solution ?

also, I just read ( because I am fighting with logstash like a fool) that it is possible to reload logstash config without restarting everything.
https://www.elastic.co/guide/en/logstash/current/reloading-config.html

also for elasticsearch
https://www.elastic.co/guide/en/elasticsearch/reference/current/settings.html

(Daniel Mota Leite) #7

@ld57 Logstash can be reloaded, but elasticsearch, AFAIK, do not.

:frowning:

(Ld57) #8

yeah, I had in mind they would extend the capabilities of cluster api module, especially the dynamic module; but still very limited, and concern only node by node…

(Shimon) #10

I thought maybe the configuration could be loaded from the index during the first request, and to add a listener to the RestHandler so that every time a configuration is added to the index it would send a modified cluster state so rest of the nodes will know to update the configuration