I’m working on deploying an ELK cluster with filebeat/logstash/elasticsearch/kibana, and have the readonlyrest plugin working on the Elasticsearch side and am evaluating the enterprise Kibana plugin, but have some questions. I’ve done a bit of searching on the forum but haven’t found anything that directly addresses my questions.
Quick overview of our current setup:
- Filebeat (6.6.0) on clients (mix of CentOS and RedHat 6 and 7) using syslog listener and app logfile watching to pass on to Logstash nodes (working)
- Logstash node(s) (6.6.0) configured with necessary grok and such, passing data to ES cluster over encrypted port 9200 (working)
- Elasticsearch cluster (4 hosts in current test cluster) (6.5.4, version dictated by readonlyrest version support) using readonlyrest to enable password auth for the logstash boxes.
- Kibana (6.5.4, version dictated by elasticsearch) - originally had been using httpd reverse proxy but had a number of odd issues with that so am currently trying to deploy RoR Kibana plugin listening directly.
- All of the logstash/elasticsearch/kibana hosts are CentOS 7, current version and fully patched.
We use puppet enterprise to deploy the configuration, so I’m working with the appropriate elastic modules for each of the above. I looked and didn’t see any community modules for readonlyrest so have been configuring that with fairly simple puppet configurations. Can supply that info if needed/interested.
I’ve been trying to follow the instructions found in the kibana.md on github, but don’t fully follow.
Here is the kibana.yml as it stands right now:
--- elasticsearch.password: derpderpderp elasticsearch.requestTimeout: '60000' elasticsearch.ssl.certificateAuthorities: "/etc/puppetlabs/puppet/ssl/certs/ca.pem" elasticsearch.url: https://elasticsearch1-0:9200 elasticsearch.username: elasticsearch kibana.index: ".bz1kibana" logging.dest: "/var/log/elk/kibana.log" server.host: derp.internal.maas360.com server.port: '8888' server.ssl.enabled: false xpack.graph.enabled: false xpack.ml.enabled: false xpack.monitoring.enabled: true xpack.security.enabled: false xpack.watcher.enabled: false
readonlyrest.yml - only installed on the elasticsearch hosts, but the same file is installed and configured on all four hosts.
# yamllint disable rule:line-length readonlyrest: enable: true ssl: enable: true keystore_file: "elasticsearch.jks" keystore_pass: "derpderpderp" key_pass: "derpderpderp" access_control_rules: - name: "Allow localhost" hosts: [127.0.0.1] - name: "::ADMIN::" type: allow auth_key: "admin:derp" # KIBANA ADMIN ACCESS NEEDED TO EDIT SECURITY SETTINGS IN ROR KIBANA APP! kibana_access: admin - name: "::LOGSTASH::" type: allow auth_key: "elasticsearch:derpderpderp" verbosity: error - name: "readonly" type: allow auth_key: "readonly:ylnodear" kibana_access: ro indices: [".kibana", ".kibana-devnull", "logstash-*"] kibana_hide_apps: ["readonlyrest_kbn", "timelion", "kibana:dev_tools", "kibana:management"] - name: "ldap for the rest" ldap_authentication: name: "ldap1" groups: ["ipausers"] type: allow indices: ["*-*"] verbosity: error ldaps: - name: ldap1 host: "ldapserver.example.com" port: 636 ssl_enabled: true ssl_trust_all_certs: true search_user_base_DN: "cn=accounts,dc=derp,dc=localdomain" search_groups_base_DN: "cn=accounts,dc=derp,dc=localdomain" user_id_attribute: "uid" unique_member_attribute: "member" connection_pool_size: 10 connection_timeout_in_sec: 30 request_timeout_in_sec: 30 cache_ttl_in_sec: 60 group_search_filter: "(objectclass=top)" group_name_attribute: "cn"
The readonlyrest.yml is modified from the version that I inherited which is being used on our current Elasticsearch 2.x cluster which is in production. That version only specifies the logstash user/pass and the LDAP.
Initial install of the plugin doesn’t go smoothly, as it hangs up on the “Optimizing and caching browser bundles…” step - have let it run for hours and it doesn’t complete. I have done this step from the installation instructions, but it doesn’t seem to help:
$ touch optimize/bundles/readonlyrest_kbn.style.css
Each time I’ve tried to install I wind up cancelling out of the Optimizing step and hoping the plugin was installed properly.
readonlyrest.yml only goes on the elasticsearch systems, correct? I don’t need a version of it on the kibana host? There’s no config specific to the kibana plugin on the kibana side beyond what goes in kibana.yml?
I’ve been unable to login to Kibana with anything but the logstash password.
Is it possible to test the kibana access with simple
curlcommands to the elasticsearch port 9200? I’ve gotten used to using that and found I had to add
type: allowto each entry under
access_control_rulesin the readonlyrest.yml file to get that access to authenticate properly, but I don’t know if that translates properly. Also, using the above readonlyrest.yml on the elasticsearch boxes I can curl the
_cat/indiceson port 9200 using the “admin” account but not the “readonly” account, which gets a JSON “forbidden” error.
I think that’s it for now, but am quite sure there will be more questions. Thanks for the assistance.