Securing Elasticsearch communication with Readonlyrest plugin


#1

Hello,

I am running Elasticsearch and Kibana stack 5.5.0 with Readonlyrest plugin on server side and Filebeat agent on client side. I have defined groups and users rules and everything works like a charm.
The only thing that I didn’t find out how to add it, is SSL encryption.

How can I protect data sent by Filebeat to Elasticsearch against sniffing with Readonlyrest plugin?


(Simone Scarduzio) #2

Hello @madou23, glad it works well! This documentation might help you:

And use this build, because there’s a bug in ReadonlyREST 1.16.8 right on the SSL configuration.

https://readonlyrest-data.s3-eu-west-1.amazonaws.com/build/1.16.9-pre4/readonlyrest-1.16.9-pre4_es5.5.0.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJEKIPNTOTIVGQ4EQ/20170727/eu-west-1/s3/aws4_request&X-Amz-Date=20170727T105925Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=a32793e0201e85eaf69b7d520e9dfdeec33b73e379ddfb57a50ce7636cc44710


#3

Thank you @sscarduzio
I tried to follow the documentation but I have a problem in metricbeat side.
First, I generated self signed .jks key for a my private network host server (elastic.lan) using keytool

keytool -keypasswd readonlyrest -genkey -keyalg RSA -alias selfsigned -keystore keystore.jks -storepass readonlyrest -validity 3650 -keysize 2048 -dname "cn=elastic.lan"

then I added to elasticserach.yml

http.type: ssl_netty4
readonlyrest:
    ssl:
       keystore_file: "/elasticsearch/plugins/readonlyrest/keystore.jks"
       keystore_pass: readonlyrest
       key_pass: readonlyrest

and everything is working well when I tested it on browser or with curl command.
For metricbeat side, I extracted from .jks file the ceritficate and the key files using these commands

keytool -importkeystore -srckeystore keystore.jks -destkeystore keystore.p12 -deststoretype PKCS12
openssl pkcs12 -in keystore.p12 -nokeys -out cert.crt
openssl pkcs12 -in keystore.p12 -nocerts -nodes -out key.key

and copied them to my client server and edited metricbeat.yml

output.elasticsearch:
# Array of hosts to connect to.
hosts: ["elastic.lan:9200"]
template.enabled: false
template.versions.2x.enabled: false
template.overwrite: false
username: "client"
password: "XXXXXXX"
protocol: "https"
ssl.enabled: true
ssl.certificate: "/etc/pki/tls/certs/cert.crt"
ssl.key: "/etc/pki/tls/certs/key.key"

Metricbeat was running but I get this error on log file

ERR Connecting error publishing events (retrying): Get https://elastic.lan:9200: x509: certificate signed by unknown authority

I know that I didn’t have a signed certificate from a certificate authority, is this the only problem or there is a messing up on my configuration ? If it is, how can I solve it ?


(Simone Scarduzio) #4

With self signed certificates, you need the client to be configured in such a way that it does not check for certificates.

In the case of metrictbeats a google search brought me to this setting (put it to ‘none’)
https://www.elastic.co/guide/en/beats/metricbeat/current/configuration-output-ssl.html#_verification_mode


#5

I have already found out how to create self signed certificate with certificate authority that I have made and created certificate/key for both server and client signed with the CA following this article [here]
(https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-tutorial-mutual-auth-tls/html/mutual-authentication-certificates.html).
So I created on server side

cacert.pem
servercert.pem
serverkey.pem

then I extracted from them the keystore file and for the client side I have

clientcert.pem
clientkey.pem

with this configuration on elasticsearch.yml

http.type: ssl_netty4
readonlyrest:

    ssl:
       enable: true
       keystore_file: "/usr/share/elasticsearch/plugins/readonlyrest/keystore.jks"
       keystore_pass: readonlyrest
       key_pass: readonlyrest

and metricbeat.yml

output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["elastic.lan:9200"]
  template.enabled: false
  template.versions.2x.enabled: false
  template.overwrite: false
  # Optional protocol and basic auth credentials.
  username: "client"
  password: "XXXXXX"
  protocol: "https"
  ssl.enabled: true
  ssl.certificate_authorities: ["/etc/pki/tls/certs/cacert.pem"]
  ssl.certificate: "/etc/pki/tls/certs/clientcert.pem"
  ssl.key: "/etc/pki/tls/certs/clientkey.pem"

everything is working like a charm, metricbeat is sending metrics without any error but the problem now is with kibana it can’t access to elasticsearch any more.
I tried to add these lines in kibana.yml

elasticsearch.url: "https://localhost:9200"
elasticsearch.ssl.certificate: /etc/pki/tls/certs/servercert.pem
elasticsearch.ssl.key: /etc/pki/tls/certs/serverkey.pem
elasticsearch.ssl.certificateAuthorities: [ "/etc/pki/tls/certs/cacert.pem" ]

but I get always this error in kibana status page

plugin:[email protected]	 Unable to connect to Elasticsearch at https://localhost:9200.

When I try to comment https/ssl block in elasticsearch.yml and connect kibana to http://localhost:9200 it works and shows the latest metrics from metricbeat index but metricbeat stop from sending metrics because https/ssl block in elasticsearch was disabled. How can I run ssl encryption for both ES <=> metricbeat and ES <=> kibana sides ?


(Simone Scarduzio) #6

Make sure you tell both Metricbeat AND Kibana what cert and CA to use.


#7

I find out the error, I must change

elasticsearch.url: "https://localhost:9200"

to my private network host server that I indicated on certificate files in my case elastic.lan.

elasticsearch.url: "https://elastic.lan:9200"

Thank you for your help everything works fine now :smiley:


(Simone Scarduzio) #8

Yeah, what you did is more delicate procedure, but definitely more secure than doing self-signed cert + skip cert checking.

Actually I like your way very much as you don’t need an external CA, so it’s free (as in $$$) and secure too, if you distribute the CA files.

Good job @madou23 :thumbsup: