JWT auth in free version

I’ve just tested:

{"type": "server", "timestamp": "2020-08-27T15:09:14,203Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "720985957be6", "message": "version[7.7.1], pid[6], build[default/docker/ad56dce891c901a492bb1ee393f12dfff473a423/2020-05-28T16:30:01.040088Z], OS[Linux/4.19.76-linuxkit/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/14.0.1/14.0.1+7]" }
{"type": "server", "timestamp": "2020-08-27T15:09:20,797Z", "level": "INFO", "component": "t.b.r.b.LogPluginBuildInfoMessage$", "cluster.name": "docker-cluster", "node.name": "720985957be6", "message": "Starting ReadonlyREST plugin v1.22.1 on ES v7.7.1" }

Using:

docker run -d -p 9202:9200 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.7.1

and config like the one above. Everything started fine.

Maybe you are able to prepare docker image, so I can easily reproduce your issue. I’m not able to do it.

Okay, here is the whole docker file, im just adding volumes and user with certain ID for our environment

FROM docker.elastic.co/elasticsearch/elasticsearch:7.7.1
 
# Adding tech user & directories
USER root
RUN userdel elasticsearch && \
    groupadd -r techali -g 10000 && \
    useradd -s /bin/bash -m -d /usr/share/elasticsearch -u 1000 -r -g techali techali > /dev/null 2>&1 && \
    ln -s /usr/share/elasticsearch /opt/elasticsearch && \
    mkdir -p /var/log/insightsre/ && \
    mkdir -p /var/log/insightsre/elasticsearch && \
    mkdir -p /opt/elasticsearch/config/default && \
    mv /opt/elasticsearch/config/jvm.options /opt/elasticsearch/config/default/jvm.options && \
    mv /opt/elasticsearch/config/elasticsearch.yml /opt/elasticsearch/config/default/elasticsearch.yml && \
    mv /opt/elasticsearch/config/log4j2.properties /opt/elasticsearch/config/default/log4j2.properties && \
    chown techali:techali -R /usr/share/elasticsearch/ /opt/elasticsearch/ /var/log/insightsre/
 
# Install readonlyrest plugin
USER techali
ENV PATH="/usr/share/elasticsearch/jdk/bin:${PATH}"
COPY --chown=techali:techali ./plugins/readonlyrest-1.20.0_es7.7.1.zip /tmp
RUN elasticsearch-plugin install file:///tmp/readonlyrest-1.20.0_es7.7.1.zip -b && rm /tmp/readonlyrest-1.20.0_es7.7.1.zip
 
# Copy configurations
COPY --chown=techali:techali ./config/jvm.options /opt/elasticsearch/config/jvm.options
COPY --chown=techali:techali ./config/elasticsearch.yml /opt/elasticsearch/config/elasticsearch.yml
COPY --chown=techali:techali ./config/readonlyrest.yml /opt/elasticsearch/config/readonlyrest.yml
COPY --chown=techali:techali ./config/log4j2.properties /opt/elasticsearch/config/log4j2.properties
 
# Prepare needful volumes
VOLUME /opt/elasticsearch/config/keystore.jks
VOLUME /opt/elasticsearch/config/truststore.jks
VOLUME /opt/elasticsearch/data
VOLUME /var/log/insightsre/elasticsearch
 
# Start elasticsearch
CMD ["/bin/bash", "-c", "elasticsearch 2>&1 >> /dev/null"]

what about form like this one: https://github.com/beshu-tech/rorproxy_example?

I’d like to call:

  • docker build
  • docker run

and see results. The above doesn’t help me much

docker build --rm . -t elasticsearch:7.7.1
 
docker run -d --restart unless-stopped -p 9200:9200 -p 9300:9300 \
    --env-file /opt/elasticsearch/env \
    --ulimit nofile=65536:65536 --ulimit memlock=-1:-1 \
    -v /opt/elasticsearch/certs/keystore.jks:/opt/elasticsearch/config/keystore.jks:ro \
    -v /opt/elasticsearch/certs/truststore.jks:/opt/elasticsearch/config/truststore.jks:ro \
    -v /opt/insightsre/elasticsearch:/opt/elasticsearch/data \
    -v /var/log/insightsre/elasticsearch:/var/log/insightsre/elasticsearch \
    --name elasticsearch \
    elasticsearch:7.7.1

env file is containing all the password needed to configure block and certificates, there are volume for logs, for data and also im using volumes to connect certificates.

all good, but IMO the critical part are files:

  • jvm.options
  • elasticsearch.yml
  • readonlyrest.yml
  • log4j2.properties

As I said before - it works for me, so for sure the problem is configuration or starting options.

ror.zip (8.9 KB)

Ok, i uploaded every config for elasticsearch, you can check, there is also issue with _cat/indices with the same cofiguration already reported on https://forum.readonlyrest.com/t/cat-indices-not-working-again/1627/10, but this is related to the same cluster and same configuration.

Hopefully we can resolve both.

User alibaba is only see the alibaba indices in the _cat/index, but in monitoring he can see every index, but if i use index pattern on lets say logs indices, then he can browse the documents normally, its seems that only _cat/indices is not working correctly.

Its first time i have two bugs in one configuration

Any updates ? Our development is blocked by this bug.

Hi @Sinedko at the moment our engineers are all busy working on roadmap and ROR Enterprise customers’ support cases. We only provide support to Free and PRO users on a best effort basis.

Sure, understood, i will explain this to the customer and try to convince him to purcharse Enterprise version, because of the support and cool kibana features.

Thank you.

Hi Simone.
I am heading the Information Retrieval Solutions team for a large Insurance company and as Sinedko explained to you we are currently using the Free version.
Now we are evaluating to buy the Enterprise license but before to make the decision it will help us to have more clarity on the issue Sinedko showed you, I see that you put free version under low priority but the issue is preventing us to move forward with the evaluation of the licensed one.
In order to give you more context: we are going to have different deployment: one already in place that requires LDAP and we are fine, one in progress that requires JWT and here we have the issue.
If at the end you will find the time to give us some heads up, it will be very appreciated.

Beside that, as I said we are in the process to buy the Enterprise license, for a couple of time I submitted a request on your portal to get a quote and be in contact for further questions but none replied to us.
In that respect, I would ask you if you can contact me, based on the request I already sent you or directly getting my contact from this blog.

Thank you a lot.

Kind regards,
Paolo.

Hi Paolo, sorry about the inconvenience, turns out our contact form had an issue. Just answering this thread for completeness, as we already have an email dialogue ongoing. Let’s continue over there.

Hello just for info, this issues is also present on 1.22.1, also that _cat/indices and _cat/aliases, i just updated the version and the bug is still there. Its quite annonying, because i cannot check aliases or all indices that certain user have.

We are considering to buy license but if those issues are present we are not very keen to do it. Hope you understand us.

Hi @Sinedko. We have scheduled the _cat/indices fix for the next week.

Please check out this build, because I think _cat/aliases may be already solved:

https://readonlyrest-data.s3.amazonaws.com/build/1.23.0-pre7/readonlyrest-1.23.0-pre7_es7.7.1.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA5SJIWBO54AGBERLX/20200916/eu-west-1/s3/aws4_request&X-Amz-Date=20200916T190734Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=f38abd79039f658be552e7b042b97fd4b57a699c583e1917d80b7b0e10e7c1f5

i tested it now, but no changes, the _cat/aliases still respond with blank response, but the good thing is the JWT issue is solved, elastic started without any issues with JWT block

ok, i was happy, but the issue is there again, i did only one restart of es bevause of some change in readonlyrest and it appears again, so i cannot start es again when the jwt block is present

ok, noted. As I said - next week someone’s going to look at it

@Sinedko this weekend is release time for us, we will have this bumped in priority first thing next week.
Thanks for the patience.

Hi @Sinedko, I’ve been assigned to work on your issue with JWT token. I tried to reproduce this issue with provided Dockerfile and configuration files, unfortunately without any success. readonlyrest.yml had commented out JWT TEST section, but even after uncommenting it everything worked fine for me. Because some parts were still missing, like for example env file, I had to fill these gaps with some dummy data. I thought about how can we separate other factors and at the same time work on the same configuration. Because you are already using docker I created self-sufficient docker compose bundle based on what you sent. If you aren’t familiar with docker compose and you don’t have it on your system here’s page with instructions how to install it https://docs.docker.com/compose/install/. Please run ROR from this bundle and check if it still hangs on Loading ReadonlyREST settings from file:.

Here are instructions how to run it:

  1. Download this bundle ror-dc-bundle.zip (31.1 KB). It contains everything you sent + dummy data in env file + generated certs.
  2. Extract this archive in some convenient for you directory.
  3. Copy readonlyrest-1.23.0-pre7_es7.7.1.zip into plugins directory of extracted archive. It’s the same build coutoPL mentioned above. https://readonlyrest-data.s3.amazonaws.com/build/1.23.0-pre7/readonlyrest-1.23.0-pre7_es7.7.1.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA5SJIWBO54AGBERLX/20200916/eu-west-1/s3/aws4_request&X-Amz-Date=20200916T190734Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=f38abd79039f658be552e7b042b97fd4b57a699c583e1917d80b7b0e10e7c1f5
  4. Open terminal and go to root directory of extracted archive, where docker-compose.yml file is located.
  5. Run docker-compose up -d. It will build image using Dockerfile and run it in the background.

You don’t have to create data and log directory, they will be created by docker compose in the same directory where docker-compose.yml file is located. Additionally your previous data and logs won’t be overwritten as this compose is configured to work and modify data only inside of directory where docker-compose.yml file is located.

After you run this bundle please check and let me know if ES still hangs on Loading ReadonlyREST settings from file:. Please also let me know if you have additional questions.

Hello, thank you for your reply.

Basically we are using the same config, except that i dont use docker compose so i used my docker-run script.

If i try to first time run it, then its fine, but if i do the restart of the elasticsearch with deleteing and starting the container again, then it happend, its stuck forever on loading readonlyrest configuration.

Its very confusing because sometimes its working and sometime its not, but right now i reproduced it, firstime i run the es with this config, everything fine, then i deleted container and recreate again, and the issue appears again and now after restart its still stuck, i dont understand what is happening.

After the stuck i removed the jwt acl block, then rebuilded image and run it again, its working fine, then i added the jwt block again, rebuild, run and its still stuck, i dont really understand why the first run was succesfull.

I tried to restart docker aswell, its still stuck. I tried to restart the machine, still stuck.

Docker host versions:

docker --version
Docker version 19.03.12, build 48a66213fe

uname -a
Linux ire-dev-elasticsearch-01 4.15.0-117-generic #118-Ubuntu SMP Fri Sep 4 20:02:41 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Thank you for explanation, but I still weren’t able to reproduce this issue. I will try to write script to test it on your machine. Can you tell me in the meantime if you are able to install docker compose on your system? It makes working with docker a little easier, but If it’s not possible I will try to manage with just docker.