Direct Download links

Currently ROR download links are provided only on request, after submission via ROR website. Most of them will be hosted on AWS and the links expire after some time. Why not provide direct download links with a)latest version for specific ES version b)specific ROR version for specific ES version? I know that Simone does not recommend using outdated versions, but if I have done my testing in DEV region with some specific version, I would definitely want to install the same version in my SIT/UAT/PROD, rather than just pulling the latest version everytime.

Given how frequent the fixes come in and this change has to be deployed in multiple servers, I am planning to build some scripts that can automate the part of downloading the latest version, remove old version and install the new version of ROR. This will be partially useful during first time install as most of the work will be on setting up ROR config files. But subsequent updates, in majority of the cases, will just involve updating ROR version with no changes to config files. So I feel that having these fixed download links is going to be useful. The way that I plan to do it is look for files locally first. If not available, then download.

BTW, just FYI, these kind of direct download links are available for both XPack and Searchguard. XPack provides it on their website, where as SG has it hosted on maven repo.


Hey @askids, how is relying on third party links in your automated deployment a good strategy?
I see this very brittle from a reliability standpoint: I might decide to move old builds to another folder in my web space, plus: do you want the reliability of ROR servers to be a variable in the repeatability/immutability of your automation?.

But also from a security standpoint: security software download sites are a very hot target for malware operators.

If I were you, like: all of you, I’d host the tested plugin version in a S3/web space I manage. Or much, much better: create an immutable docker image with the tested plugin, add it to a private docker registry and sleep better nights.

BTW, the best approach I saw was at CERN: they build the plugin from sources (checking out GIT tags) as part of the their deployment strategy. Am I right @psaiz, @schwicke?

What does everyone think?

@sscarduzio when you say 3rd party link, hope that you realize that you are talking about ROR only :smiley: So if we cannot even rely on a secure link from ROR (be it hosted directly on ROR website or AWS link), how can we rely on the ROR plugin itself for security…JK

I am not saying that the script will download in all lanes. In our environment, we don’t even allow internet connectivity from dev servers, forget prod. These kind of upgrades are typically tested starting with a developer machine. So this is where the download option will be very useful and will be in play. Subsequently for the downloaded version, the script will utilize the local available zip file only for that version as that is what gets promoted. So in typical scenario, this upgrade then move on to a proper development server, SIT lane, UAT lane and finally production. So whatever was downloaded and tested in the 1st box is what gets eventually packaged and deployed into higher lanes.

We have currently automated the part of uninstall and installing using the local zip file and can upgrade to any version. But given that we went through so many upgrades starting from 1.16.9 to eventually 1.16.14, with multiple pre-release versions being tested in between, for me it makes sense to have that automated download option that developers could leverage to perform the upgrade on their individual machine.

Also, FYI, xpack plugin installation that is integrated with the windows MSI also downloads directly from Elastic site. I have open feature request with them to provide offline plugin installation capability because that is what we will use in proper server machines and that will simply the upgrade task on actual servers. But that download and install plugin option is still useful, when we have to upgrade and validate the new version on individual developer machines.

lol you have your point, I was just trying to put myself in the shoes of a very paranoid/principled dev.

Did I understand well what you are proposing? The user experience proposed would be:

  1. Download ES version X.Y.Z
  2. bin/elasticearch-plugin install readonlyrest
  3. the plugin tool automatically pulls the latest plugin code for the ES version X.Y.Z
  4. plugin installed, ready to run.

If this is true, as a user, you will have zero control in what ROR version you will be pulling, right? What if you tested 1.16.13 and last night I just released 1.16.14 and all of a sudden you’re promoting code you didn’t test?

How does a x-pack user handle this?

The way that I am building it, I pass the version number as input to my script. So in that version input, I want to provide 2 variants. First one is simple get latest ROR version for ESx.y.z. Second option is more specific ROR 1.2.34 for ESx.y.z. Currently, I am only supporting offline install. So first check, if ROR is installed. If yes, remove it. Then install the new version and restart ES service.

In case of xpack, the msi is specific to a ES version. So the xpack that they download is also specific to that ES version.

With only download option available for xpack plugin install, they also have the same problem like you have described. But on their end, they don’t have separate xpack version just with a security fix. When they release it, they will release all products with the new version.

Does that mean that if a severe security bug gets discovered in x-pack for ES 5.6.4, there’s no way for them to deliver a fix to a 5.6.4 customer other than forcing them to update the whole Elastic stack?

Have you seen how slow are people upgrading their ES clusters? We have customers in 5.1.x, 5.3.x, and refused to deal with a dozen others on 2.x. And these guys do not plan to update their cluster anytime soon. Which I understand, because it’s a extremely difficult to upgrade an ES cluster without downtime.

I really think binding x-pack version tightly to each Elastic Stack release is a bad idea. ReadonlyREST won’t follow this path and all our customers using supported ES versions will be able to receive the latest plugin code.

Another reason we can’t follow the Elastic Stack versioning alone, is that we simply don’t have control on when/how often they’ll release.

About the frequent releases of ROR

I believe ROR is releasing coming out often is symptomatic that we’re working hard on the product. The more customers we onboard, the faster we go due to the great input from very knowledgeable customers like yourself.

Having said that, I recognise that upgrading to a new release implies engineering time on customer side, so we’ll will try to keep it down to max 1 per month. Unless major things need to be fixed.

At least that i what I have understood. If somebody else is currently using or previously used xpack for security, they should be able to confirm or correct my understanding.

1 Like

Also, just wanted to add that it looks like there is some level of tolerance on version mismatch between the products. Some mismatch might just throw a warning and not actually stop the functionality. So technically, its possible for xpack to release a newer version and not needing to upgrade the entire Elastic stack. Again, our focus shouldn’t be on what xpack is doing as it contains many plugins packaged together and not just security. So they might have their own reasons for wanting customer to upgrade and not just related to security. Lets leave it at that as there is no one here to support their viewpoint.

Coming back to the original suggestion, Building from ROR code might not be a feasible option for everyone. Take us for example. We are on Windows Stack, running applications on .Net. We wouldn’t want to be dealing with intricacies of Java, if the build fails or even to set it up for that matter. Its not that it cannot be learnt, but its an unnecessary ask to be able to use ROR.

I would still recommend that you think about providing the direct download link options - one always for latest ROR (for ES X.Y.Z) and second being version specific (ROR for ES X.Y.Z). For some versions, which you want to stop supporting OR has known critical vulnerabilities and want to force users to upgrade to a newer version, you can remove the links.

I don’t mind frequent updates as long as I can automate some or most part of the install process. This might not look very useful during first time setup. But all subsequent updates can be possibly made simple by providing this option. This could also ensure that the ROR customers can think about making the required security upgrade more frequently, if the process is less painful. This way, they are not stuck in some old version with known security vulnerabilities.

Sorry for the late reply. Indeed, we’ve decided to package not only readonlyRest but also all other plugins we use for ES and kibana as rpms. We build them from source. This allows us to control the version across all clusters and on all cluster nodes (and we have many of them).

1 Like

Here it is:



@coutoPL looks like ROR Kibana Free edition, still asks for email id to be passed on as mandatory parameter, where as ROR ES free edition does not ask for it. Do you plan to retain the email check or make it consistent with what you have for ES edition?

@sscarduzio could you take a look, please?

If someone else needs it, I ended up using below link to download the kibana free plugin.

This functionality seems to be broken. I am trying to download older version of the plugin. But its always downloading 1.23.0. Can you please check? This is working fine for Kibana plugin.

yes that thing needs a fix. Will add an internal task for it. In the meantime you can download that build from here

Thanks @sscarduzio. I was trying to troubleshoot the other issue on index patterns. So looks like it may not be related to ROR ES, but actually related to ROR Kibana. Since direct download link working fine for ROR Kibana, i can work with it. Thanks for getting back!

Hi, this is fixed now in ROR devops API.

Looks like it works for pre release version also. Great. Thanks!

Now, the ROR_Kbn link seems to be broken. Getting internal server error.

Internal Server Error

The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

Yes I enabled this for ES plugin only.