I just recently downloaded readonlyrest-1.37.0_es7.16.2.zip along with the accompanying sha1, and was alarmed to find that the sha1 didn’t match the zip file. We had someone else repeat the process (initiate separate request from readonlyrest.com, get a new email with download links, etc), and we got the same zip file with the same mismatching sha1. This either means that someone has hacked the download, or the wrong sha1 has been published. Given the heightened security hackles after the recent log4j vulnerability, we’d really like to get a clarification on this. Anybody else encounter this issue?
Hello @TomChatt!
This is what I get from our builds bucket (accessed directly).
$ cat readonlyrest-1.37.0_es7.16.2.zip.sha1
a5dac0750432445fc7f08a5688a73d0d5117d2f7
$ shasum readonlyrest-1.37.0_es7.16.2.zip
a5dac0750432445fc7f08a5688a73d0d5117d2f7 readonlyrest-1.37.0_es7.16.2.zip
And this is for another random one, just to be sure.
$ cat readonlyrest-1.37.0_es7.9.3.zip.sha1
2886d415f29b30c7081092dfc9b0fb92114e1b81
$ shasum readonlyrest-1.37.0_es7.9.3.zip
2886d415f29b30c7081092dfc9b0fb92114e1b81 readonlyrest-1.37.0_es7.9.3.zip
I will try to see if the emailer went nuts and report back.
I checked an email and it’s all right.
Can you forward the problematic email you got to support at readonlyrest dot com?
What are you using to calculate the sha1?
Weird. I just forwarded you the email. The link for the sha1 gives the same sha1 as you are showing above, but the zip file we get from the download link does not match. I’ve done it both with certutil on Windows and shasum on Linux and get the same mismatched sha1. Have also tried my process on previous ROR downloads, and those do match, so the process is ok.
certutil -hashfile readonlyrest-1.37.0_es7.16.2.zip sha1
SHA1 hash of readonlyrest-1.37.0_es7.16.2.zip:
7462d715f3b8cb0a4d032751f18f504fbbdf4279
CertUtil: -hashfile command completed successfully.
cat readonlyrest-1.37.0_es7.16.2.zip.sha1
a5dac0750432445fc7f08a5688a73d0d5117d2f7
We’ve had two different people request the download on different days, and both got the same mismatched zip file.
Does the downloaded zip work? You can extract it etc?
I don’t know if the download works. Given that the checksum didn’t match, our security people would be extremely unhappy if I were to extract it, or worse, try running it.
Did you try the link in the email that I forwarded to you? Do you get the same result as we do? Since it seems from your description that the shasum is correct and our download is not, could you forward us a link to the correct download?
@TomChatt the email links didn’t work because they are made to expire after a day or so. Maybe try requesting another email and forwarding it?
Just got a brand new email. Still getting a zip file with the wrong checksum. I just forwarded the new email to support@readonlyrest.com.
1 Like
Email received, downloaded both files and they check all right
$ shasum readonlyrest-1.37.0_es7.16.2.zip
a5dac0750432445fc7f08a5688a73d0d5117d2f7 readonlyrest-1.37.0_es7.16.2.zip
$ cat readonlyrest-1.37.0_es7.16.2.zip.sha1
a5dac0750432445fc7f08a5688a73d0d5117d2f7
And it’s a regular zip that works:
$ jar -ft readonlyrest-1.37.0_es7.16.2.zip | head
./
./core-1.37.0.jar
./transport-netty4-client-7.16.2.jar
./audit-1.37.0.jar
...
I’m not sure how I can help you any further. May this be something to do with the zip format and some antivirus? Do you have another computer where you can test this? Maybe a mac, or a linux server where you can directly download with wget?
So I tried going to a different network and doing a “wget” on the download URL from the email. To my amazement, the zip file I pulled down had yet a different sha1 sum, neither matching the published one nor matching the one we’d been getting previously. At first, my suspicion was that our corporate proxies were perhaps doing something to modify the zip contents. But on closer inspection, I discovered that the downloaded zip was significantly different, even though using the same download URL, just at a later time. The zip file that I downloaded today, even though I specifically requested plugin for ES 7.16.2 (and the URL param says esversion=7.16.2), seems to be for ES 7.16.3. This difference manifests in the names of two of the contained files (readonlyrest-1.37.0_es7.16.3.jar and transport-netty-client-7.16.3.jar) as well as the elasticsearch-version property inside the plugin-descriptor.properties file. In addition, although the name of audit-1.37.0.jar didn’t change, the one I pulled down today had files all dated Jan 14, and was different from the previous one. I’m not sure what to make of all of that, except that the same download URL definitely is not delivering the same content one day to the next.
1 Like
OK maybe I understood. In our API, we look for “esVersion” (case sensitive) query parameter. If not found, we default to the latest ES version (7.16.3 now).
In your test, you are querying for “esversion”.
Ah, good catch, my apologies. I was tricked by Outlook. When I look at the email and hover over the link to view the link target, it displays the link in all lower case! However now I see when I look at the actual HTML source of the email, it is indeed “esVersion” and not “esversion”. When I correct it, I am able to download the right zip file, and the checksum matches the published checksum. I was also able to confirm the same result from home.
We can conclude that the checksum problem appears to arise from something probably in our main corporate proxy that is altering the zip file. I was also able to verify that the alteration did not appear significant. Running separate checksums on each of the unzipped jars revealed no difference.
Thanks for your help in identifying the error that is on our side.
Great, thanks for confirming!
You know, I decided I will make the query parameters case insensitive in our API. I had no idea things like these can happen, especially on a very popular Email client.
And if it happened to you, it definitely has happened to others who didn’t bother investigating as thoroughly as you did.
Kudos for reporting this and being willing to investigate this with me for multiple days.
1 Like
The change in the API is now implemented. Closing this.