Build for 5.4.0

sscarduzio · 2017-05-07 16:52:29 UTC

Hey @coutoPL, I’ve been trying to build for 5.4.0, but I get a Gradle issue. Have you tried?

18:49:27.763 [DEBUG] [org.gradle.configuration.project.BuildScriptProcessor] Timing: Running the build script took 35.857 secs
18:49:27.779 [ERROR] [org.gradle.BuildExceptionReporter]
18:49:27.779 [ERROR] [org.gradle.BuildExceptionReporter] FAILURE: Build failed with an exception.
18:49:27.779 [ERROR] [org.gradle.BuildExceptionReporter]
18:49:27.779 [ERROR] [org.gradle.BuildExceptionReporter] * Where:
18:49:27.779 [ERROR] [org.gradle.BuildExceptionReporter] Build file '/me/elasticsearch-readonlyrest-plugin-5.x/build.gradle' line: 40
18:49:27.779 [ERROR] [org.gradle.BuildExceptionReporter]
18:49:27.779 [ERROR] [org.gradle.BuildExceptionReporter] * What went wrong:
18:49:27.779 [ERROR] [org.gradle.BuildExceptionReporter] A problem occurred evaluating root project 'readonlyrest'.
18:49:27.779 [ERROR] [org.gradle.BuildExceptionReporter] > Failed to apply plugin [id 'carrotsearch.randomized-testing']
18:49:27.779 [ERROR] [org.gradle.BuildExceptionReporter]    > Could not generate a proxy class for class com.carrotsearch.gradle.junit4.RandomizedTestingTask.
18:49:27.780 [ERROR] [org.gradle.BuildExceptionReporter]
18:49:27.780 [ERROR] [org.gradle.BuildExceptionReporter] * Try:
18:49:27.780 [ERROR] [org.gradle.BuildExceptionReporter] Run with --stacktrace option to get the stack trace.
18:49:27.781 [LIFECYCLE] [org.gradle.BuildResultLogger]
18:49:27.781 [LIFECYCLE] [org.gradle.BuildResultLogger] BUILD FAILED

Been googling, deleted my .gradle cache and everything. No dice.
Do you have any Gradle tricks?

coutoPL · 2017-05-07 20:41:39 UTC

Hey, I have no idea what this is … but I’ll try to build it on my machine and will see what’s going on. But not today … I will back to the issue tomorrow evening.

coutoPL · 2017-05-08 21:28:14 UTC

have the same error … it’s related to 5.4.0 elasticsearch.gradle build-tools. It looks like a bug. Don’t have workaround at the moment

sscarduzio · 2017-05-09 09:00:41 UTC

I’m more and more uncomfortable with depending on gradle + ES convoluted tooling. Which we are actually not using (as we are happy users of testcontainers).

AFAIK the only thing we are using of es build tools is the making of the zip file. Right?

coutoPL · 2017-05-09 18:55:46 UTC

you are right … maybe it’s time to migrate to sbt & scala and do not develop like animals any more :D:D:D

sscarduzio · 2017-05-10 09:05:42 UTC

Yeah I too, secretly dream about rewriting all this huge pile of Java in 1/10 lines of Scala
Notice how I symbolically marked your previous post as “THE SOLUTION” lol.

But before that, we need segregate the ES glue code really well and expose a neat API we can build on top, eventually in Scala.

Having said that, very soon today I will scan through your amazing PR which - incidentally - address exactly this aspect.

coutoPL · 2017-05-10 11:16:56 UTC

ok, waiting for comments and suggestions.

sscarduzio · 2017-05-10 15:41:15 UTC

I added a few comments, very satisfied overall. Much more structure, tidier everything. Thanks mate

coutoPL · 2017-05-10 17:18:26 UTC

No problem. Nice that I could help
Next step is introducing modules. I don’t feel very comfortable with ES wiring parts, so I’ll create es53x and es52x modules in next PR, configure building and tests and let you port rest of supported versions in the same way. What do you think?

knes1 · 2017-05-12 19:27:33 UTC

I googled and seems that this is somewhat related to ES team using internal non-documented features of Gradle that break depending on which Gradle version you use. I managed to build 5.4.0 (with tests failing due to lack of docker on my system - it seems docker is used for some integration tests). It seems to work when using gradle 3.3, maybe with newer versions as well.

Anyhow here are my steps:

download latest version of Gradle (3.5) and make sure you are using it:
gradle -v
(should show 3.5)
create gradle wrapper that will use Gradle 3.3 for this build:
gradle wrapper --gradle-version 3.3
run build without tests:
./gradlew updateSHAs check assemble -x test

sscarduzio · 2017-05-12 20:39:56 UTC

@knes1 came and saved the day. Quick, give a medal to this guy.

Can you make a pull request to change the gradle version? Travis CI will instantly test the docker stuff for you

sscarduzio · 2017-05-15 10:05:19 UTC

I gave it a shot.
I updated the the gradle wrapper and the dependencies (used in tests).
Now it builds, but tests fail in the Gradle integration (as it was built against 2.13)

ERROR   0.00s | LdapIntegrationSecondOptionTests (suite) <<< FAILURES!
   > Throwable #1: org.gradle.tooling.GradleConnectionException: Could not execute build using Gradle distribution 'https://services.gradle.org/distributions/gradle-3.5-bin.zip'.
   > 	at org.gradle.tooling.internal.consumer.ExceptionTransformer.transform(ExceptionTransformer.java:55)
   > 	at org.gradle.tooling.internal.consumer.ExceptionTransformer.transform(ExceptionTransformer.java:29)
   > 	at org.gradle.tooling.internal.consumer.ResultHandlerAdapter.onFailure(ResultHandlerAdapter.java:41)
   > 	at org.gradle.tooling.internal.consumer.async.DefaultAsyncConsumerActionExecutor$1$1.run(DefaultAsyncConsumerActionExecutor.java:57)
   > 	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
   > 	at org.gradle.internal.concurrent.StoppableExecutorImpl$1.run(StoppableExecutorImpl.java:46)
   > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   > 	at java.lang.Thread.run(Thread.java:745)
   > 	at org.gradle.tooling.internal.consumer.BlockingResultHandler.getResult(BlockingResultHandler.java:46)
   > 	at org.gradle.tooling.internal.consumer.DefaultBuildLauncher.run(DefaultBuildLauncher.java:77)
   > 	at org.elasticsearch.plugin.readonlyrest.utils.gradle.GradleProjectUtils.runTask(GradleProjectUtils.java:56)
   > 	at org.elasticsearch.plugin.readonlyrest.utils.gradle.GradleProjectUtils.assemble(GradleProjectUtils.java:31)
   > 	at org.elasticsearch.plugin.readonlyrest.utils.containers.ESWithReadonlyRestContainer.create(ESWithReadonlyRestContainer.java:67)
   > 	at org.elasticsearch.plugin.readonlyrest.utils.containers.ESWithReadonlyRestContainerUtils.lambda$create$0(ESWithReadonlyRestContainerUtils.java:56)
   > 	at org.elasticsearch.plugin.readonlyrest.utils.containers.MultiContainerDependent.starting(MultiContainerDependent.java:49)
   > 	at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:28)
   > Caused by: org.gradle.api.GradleException: Unable to start the daemon process.
   > This problem might be caused by incorrect configuration of the daemon.
   > For example, an unrecognized jvm option is used.
   > Please refer to the user guide chapter on the daemon at https://docs.gradle.org/3.5/userguide/gradle_daemon.html

See: https://travis-ci.org/sscarduzio/elasticsearch-readonlyrest-plugin/jobs/232346276

@coutoPL is there something to update in GradleProjectUtils ?

knes1 · 2017-05-15 10:12:04 UTC

Ok, I’ve seen you made the changes to the gradle wrapper recently so no need for PR

I have one more small suggestion: Add the version of the gradle to be used for the build in gradle wrapper in gradle.build file. That way you add a bit more explicitness regarding the required gradle version to build the system and, if in the future gradle needs to be updated, you just:

increment the version
run gradle wrapper to install new version of the wrapper.

To do this, add the following to gradle.build:

task wrapper(type: Wrapper) {
 gradleVersion = '3.5' 
}

knes1 · 2017-05-15 10:29:12 UTC

@sscarduzio I’ve seen the same error when I first run the build on my local machine but I’m not seeing it anymore. Since gradle runs a deamon on a first start and keeps it running for purpose of speeding up startup time when buliding again, there could be a chance a deamon from the old version of gradle somehows kept running.

I’m not too familiar with Travis, does it keep some files between the test runs? Since gradle deamon might not make sense in CI testing environment (e.g. you’re starting everything from scratch anyway) maybe you could add a flag --no-daemon to gradle (just see if that would help, or if the issue is somewhere else)? E.g. gradle --no-daemon … in bin/build.sh.

knes1 · 2017-05-15 11:04:47 UTC

It could also be this bug, and it’s triggered by new Gradle version: https://github.com/gradle/gradle/issues/1375

If it’s that - then it fails because tmp directory in CI build environemnt doesn’t exist…

sscarduzio · 2017-05-15 11:09:17 UTC

AFAIK it should not keep anything between builds. I never specified a gradle version for it to use either. Just checked the travis.yml and the travis-ci.com settings. Nothing about gradle or environments.

knes1 · 2017-05-15 11:25:06 UTC

Yeah… I checked out travis config and there’s nothing special in there that looks problematic

As for gradle versions: you did actually implicitly specify gradle version through gradle wrapper. E.g. gradle version you used is embedded in the gradle wrapper (which is a gradle binary that’s packed together with the project so that one can use the same build system everytime consistently without need to actually download gradle - it’s located in ./gradle/wrapper). Up until last update you used 2.13 (see history of gradle-wrapper.properties).

The https://github.com/gradle/gradle/issues/1375 could theoretically still be the issue, although, that would mean that almost none of the gradle projects would work in Travis without special config of temp directory, which I doubt is the case (there’d be far more references to it on the internet if it were).

It’s really strange though that the issue still pops out even when --no-daemon option is used.

sscarduzio · 2017-05-15 11:29:44 UTC

The --no-deamon affects the command line launch, then some of the tests interact with gradle as a library and call the assemble task. And there is where it fails. I’m not sure how to disable daemon from the programmatic side.

https://github.com/sscarduzio/elasticsearch-readonlyrest-plugin/blob/master/src/test/java/org/elasticsearch/plugin/readonlyrest/utils/gradle/GradleProjectUtils.java

knes1 · 2017-05-15 11:33:57 UTC

Aaaa, so tests depend on gradle. This could be the culprit then (in build.gradle):

testCompile 'org.gradle:gradle-tooling-api:2.13’
testRuntime ‘org.gradle:gradle-tooling-api:2.13’

This should probably be updated to 3.5.

sscarduzio · 2017-05-15 11:40:28 UTC

Yes, that’s why I changed it (see the current)
https://github.com/sscarduzio/elasticsearch-readonlyrest-plugin/blob/master/build.gradle