Gradle, Bazel and gRPC: A Song of Ice and Fire

Author profile picture

@AlexeySoshinAlexey Soshin

Really sorry for the cheesy title. No Game of Thrones jokes below. Promise.
So, you’re using Bazel. Maybe because you had a monorepo before. Or maybe you just decided that you have too many modules in your Android project. Or just because it was hard to manage all those microservices you had in your neatly separated repos on GitHub. It doesn’t matter. Now you stuck with Bazel.
One of the things you’ve lost the moment you switched to Bazel was rich ecosystem of plugins that tools like Maven and Gradle provided.
Moreover, Gradle also allowed you writing DSLs and even custom tasks in Kotlin. All that goodness gone for good? Maybe not.
Let’s see how we can take one Gradle plugin, KrotoPlus in our case, and make it work with Bazel.
Before we start, let’s have a few words about KrotoPlus plugin and gRPC in general.
All gRPC libraries take 
.proto
 files as input and output three kinds of classes in your language of choice:
  • Messages
  • Clients
  • Server interfaces
Messages are just your plain data objects, for the sake of this discussion. They would have some fields, and those fields would have types. That’s all you need to know for now.
Clients, also called stubs, are what you use to call your gRPC service. To create a client you would usually specify the host and port of your gRPC server, and then you would call a method, passing it your message of choice.
Servers implement generated server interfaces and then begin to listen on the correct host and port to be able to serve clients.
What you need to take out of this is the follows:
gRPC Library: (proto files) => (generated classes)
Gradle likes to work with a well defined project structure, namely:
your-gradle-project
  src
    main
      proto

When you run your Gradle plugin and it “simply works”, that’s because Gradle was smart enough to recognise your Source Sets and apply plugins to them.
So, in case you had some 
.proto
 files in your 
proto
 directory, our Gradle plugin would detect them, and then generate some Java and Kotlin files in build directory:
your-gradle-project
  build <-- your generated files will appear there
  src
    main
      proto
Bazel, on the other hand, knows nothing of that neat structure.
Moreover, your 
proto
 files would be located outside of 
your-gradle-project
directory, like so:
some-proto-project
   proto <-- some proto files
...
your-gradle-project
  src
    main
      proto <-- empty
...
other-proto-project
  api
    proto <-- more proto files
What we would like to do is to use our Gradle project sitting under Bazel as a function, that would get path to 
.proto
 files, as if they were part of its Source Set, and then generate classes based on those files.
There are a few ways to achieve the behavior with Gradle, but here’s one of them:
import com.google.protobuf.gradle.*
...
protobuf {
    ...
    generatedFilesBaseDir = "$buildDir/generated-sources"
    ...
    dependencies {
        protobuf(files(project.properties["protoDir"].toString()))
    }
    generateProtoTasks {
        ...
    }
}
The important part is the 
dependencies
 block. Here, instead of hardcoding our dependencies as we usually do, we get them from command line:
project.properties["protoDir"].toString()
What that means is that when we run our Gradle wrapper, we now pass relative path to the 
.proto
 files:
./gradlew generateProto -PprotoDir=../path/to/protos
Our Gradle is good and ready to go now. Next, let’s start working with Bazel.
One of your main tools when you want to hack with Bazel will be genrule.
This is your basic way to express that you would like to run something in Bash shell.
But what’s that “something”? It should be 
gradlew
 , same wrapper we used before. But the problem is, Bazel is not aware of it.
So, let’s create a file called 
BUILD
 in the same directory your 
gradlew
 is, and add the following block:
filegroup(
    name = "gradle",
    srcs = ["gradlew"],
)
That way we tell Bazel: hey, that’s a new file you should be aware of.
But if you try to run it like that, Gradle would complain that it can’t find its configuration.
That’s because we only told Bazel to copy 
gradlew
 to its sandbox, not anything else.
Let’s fix that by adding another 
filegroup
 that would hold the configurations:
filegroup(
    name = "gradle_config",
    srcs = ["build.gradle.kts", "krotoPlusConfig.asciipb", "settings.gradle"]
            + glob(["gradle/**/*"])
            + glob(["scripts/**/*"]),
    visibility = ["//visibility:public"]
)
As you can see, we grab 
build.gradle.kts
, as well as configurations files for KrotoPlus, and Gradle wrapper itself, which is located under 
/gradle
directory.
Now, to the rule itself. At first, it will look intimidating, but we’ll break it line by line.
genrule(
    name = "run_gradle",
    cmd = """
    ./$(location gradlew) -p $$(dirname ./$(location gradlew)) generateProto -PprotoDir=../proto/ &&
    cp -R $$(dirname ./$(location gradlew))/build/generated-sources  $(@D)
    """,
    outs = ["build/generated-sources"],
    message = "Generating protos",
    srcs = [],
    tools = [":gradle"] + [":gradle_config"],
)
  • name
     is the label you want to refer to when specifying dependency of your library.
  • cmd
     is the command that would run inside the shell. We’ll break it down later
  • outs
     is what this command produces. Bazel tries to track every output of your script, so you need to tell it what you expect to change in the filesystem after you execute cmd. If directory build/generated-sources is not changed, Bazel will complain.
  • message
     is what will be displayed while you wait
  • srcs
     are the input files you would like to run your script on. Ideally, those would be your 
    .proto
     files. But in this case, we’ll leave them empty, as we use 
    -PprotoDir
     instead
  • tools
     is the list of files you need to use to produce the output. In our case, it’s the Gradle Wrapper and all the configurations.
Now let’s go back to 
cmd
 and understand what’s going on inside.
When you start Bazel, its root is at 
WORKSPACE
 file location, not the directory where 
gradlew
 is located. So, we use 
$(location gradlew)
 to expand label to its relative location. In our example, that would be something like 
./your-gradle-project
Once we located the Gradle Wrapper executable, we need to tell it where to find its configuration. We do that with 
-p
, that’s standard Gradle flag. We know that this directory is again relative to our 
WORKSPACE
, hence 
./$(location gradlew)
. But this time, it must be a directory, and location will expand to path to file, so we use 
$(dirname)
. But Bazel already uses 
$
for 
$(location)
. So we end up with screening the command: 
$$(dirname)
Once we located the configuration, we need to tell Gradle which task to run. That’s 
generateProto
And now we need to point Gradle to the location of our 
.proto
 files. But the context switches here, from Bazel to Gradle, so the path is relative to 
gradlew
 location now: 
-PprotoDir=../proto/
Now Gradle and KrotoPlus work hard to generate our gRPC classes. The problem is, though, they’re generated in the Gradle directory, and not inside Bazel sandbox.
So, we need to copy them. 
cp -R
 is a standard shell command, and we already covered what 
$$dirname()
 stands for. The only part you’re not familiar with yet, is 
$(@D)
That’s the output directory we specified with 
outs
 , and as a result, copies generated files where we want them to be.

Conclusions and next steps

Combining Bazel and Gradle to generate files is possible, if a bit cumbersome.
If you want to go down that path, your next steps should be:
  • Remember to cleanup after yourself. 
    rm -rf $$gradleDir/build
     would be nice
  • You probably want to make a function out of this 
    genrule
    . Syntax stays almost the same, but you need to prefix 
    genrule
     with 
    native
     when inside a function: 
    native.genrule
  • In the long run, using relative paths as we did with 
    -PprotoDir
     isn’t the right choice. Although it’s useful for testing KrotoPlus plugins. Better choice would be to rely on labels.
Hopefully, if you’re already using Bazel, this article will set you on the right track.

Tags

The Noonification banner

Subscribe to get your daily round-up of top tech stories!