TL;DR: Using gRPC with Kubernetes, cluster-internally, is straight-forward. Exposing a gRPC service cluster-externally not so much. Maybe a good practice could be: use gRPC cluster-internally and offer public interfaces using HTTP and/or WebSockets? Starting point was a simple gRPC server example called . This YAGES example shows end-to-end how to develop and deploy a gRPC service using Go and Kubernetes. You can of course your favourite language (as long as it’s by gRPC) rather than Go to implement the service. Now, after the gRPC service as an Kubernetes app, accessing the gRPC service from within the cluster is straight forward enough, for example, with installed: Yet another gRPC echo server (YAGES) use programming supported deploying using a jump pod grpcurl $ grpcurl --plaintext yages.grpc-demo:9000 yages.Echo.Ping{"text": "pong"} OK, cool. How about if I want to access this from outside of the cluster? Well, let’s see … The gRPC blog provides a nice introduction into the topic in the post discussing the options and the docs have some . For a nice intro see also Tom Wilkie’s on the topic. gRPC Load Balancing related info presentation Equipped with the basics I set out to get a setup working with the focus on UX and ease of use. Ideally, it would work out of the box or at least with minimal configuration effort. I had a look at a couple of load balancer/reverse proxy/service mesh options: , three Envoy-based solutions (Ambassador, Contour, and Istio), Linkerd, and Træfik . NGINX Here’s what I found … Ambassador Datawire’s Envoy-based API gateway has first class and we’re also . The UX is good, only thing to be aware of is that it requires a cluster role and respective binding. I haven’t figured out yet how to use it on a namespace-level, that is, in an environment where I don’t have the permission to create cluster-wide resources. Ambassador gRPC support using it in Kubeflow Contour Heptio’s is a Kubernetes ingress controller using Envoy. It seems gRPC support is actively being worked on, I found one that needs to be addressed in order to make it work. Contour issue Linkerd is a CNCF inception project, originally developed by the service mesh pioneers Buoyant. It out of the box. There are very nice blog posts available on the topic— for example from 04/2017 and from 02/2018—but I have yet to try it out. Linkerd supports gRPC A Service Mesh For Kubernetes Part IX: gRPC for fun and profit Building scalable micro-services with Kubernetes, GRPC & Linkerd Træfik is a HTTP reverse proxy and load balancer with . Also on my to do list to try out. Træfik built-in support for gRPC NGINX I think it’s fair to say that NGINX doesn’t need an introduction. Amongst many other things, you can use it as an . One of the things it now also supports is (since 1.13.10/mid-March 2018). I tried patching it on Minikube (which still uses v0.9.0 of the controller) but no luck so far. I also will give it a try in the context of . Kubernetes Ingress controller gRPC GKE Istio Finally , another project from the Cloud Native ecosystem, is a service mesh that uses Envoy by default for the data plane. Seems the community is actively . I’ll give it a try once it’s available. Istio working on gRPC support For completeness sake I’ll also mention that it’s apparently possible to as well to handle gRPC but I didn’t have a closer look at it. Also, there’s , a client-side load balancer/Kubernetes name resolver which sounds like a good option for cluster-internal use cases. use HAProxy sercand/kuberesolver Note that my setup is targeting in Minikube (v0.25) and GKE and my only hard requirement is that it has to work with . Kubernetes 1.9 RBAC enabled Conclusion: I haven’t been able to use any of the above options successfully to expose my little YAGES example to the outside world. I got the farthest with Ambassador so far but not yet able to get to it with . The fact that gRPC requires HTTP/2 seems to be part of the challenge, RBAC another one. grpcurl I have no doubt that, given the ecosystem is moving so fast, the UX/DX around configuring and using gRPC towards cluster-external clients will improve a lot. For now it’s still early days, in my experience. Possible, but not easy in terms of UX/DX. Maybe exposing gRPC to the outside world per se is not a great idea? That is, maybe the good practice we can derive is: use gRPC within the cluster and HTTP and/or WebSockets to communicate with cluster-external clients? I’d love to learn about your experiences in this space and any thoughts or recommendations, here! UPDATE 2018–03-27: since publishing this post I’ve received a great number of valuable feedback and learned about some more things. So thank you everyone who took time to read and follow-up here, much appreciated! Here are two things I’d like to add for now: Buoyant, the makers of Linkerd have a new project and product on the market called . Now, I was aware of it, tried it out myself (super easy to use, check out for example) but so far was not able to find any ongoing work in the context of exposing gRPC services to the outside world. Turns out my dear buddy is now working at Buoyant and in a convo thankfully pointed me to an that captures the essence of it. Thanks! Conduit this video Thomas R issue Another super useful input came in via Twitter: out that there’s another very cool project one can use: — I’ll also be checking out this one, thank you! Andrew Webber pointed nghttpx Ingress Controller Finally, my colleague , our microservices and Istio subject matter expert, made me aware of in OpenShift, leveraging TCP/SNI. Very nice, cheers, another one for my to do list :) Christian Posta custom routes
