It's not new to anyone that has changed the way we ship software. The primary goal of using is containerization, and that is to have a consistent environment for your application and does not depend on the host machine where it runs. Docker Docker Also, is becoming a popular architectural choice for writing services. You can check this to get a glance of how to implement a service in . gRPC quick start gRPC Golang In this article, we'll see how to write and run a service with . We'll also cover a lot of cool and useful things, like: gRPC Docker How to compile files without the burden of installing locally; .proto protoc How we can easily use for parameterizing our application; environment variables How to write a so we end up with a small, optmized image; multistage build Docker How to easily invoke the service without having to manually write a client for it. gRPC Golang Let's go! The proposed application We'll write a service that is used to retrieve a number of random poetries. In order to achieve that, we'll use , an awesome free API for internet poets. gRPC PoetryDB We can see a simple to show how it will work: sequence diagram Protobuf Here's our file where we define the service, 'poetry.proto': .proto syntax = "proto3" package proto; option go_package = "bitbucket.org/tiagoharris/docker-grpc-service-tutorial/proto/poetry"; message Poetry { string title = 1; string author = 2; repeated string lines = 3; int32 linecount = 4; } message RandomPoetriesRequest { int32 number_of_poetries = 1; } message PoetryList { repeated Poetry list = 1; } service ProtobufService { rpc RandomPoetries(RandomPoetriesRequest) returns (PoetryList); } Now we need to compile it in order to have: Code for populating, serializing, and retrieving 'RandomPoetriesRequest' and 'PoetryList' message types; Generated client and server code. We compile it using . Generally we would need to install it, but a better idea would be to use a image that has installed, pretty much similar to what we do when we want to use for example; instead of installing it, we could use a image for it. protoc Docker protoc MySQL Docker And is exactly what we need. docker-protoc Here's our Makefile target that we use to invoke it so we compile the file: .proto .PHONY: proto ## proto: compiles .proto files proto: @ docker run -v $(PWD):/defs namely/protoc-all -f proto/poetry.proto -l go -o . --go-source-relative The '--go-source-relative' option is to keep the generated 'poetry.pb.go' file at the same folder of our file 'poetry.proto'. .proto When you call it for the first time, it will download 'namely/protoc-all:latest' image and then will compile our file: Docker .proto $ make proto Unable to find image 'namely/protoc-all:latest' locally latest: Pulling from namely/protoc-all 72a69066d2fe: Pull complete 92b40fad93be: Pull complete 6c681a0a5896: Pull complete ebc1d0ae2fce: Pull complete 8419d9b6e1d6: Pull complete bea3673d63cb: Pull complete c4795970891a: Pull complete a07bfba13570: Pull complete 390910a84268: Pull complete 3b0c06e97c77: Pull complete 02fad91bea96: Pull complete 784aa2673488: Pull complete c5446e8648ec: Pull complete f3170de720de: Pull complete dbd0d73172b5: Pull complete 3516e04721f7: Pull complete b91f69a87fb4: Pull complete 37490bcef5e6: Pull complete fd5de9fd6a61: Pull complete 35f2a04b2c22: Pull complete 075200f557a8: Pull complete 017c387ae8e9: Pull complete Digest: sha256:5406210e1dc68ffe4f36fa1ee98214bb50614d3a44428bf33ffca427079dd3d2 Status: Downloaded newer image for namely/protoc-all:latest Reading environment variables Before implementing our service, let's make an engineering decision about parameterizing the application. A good idea would be to parameterize the base for , as well as how many seconds we want to wait until an HTTP timeout occur. So how can we achieve that in a safe, clean way? gRPC URL PoetryDB Using is a common technique, right? A good solution would be: environment variables Defining the variables in some sort of file; Reading this file and exporting those values as ; environment variables Get those into a so we can easily use them. environment variables struct Godotenv is a package that solves the two bullet points defined above. We define our variables in an '.env' file and then we invoke it to export all those values as . Godotenv Golang environment variables Here's our 'config.env' file: POETRYDB_BASE_URL=https://poetrydb.org POETRYDB_HTTP_TIMEOUT=10 Envconfig is a package that solves the last bullet point: it encapsulates that are correctly exported into a . Envconfig Golang environment variables struct Here's 'configreader/config_reader.go', which we use to get a configuration fulfilled with : struct environment variables // Copyright (c) 2022 Tiago Melo. All rights reserved. // Use of this source code is governed by the MIT License that can be found in // the LICENSE file. package configreader import ( "github.com/joho/godotenv" "github.com/kelseyhightower/envconfig" "github.com/pkg/errors" ) // These global variables makes it easy // to mock these dependencies // in unit tests. var ( godotenvLoad = godotenv.Load envconfigProcess = envconfig.Process ) // GoDotEnv is an interface that defines // the functions we use from godotenv package. // It enables mocking this dependency in unit testing. type GoDotEnv interface { Load(filenames ...string) (err error) } // EnvConfig is an interface that defines // the functions we use from envconfig package. // It enables mocking this dependency in unit testing. type EnvConfig interface { Process(prefix string, spec interface{}) error } // Config holds configuration data. type Config struct { PoetrydbBaseUrl string `envconfig:"POETRYDB_BASE_URL" required:"true"` PoetrydbHttpTimeout int `envconfig:"POETRYDB_HTTP_TIMEOUT" required:"true"` } // ReadEnv reads envionment variables into Config struct. func ReadEnv() (*Config, error) { err := godotenvLoad("configreader/config.env") if err != nil { return nil, errors.Wrap(err, "reading .env file") } var config Config err = envconfigProcess("", &config) if err != nil { return nil, errors.Wrap(err, "processing env vars") } return &config, nil } gRPC service implementation Now that we compiled our file 'poetry.proto', we'll write a server that implements the service defined in 'poetry.pb.go': .proto gRPC // Copyright (c) 2022 Tiago Melo. All rights reserved. // Use of this source code is governed by the MIT License that can be found in // the LICENSE file. package server import ( "context" "encoding/json" "fmt" "net" "bitbucket.org/tiagoharris/docker-grpc-service-tutorial/configreader" "bitbucket.org/tiagoharris/docker-grpc-service-tutorial/poetrydb" poetry "bitbucket.org/tiagoharris/docker-grpc-service-tutorial/proto" "github.com/pkg/errors" "google.golang.org/grpc" "google.golang.org/grpc/reflection" "google.golang.org/protobuf/encoding/protojson" ) // These global variables makes it easy // to mock these dependencies // in unit tests. var ( netListen = net.Listen configreaderReadEnv = configreader.ReadEnv jsonMarshal = json.Marshal protojsonUnmarshal = protojson.Unmarshal ) // Server defines the available operations for gRPC server. type Server interface { // Serve is called for serving requests. Serve() error // GracefulStop is called for stopping the server. GracefulStop() // RandomPoetries returns a random list of poetries. RandomPoetries(ctx context.Context, in *poetry.RandomPoetriesRequest) (*poetry.PoetryList, error) } // server implements Server. type server struct { listener net.Listener grpcServer *grpc.Server poetryDb poetrydb.PoetryDb } func (s *server) Serve() error { return s.grpcServer.Serve(s.listener) } func (s *server) GracefulStop() { s.grpcServer.GracefulStop() } // NewServer creates a new gRPC server. func NewServer(port int) (Server, error) { server := new(server) listener, err := netListen("tcp", fmt.Sprintf(":%d", port)) if err != nil { return server, errors.Wrap(err, "tcp listening") } server.listener = listener config, err := configreaderReadEnv() if err != nil { return server, errors.Wrap(err, "reading env vars") } server.poetryDb = poetrydb.NewPoetryDb(config.PoetrydbBaseUrl, config.PoetrydbHttpTimeout) server.grpcServer = grpc.NewServer() poetry.RegisterProtobufServiceServer(server.grpcServer, server) reflection.Register(server.grpcServer) return server, nil } func (s *server) RandomPoetries(ctx context.Context, in *poetry.RandomPoetriesRequest) (*poetry.PoetryList, error) { pbPoetryList := new(poetry.PoetryList) poetryList, err := s.poetryDb.Random(int(in.NumberOfPoetries)) if err != nil { return pbPoetryList, errors.Wrap(err, "requesting random poetry") } json, err := jsonMarshal(poetryList) if err != nil { return pbPoetryList, errors.Wrap(err, "marshalling json") } err = protojsonUnmarshal(json, pbPoetryList) if err != nil { return pbPoetryList, errors.Wrap(err, "unmarshalling proto") } return pbPoetryList, nil } Running it Here's the related targets in to run the server: Makefile .PHONY: build ## build: builds server's binary build: @ go build -a -installsuffix cgo -o main . .PHONY: run ## run: runs the server run: build @ ./main So let's run the server: $ make run GRPC SERVER : 2022/03/21 10:07:14.846821 main.go:19: main: Initializing GRPC server GRPC SERVER : 2022/03/21 10:07:14.847286 main.go:32: main: GRPC server listening on port 4040 Invoking the gRPC service Now the cool part: what if we wanted a nice, clean tool like (used to test APIs) to makes it easy to call services? Postman REST gRPC is here to help. You just browse your files and you'll be ready to invoke it: Bloomrpc .proto And here you go. We've asked one random poetry, and now we can appreciate it. Time to Dockerize it! Now that our app is working, let's bake a image for it. Docker Here's our : Dockerfile FROM golang:alpine # Install git and ca-certificates (needed to be able to call HTTPS) RUN apk --update add ca-certificates git # Move to working directory /app WORKDIR /app # Copy the code into the container COPY . . # Download dependencies using go mod RUN go mod download # Build the application's binary RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags '-extldflags "-static"' -o main . # Command to run the application when starting the container CMD ["/app/main"] Notice: we're installing git because tooling uses it, otherwise we would get an error "go: missing Git command. See https://golang.org/s/gogetcmd"; Golang we're installing 'ca-certificates' because uses , otherwise we would get an error "certificate signed by unknown authority". PoetryDB HTTPS Building the image Now let's build it. Here's the target: Makefile .PHONY: build-docker-image ## build-docker-image: builds the docker image build-docker-image: @ docker build . -t docker-grpc-service-tutorial Invoking it: $ make build-docker-image Sending build context to Docker daemon 13.71MB Step 1/7 : FROM golang:alpine ---> 0e3b02146c47 Step 2/7 : RUN apk --update add ca-certificates git ---> Using cache ---> c326d9aa8cfc Step 3/7 : WORKDIR /app ---> Using cache ---> 6c485ff6b69d Step 4/7 : COPY . . ---> 9af131a39537 Step 5/7 : RUN go mod download ---> Running in a644255b6578 Removing intermediate container a644255b6578 ---> 3b43ba797d11 Step 6/7 : RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags '-extldflags "-static"' -o main . ---> Running in 01b6ce6172a9 Removing intermediate container 01b6ce6172a9 ---> b1eaebffb306 Step 7/7 : CMD ["/app/main"] ---> Running in e0ed88e2a687 Removing intermediate container e0ed88e2a687 ---> edb7869f01a6 Successfully built edb7869f01a6 Successfully tagged docker-grpc-service-tutorial:latest Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them Running it as a Docker container Here's the target: Makefile .PHONY: build-docker-image ## build-docker-image: builds the docker image build-docker-image: @ docker build . -t docker-grpc-service-tutorial .PHONY: run-docker ## run-docker: runs the server as a Docker container run-docker: build-docker-image @ docker run -p 4040:4040 docker-grpc-service-tutorial Invoking it: $ make run-docker Sending build context to Docker daemon 13.71MB Step 1/7 : FROM golang:alpine ---> 0e3b02146c47 Step 2/7 : RUN apk --update add ca-certificates git ---> Using cache ---> c326d9aa8cfc Step 3/7 : WORKDIR /app ---> Using cache ---> 6c485ff6b69d Step 4/7 : COPY . . ---> 95f88bbbc63e Step 5/7 : RUN go mod download ---> Running in 227656a7a691 Removing intermediate container 227656a7a691 ---> b6765354b254 Step 6/7 : RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags '-extldflags "-static"' -o main . ---> Running in a8e881248c52 Removing intermediate container a8e881248c52 ---> a05de0412553 Step 7/7 : CMD ["/app/main"] ---> Running in e0009bc99088 Removing intermediate container e0009bc99088 ---> 351069eab03d Successfully built 351069eab03d Successfully tagged docker-grpc-service-tutorial:latest Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them GRPC SERVER : 2022/03/21 13:30:18.404250 main.go:19: main: Initializing GRPC server GRPC SERVER : 2022/03/21 13:30:18.404646 main.go:32: main: GRPC server listening on port 4040 And then you can invoke the service via like we did before. Bloomrpc Multistage build One of 's best practice is keeping the image size small, by having only the binary file then we make our image even smaller from the previous one. To achieve this we will use a technique called which means we will build our image with multiple steps. Docker multistage build Here's our .multistage: Dockerfile FROM golang:alpine AS builder # Install git and ca-certificates (needed to be able to call HTTPS) RUN apk --update add ca-certificates git # Move to working directory /app WORKDIR /app # Copy the code into the container COPY . . # Download dependencies using go mod RUN go mod download # Build the application's binary RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags '-extldflags "-static"' -o main . # Build a smaller image that will only contain the application's binary FROM scratch # Move to working directory /app WORKDIR /app # Copy certificates COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt # Copy application's binary COPY --from=builder /app . # Command to run the application when starting the container CMD ["./main"] Here are the targets: Makefile .PHONY: build-docker-image-multistage ## build-docker-image-multistage: builds a smaller docker image build-docker-image-multistage: @ docker build -f Dockerfile.multistage . -t docker-grpc-service-tutorial .PHONY: run-docker-multistage ## run-docker-multistage: runs the server as a Docker container, using the smaller image run-docker-multistage: build-docker-image-multistage @ docker run -p 4040:4040 docker-grpc-service-tutorial Notice that we can name whatever we want it, as long as we speficy if via '-f' flag. Dockerfile Invoking it: $ make run-docker-multistage Sending build context to Docker daemon 13.71MB Step 1/11 : FROM golang:alpine AS builder ---> 0e3b02146c47 Step 2/11 : RUN apk --update add ca-certificates git ---> Using cache ---> c326d9aa8cfc Step 3/11 : WORKDIR /app ---> Using cache ---> 6c485ff6b69d Step 4/11 : COPY . . ---> 8be509098958 Step 5/11 : RUN go mod download ---> Running in 776490901c8e Removing intermediate container 776490901c8e ---> ec0d94130a65 Step 6/11 : RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags '-extldflags "-static"' -o main . ---> Running in fa2e87f052ad Removing intermediate container fa2e87f052ad ---> 57680526aaa1 Step 7/11 : FROM scratch ---> Step 8/11 : WORKDIR /app ---> Running in 0cc6905bb002 Removing intermediate container 0cc6905bb002 ---> e41a9cb16982 Step 9/11 : COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt ---> 8429312feec5 Step 10/11 : COPY --from=builder /app . ---> 3d1c20349d38 Step 11/11 : CMD ["./main"] ---> Running in a4a88a400a96 Removing intermediate container a4a88a400a96 ---> 0d27b2b85769 Successfully built 0d27b2b85769 Successfully tagged docker-grpc-service-tutorial:latest Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them GRPC SERVER : 2022/03/21 13:37:50.456686 main.go:19: main: Initializing GRPC server GRPC SERVER : 2022/03/21 13:37:50.457085 main.go:32: main: GRPC server listening on port 4040 And then you can invoke the service via like we did before. Bloomrpc Conclusion In this article, we've covered a lot of nice things when building a service in from scratch: gRPC Golang How to read into a so we can easily use them; environment variables struct How to delegate to an external image to compile our files; Docker .proto How to easily invoke service via ; gRPC Bloomrpc How to create a image using . Docker multistage build Download the source Here: https://bitbucket.org/tiagoharris/docker-grpc-service-tutorial/src/master/ Also Published Here
