How to Migrate a Go API to Serverless (in under 10 mins)

I’m going to step you through the process converting an existing Go to and deploying it to to AWS Lambda & API Gateway with . The whole process should take under 10 minutes. Let’s get started! API serverless AWS Severless Application Model (SAM) 1. Setup Our example API uses the so let’s install that first. [HttpRouter](https://github.com/julienschmidt/httprouter) package $ go get github.com/julienschmidt/httprouter We have a single HTTP handler defined that will return a 200 HTTP response with the body . ok # handlers.gopackage main import "net/http" func HealthHandler(w http.ResponseWriter, r *http.Request) {w.WriteHeader(http.StatusOK)w.Write([]byte("ok"))} Our entrypoint to the application, the function, attaches the to the route and listens for HTTP requests on port . main HealthHandler /healthz 8080 # main.gopackage main import ("fmt""log""net/http""github.com/julienschmidt/httprouter") const (serverPort = 8000) func main() {router := httprouter.New()router.Handler("GET", "/healthz", http.HandlerFunc(goserverlessapi.HealthHandler)) fmt.Printf("Server listening on port: %d\\n", serverPort) log.Fatal(http.ListenAndServe(fmt.Sprintf(":%d", serverPort), router), nil) } Let’s build and run this locally to check everything’s working okay. $ go build -o go-serverless-api && ./go-serverless-apiServer listening on port: 8080 2. Convert Application Code to Serverless In order to deploy to a serverless backend we need to be able to handle requests from AWS Lambda. Lambda functions have a different signature to . Imagine if we had not one but hundreds in our application. We would have to manually update all the functions and re-write tests and in doing so we would forfeit our ability to deploy to non-serverless backends. regular HTTP handlers There is a solution which avoids all the problems just listed. We will create a modified entrypoint just for AWS Lambda. Using the we will swap out ’s for which will convert the payload that AWS Lambda provides into the type that HTTP handlers accept. [gateway](https://github.com/apex/gateway) package net/http ListenAndServe gateway.ListenAndServe [*http.Request](https://godoc.org/net/http#Request) In order to implement this second entrypoint we need to reorganise our project. We will create a directory for the AWS Lambda one and move our original entrypoint to a new folder too. # original entrypoint moved to new location$ mkdir -p cmd/go-serverless-api # new entrypoint for lambda$ mkdir -p cmd/go-serverless-api-lambda We will copy the file into each of these new directories and then remove it from the root of our project. main.go $ cp main.go cmd/go-serverless-api$ cp main.go cmd/go-serverless-api-lambda$ rm main.go The package in the root of our project is no longer going to be the package (the one Go uses to run your application). We will rename it to so that we can import it as a library into our new entrypoints which will both become packages. main goserverlessapi main $ grep -l 'package main' *.go | xargs sed -i 's/package main/package goserverlessapi/g' After a simple find and replace on the Go files in the root of your project your should look like this. handlers.go # handlers.gopackage goserverlessapi import "net/http" func HealthHandler(w http.ResponseWriter, r *http.Request) {w.WriteHeader(http.StatusOK)w.Write([]byte("ok"))} Now we need to update our original entrypoint to import the package and access the function as an export from that. Note that you will need to modify the import path of the package to match that of your project root’s location in your . The correct path for the used in this tutorial is . goserverlessapi HealthHandler goserverlessapi $GOPATH example application on github github.com/techjacker/go-serverless-api # cmd/go-serverless-api/main.gopackage main import ("fmt""log""net/http" "github.com/julienschmidt/httprouter""github.com/techjacker/go-serverless-api") const (serverPort = 8080) func main() {router := httprouter.New()router.Handler("GET", "/healthz", http.HandlerFunc(goserverlessapi.HealthHandler)) fmt.Printf("Server listening on port: %d\\n", serverPort) log.Fatal(http.ListenAndServe(fmt.Sprintf(":%d", serverPort), router), nil) } Next update the AWS Lambda entrypoint. We are going to use the package which is a drop-in replacement for Go net/http when running on AWS Lambda & API Gateway so let’s install it first. [gateway](https://github.com/apex/gateway) $ go get github.com/apex/gateway Then update the AWS Lambda entrypoint file to the following. # cmd/go-serverless-api-lambda/main.gopackage main import ("log""net/http" "github.com/apex/gateway" "github.com/julienschmidt/httprouter" "github.com/techjacker/go-serverless-api" ) func main() {router := httprouter.New()router.Handler("GET", "/healthz", http.HandlerFunc(goserverlessapi.HealthHandler))log.Fatal(gateway.ListenAndServe("", router), nil)} We have added to our imports and swapped it out for . The port value is redundant in the Lambda context and the package discards it so we can safely remove the port constant and replace it with an empty string. In addition we have included our from the package (the package in the root our of project which used to be our main package) as our handler for the path. gateway http.ListenAndServe gateway HealthHandler go-serverless-api /healthz 3. Build and Run Locally Let’s build and run our original HTTP API again. $ go build \-o go-serverless-api \./cmd/go-serverless-api $ ./go-serverless-apiServer listening on port: 8080 Open another terminal window and query it. $ curl -s http://localhost:8080/healthz ok Everything still works! Let’s do the same with the AWS Lambda version. $ go build \-o go-serverless-api-lambda \./cmd/go-serverless-api-lambda $ ./go-serverless-api-lambda Again, open a new terminal and query it. $ curl -s http://localhost:8080/healthzhttp: error: ConnectionError: HTTPConnectionPool(host='localhost', port=8080): Max retries exceeded with url: /healthz (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused',)) while doing GET request to URL: http://localhost:8080/healthz Don’t worry this error is expected as the AWS Lambda version is not listening for HTTP connections but instead expects to be fed an type. [APIGatewayProxyRequest](https://godoc.org/github.com/aws/aws-lambda-go/events#APIGatewayProxyRequest) 4. Package Application For AWS Lambda Build the binary for linux, the operating system that AWS Lambda uses. $ GOOS=linux go build \-o go-serverless-api-lambda \./cmd/go-serverless-api-lambda AWS Lambda requires that function code be bundled into a zip so let’s go ahead and compress the binary. $ zip go-serverless-api-lambda.zip go-serverless-api-lambda We will be using the created in the final step - deployment. go-serverless-api-lambda.zip 5. Deploy I’ve seen some tutorials that use the to deploy to AWS Lambda using ad hoc commands. This is absolutely the wrong approach to take! You should be automating your infrastructure just like every other aspect of your application. The industry standard for deployment is either or . Both give you a declarative way to build your infrastructure. You save this configuration in YAML/JSON (Cloudformation) or HCL (Terraform) files which you commit to your repository. The problem with doing things this way is that you have to deal with all of the low level details of the stack. It would be nice if we had a way of describing our infrastructure at a high level in under 20 lines of code instead of hundreds. Enter . AWS CLI tools Terraform AWS Cloudformation AWS Severless Application Model (SAM) AWS Severless Application Model (SAM) SAM is a new standard spearheaded by Amazon aimed at making deploying serverless infrastructure simpler and more concise. SAM is an open source specification — . Hopefully other cloud vendors will adopt this in the future and it will become possible to deploy seamlessly to multiple clouds with a single configuration. see the full reference guide Deploy with AWS SAM Add the following YAML file to the root of your project. This is a SAM template that configures an AWS Lambda Function that runs your Go app and deploys it behind an HTTP interface provided by AWS API Gateway. # template.yamlAWSTemplateFormatVersion: '2010-09-09'Transform: 'AWS::Serverless-2016-10-31'Description: 'Boilerplate Go API.'Resources:GoAPI:Type: AWS::Serverless::FunctionProperties:Handler: go-serverless-api-lambdaRuntime: go1.xCodeUri: ./go-serverless-api-lambda.zipEvents:Request:Type: ApiProperties:Method: ANYPath: /{proxy+} The line creates a Lambda function that is handled ( ) by the binary file we built earlier. This Lambda function can respond to any number of events triggered by other AWS services such as those triggered by AWS S3 and Kinesis. The documentation contains the . In our case we want it to respond to HTTP requests via API Gateway therefore we set the event to . SAM implicitly creates an API Gateway for us as part of this which we then configure to respond to any type of HTTP request by setting the method to ). We tell our API to handle all paths below and including the root by adding . Type: AWS::Serverless::Function Handler: go-serverless-api-lambda full list of event of sources Type: Api ANY Path: /{proxy+} We still need to upload the zip containing our Go binary to AWS S3. Ensure you have an S3 bucket created ready to receive our zip. This is a one-time operation you’ll want to do manually. $ aws s3 mb s3://my-bucket The following command will upload the zip and create a packaged SAM template. $ aws cloudformation package \--template-file template.yaml \--s3-bucket my-bucket \--output-template-file packaged-template.yaml You should now have a file pointing to the uploaded zip. packaged-template.yaml # packaged-template.yamlAWSTemplateFormatVersion: '2010-09-09'Transform: 'AWS::Serverless-2016-10-31'Description: 'Boilerplate Go API.'Resources:GoAPI:Type: AWS::Serverless::FunctionProperties:Handler: go-serverless-api-lambdaRuntime: go1.xCodeUri: s3://my-bucket/8982639e71e0d433cd99f9fa4207ecbeEvents:Request:Type: ApiProperties:Method: ANYPath: /{proxy+} Now let’s deploy using this new packaged template. $ aws cloudformation deploy \--template-file packaged-template.yaml \--stack-name go-serverless-api-stack \--capabilities CAPABILITY_IAM The flag is required for cloudformation to create the stack for you as it will involve modifying IAM permissions - AWS mandate you set this explicitly as a safety measure. Under the hood the SAM template is compiled into a regular cloudformation template which is hundreds of lines longer. All of is completely hidden from the user (although you are free to inspect the compiled cloudformation template if you wish). --capabilities CAPABILITY_IAM Setup 6. Test Deployed Serverless API In order to discover the endpoint of our deployed API we need find out the API Gateway REST . id $ aws apigateway get-rest-apis{"items": [{"id": "0qu18x8pyd","name": "go-serverless-api-stack","createdDate": 1523987269,"version": "1.0","apiKeySource": "HEADER","endpointConfiguration": {"types": ["EDGE"]}}]} AWS API Gateway addresses take the following format. https:// .execute-api. .amazonaws.com/ A is Amazon’s term for a deployment. SAM creates two different stages for you: and . Note the title case which is also used in the URLs! I think AWS forgot that everyone calls their test environment not but nevermind! stage Stage Prod Staging Stage So SAM has set up endpoints for us at the following locations. https://0qu18x8pyd.execute-api.eu-west-1.amazonaws.com/Stage https://0qu18x8pyd.execute-api.eu-west-1.amazonaws.com/Prod Let’s invoke our API. $ curl -s https://0qu18x8pyd.execute-api.eu-west-1.amazonaws.com/Prod {"message":"Missing Authentication Token"} No need to panic! This is the when you make a request to the root resource without defining a handler for it. The only handler we have defined is , so let’s try that. standard API Gateway error /healthz $ curl -s https://0qu18x8pyd.execute-api.eu-west-1.amazonaws.com/Prod/healthz ok Voila! Our API is now being powered by a serverless backend. Full code for this tutorial available on . github I’ll be posting more articles on Go and Serverless soon. Follow me on to get notified when I do. twitter Originally published at andrewgriffithsonline.com .