Make yourself a Go web server with MongoDb. Go on, Go on, Go on…

I mostly program in c# and node.js, this article is a result of my recent dabbling in go. I set out to create a simple web back end in go to see how it compares with node. I like node because it allows me to create apis extremely quickly, but on the downside most projects depend heavily on external libraries leading to huge code bloat and . potentially malicious code I have created a small app to demonstrate setting up a rest api, testing, hashing user passwords, and storing data in MongoDb, the full code can be viewed here: _go_rest_api - Go web server with mongoDb_github.com eamonnmcevoy/go_rest_api Create and retrieve users from mongoDb Salted passwords REST API — creating and retrieving users The cmd folder — link it all up Authentication (sort of) Package layout In this project I have tried to follow the . ‘Standard package layout’ described by Ben Johnson Here’s what this looks like: go_rest_api /cmd /app -app.go -main.go /pkg /mongo /server /mock -user.go In the folder we have our main application code, this is simply used to wire up our dependencies and start the server. /cmd The folder is where the bulk of our code lives. At the root level we define our domain types and interfaces, our root package will not depend on any other package. /pkg Part 0 — Environment Setup If you haven’t already installed Go, go do that: Install guide If you haven’t already learned the basics of Go, go do that: Go tour Ensure your GOPATH directory is layed out in this format: GOPATH//bin (to output your built executable files)/pkg (to store your projects dependencies)/src (your code goes here) We will use a few external dependencies in this project so lets download them all now. Run the following commands in your GOPATH dir. //requiredgo get gopkg.in/mgo.v2 //MongoDb drivergo get golang.org/x/crypto/bcrypt //password hashing //could live withoutgo get github.com/gorilla/mux //http routergo get github.com/gorilla/handlers //http request logginggo get github.com/google/uuid //generate password salt We could implement this project without Gorilla thanks to Go’s fantastic built in http library. However, once your rest api starts to increase in complexity a more fully featured routing toolkit like Gorilla will come in very handy. Last but not least, install a version of MongoDb and start the server by running on the command line. mongod https://docs.mongodb.com/manual/installation/ Part 1 — Create and retrieve users from Mongo Lets get started, the first thing we need is a User data type. In the folder create the file and define the type and service interface. pkg user.go Next we need to implement our , since we have decided to use MongoDb as our database lets create a package and implement the service in that. UserService mongo Create a folder , and add two files , and pkg/mongo user_model.go user_service.go In this file we define 3 important things The document structure we will be inserting into Mongo — . We are not using here so that we can keep our dependency on confined to the package. userModel root.Model gopkg/mgo.v2/bson mongo . This index allows us to quickly query Mongo on the field and prevents duplicate 's. The index to be applied to our user collection Username Username and functions to convert between and newUserModel toRootUser userModel root.User You may have noticed the obvious security flaw in this model, we are storing passwords in plain text; Don’t worry we will address this with a salted hash in part 2. This is the basis for our implementation. The constructor gets a collection from the parameter and sets up the user index. UserService session Implementing our interface is very simple. The function is pretty self explanatory, convert to and insert into our collection. performs a M for a single user with a matching . Create root.User userModel GetByUsername ongo find operation username Lastly, we will create a file to manage the connection to Mongo. session.go This file defines three simple functions to create, access, and close a Mongo session, as well as get a pointer to a collection from that session. That’s it, we have fully implemented our as defined in the package, and exported functions for creating and maintaining a Mongo session; time to write some tests and see if it works. Create another new file in the Mongo package UserService root mongo_test.go In this integration test we insert a single user into our database, verify that 1 user exists in the collection, and that they matches our test data. Head to the command line and run the test ➜ go test ./src/go_rest_api/pkg/mongo/ok go_rest_api/pkg/mongo 0.145s Success! …or is it? You’ll notice that, if we run the test a second time we get an error. 2017/03/14 22:58:04 Unable to create user: E11000 duplicate key error collection: test_db.user index: username_1 dup key: { : "integration_test_user" } We need a way to clean up the test database after ourselves, lets extend to provide a function to drop a database. Open up and add the following function: session.go session.go Then modify the defer statement in mongo_test.go Now we run the tests to our hearts content! Part 2 — Salted passwords In part 1 we implemented a service to store user documents in MongoDb; however we are storing the user password in plain text. Lets introduce a new go package to allow us to generate and compare salted hashes. Define an interface for our hash functions in the root package: And the implementation, in /pkg/crypto/hash.go Our implementation generates a hash with the format , this make storing the the hash-salt combination slightly more convenient as we only in a single database field instead of two. We now need to test our implementation to see if it works, we’ll test that we can successfully compare a generated hash to an input string, detect when an incorrect attempt is made, and that a generated hash produces a different result each time it is called. {hash}||{salt} Now that we know our package is working lets modify to store hashed passwords instead of plain text. Extend the function to accept an instance of , then use it to generated a hashed password in . cryto user_service.go NewUserService root.Hash Create So far so good, but the Mongo tests are now broken due to the added parameter. We’ll need to create a mock implementation in order to maintain the separation between packages. Create a very simple mock in Hash pkg/mock/hash.go Then, in simply instantiate the instance with the mock Hash implementation. mongo_test.go UserService That’s it, in this section we have created a crypto packing to handle password salting, added interaction between packages, and created the first mock implementation Part 3 — REST API for creating and retrieving users The next major component in the system is the http router, we’ll be using the package to configure our rest endpoints. This package will consist of three files: Gorrilla mux go_web_server/pkg/server/server.go -- configure router & starts listeningresponse.go -- helper functions for sending responsesuser_router.go -- routes for our user service The function initialises the server and user subrouter, this allows us to direct all requests beginning with into the file. simply starts our server on port , we are additionally passing all requests through the gorilla to provide automatic request logging to stdout. NewServer /user user_router.go Start 8080 LoggingHandler The user router exposes two rest endpoints for interacting with the Mongo service. for adding new users to the db. PUT /users/ for searching for users with a given username. GET /users/{username} You will notice a few unusal function calls to and , these are helper functions defined in Json Error response.go Part 4 — The folder, link it all up cmd Now that the and packages are working we can wire them up into a functioning web server. Create the folder and add server mongo crypto go_web_server/cmd/app main.go Now we can compile and run the server: ➜ go install ./src/go_web_server/cmd/app➜ ./bin/app.exe2017/04/25 13:50:35 Listening on port 8080 Using postman we can easily test the server endpoints. We now have a working server, at this point we could hook it up to a web front-end or mobile app and develop a user registration form. The next major piece of functionality we need is the ability to authenticate users and protect endpoints. (of course, you shouldn’t actually return the users password here). Part 5 — User authentication (sort of) I won’t create a full user authentication system here, just an endpoint to verify user credentials. In my next post I’ll show how you can offload this responsibility to the fantastic . kong api gateway Following the same patterns we used in the previous sections I’ve added a simple endpoint to accept a username / password combo, and find a user in the database with matching credentials. If a user is found, return them, otherwise return an error. After all that work we’ve created a fairly useless api, but its ours and we love it. I recommend this article if you want to be persuaded to learn go: https://medium.com/@kevalpatel2106/why-should-you-learn-go-f607681fad65 This article to read more about the folder structure I used in this project: https://medium.com/@benbjohnson/standard-package-layout-7cdbc8391fc1