Moving beyond “Hello, world” using Swift with Vapor3 After I finished a couple basic Vapor tutorials and built and wrote about I struggled to think of what to do next. I ended up coming across this post on Medium which seemed like an excellent project to try to build using . creating a Mock server using Vapor Vapor _Building a url shortner service which scales to billions is an software architecture challenge. This blog covers basic…_medium.com How to build a Tiny URL service that scales to billions? In this post I’ll look at actually building a shortening service based on some of the designs in the post above by combining , , Nginx, and and utilizing Docker to ease development and testing. URL Vapor Redis Postgres The Goal The goal for this project is to build a scalable, quick URL shortener that will support both automatic and custom URL shortening capabilities using Vapor. The goal of creating enumeration-resistant keys will be outside of the scope for this project — the assumption being that the onus is on the link creator to ensure that a shared url is protected against unauthorized users. The Design I’ll break the design down by component and then look at how each piece fits into the logical flow of creating a new url and accessing an existing url. Vapor Web Application From routing logic to the shortening algorithm itself, the majority of the actual business logic will take place inside the Vapor app. I’ve tried to plan this in such a way that the number of instances of the Vapor app could be scaled up to meet demand if necessary. Postgres Database Postgres will serve as the persistent storage for the application. Postgres seemed like a good choice just due to its ease of use, integration with vapor using Fluent, and it being cloud agnostic. Redis Redis will be serving dual duties. First, it will act as an atomic counter in order to allow for multiple instances of the application to run in parallel. Second it will act as a fast cache for responding to frequent link requests. Nginx Nginx will act as a boundary reverse-proxy and round-robin requests to any number of Vapor instances. Docker (docker-compose) I’m using docker-compose in this example to bundle everything together and ease local development and testing. Network diagram The Code Getting Started This tutorial assumes you have and installed already. docker-compose Vapor The first thing to do is create a new directory to store all of our project files in. Then initialize a new Vapor project. This command should set you up with a new Vapor template that includes code for an example ToDo application. Since I like to take advantage of XCode’s code completion I’ll also add an xcodeproj file and open our project in XCode. Scaffolding First we need to consider how docker is going to be used in this project. Since XCode really only cares about the package files, we’ll use the command line to create a and file. Since docker basics are a bit outside the scope of this tutorial I won’t go into too much detail, but everything we’ll use for development is pretty basic. I may look at configuring your application for a production deployment in a later post. Dockerfile docker-compose.yml Create a new and file by running the following commands. Dockerfile docker-compose.yml Paste the following into the . This code will build and launch our Vapor application within a docker container and expose the API on port 8080. Dockerfile Next add the following to the file. There’s a bit more going on here, but at a high level it’s defining each of our images for the Vapor App, for the Postgres database, for redis, and for nginx. It will also configure some basic authentication for Postgres and create separate networks for db operations and serving the API. docker-compose.yml api db redis nginx You’ll also need to define your configuration for nginx in order to get the forwarding to work as expected. My file looks like this. conf.d/api.conf Vapor Web Application The first bit of Vapor code to look at is our file. As I mentioned earlier we’ll want to make sure our project has all the necessary dependencies needed to interact with Redis and Postgres. Replace the existing file with the following. Package.swift Package.swift Configuration Next we can take a look at the file to see how we’re using some of the environment variables described in the docker files within the Vapor app. This file is fairly straight-forward and the comments should clear up any ambiguity, so I won’t go into much detail here. configure.swift As for the rest of the boilerplate code, in your project there should be an app.swift and a file, the defaults that are used here are fine for our purposes. boot.swift The Model Next we will remove the file included in the template and create a new Link.swift file to describe our model. The Link model does not contain any unexpected properties. It has an as well as a friendlyUrl, which represents a custom defined url, a shortUrl, which represents a shortened Url that will redirect to the originalUrl, a createdAt metadata attribute which , and finally an expiresAt attribute that could be used to clean up old links. Todo.swift Link which is a special field in Fluent id vapor can populate automatically I’ll go into how we plan to query using the short url and the id on this database in a second, but since we’ll also allow users to use a custom “friendly” url, we should add a unique index for that as well. In the directory of the project create a new directory named . Within this directory create a new file named and add the following code. You can see above in configure.swift starting on line 56 the code that will actually run this migration. App Migrations migrations.swift Routing Next we’ll look at which is where Vapor expects your routes to be defined. Since this service really only has two functions; 1. Creating a redirect (link) and 2. Redirecting a request, our file will be pretty straight-forward. Replace the autogenerated file with the following. routes.swift routes.swift routes.swift The Controller Finally we end up at the controller, the actual business logic that will serve our API. There are a few workflows here so I’ll walk through each one. Saving a new Link As you can see in above, creating a new Link is accomplished by making a POST request to . The first thing it will do when a request is made is ask redis for a new id from its atomic counter. We will then use that numeric id and the rest of the payload supplied in the POST request to create a new instance of Link and save it to Postgres. Assuming the save is successful we will add this link to our redis cache and return the object. routing.swift /links 2. Redirects The second half of this service is redirecting all GET requests to the appropriate url, if one exists. I’ll go over the shortening algorithm this service utilizes in a moment, but generally the service will first check redis to see if it has the redirect cached. Assuming it does not find a match in redis it will pass the request off to lookup() which will attempt to look up the url based on our base62 encoded id, or else based on the user-defined friendly url. Regardless of the lookup method used, if a result is found we will construct a 307 redirect response which most browsers should follow. If a result is not found in redis or Postgres, a 404 error is returned. Shortening the URLs Since we’ll primarily be relying on the integer ids for our Link lookup, we need some way of shortening integers into a more compact format. Integers work fine until you get into the 100,000s or millions, and at that point you really may not be saving too much space. This is where Base62 encoding comes into play. Base62, unsurprisingly, utilizes a 62 character mapping [a-zA-Z0–9]. What this means is that a long number such as 10,000,000 can be represented by a more compact “fxSK”. I’ve also distinguished auto-generated vs. custom urls by prepending the auto-generated Base62 strings with a “.”, when parsing the provided path during redirect handling, if the path starts with a “.” we know to query on the id index, and if it does not we know to query using the friendlyUrl. Running the Application Running the application in development mode (more on production below) should be straight-forward assuming you already have docker-compose set up and have followed the steps above. You can find all my code . here To start the application just run from your project’s root directory. It may take a few minutes to start up the first time because it needs to download all of Vapor’s dependencies and build the application. docker-compose up To scale up the number of swift nodes you have running you simply need to add where N is the number of nodes you desire. Docker should handling the routing between Nginx, Redis, and the Swift nodes. --scale api=N What’s Next? There are obviously a number of issues with this application that preclude it from being a production ready service. If I were to continue with this example there are a number of things I would add before deploying this application. Add unit and component test coverage This deserves a whole post of its own, since we would need to mock out our dependencies. Production configuration changed In our example all all of the postgres and redis configuration information is hardcoded into the application and the docker-compose file. For a production ready service we would want to harden this and likely not use some other orchestration for managing our data stores. But docker-compose is great for local development. Improving resiliency The app currently has OK error handling, but what do we do if our Postgres store become unreachable or our atomic counter is reset and conflicts with existing records — clearly there are improvements to be made. References _💧 A server-side Swift web framework. Contribute to vapor/vapor development by creating an account on GitHub._github.com vapor/vapor _Swift, it’s not just for mobile anymore…_hackernoon.com Building an Excellent Mock Server in Swift using Vapor _Building a url shortner service which scales to billions is an software architecture challenge. This blog covers basic…_medium.com How to build a Tiny URL service that scales to billions? You can view all my code for this project . here
