AWS Lambda — build yourself a URL shortener in 2 hours

An interesting requirement came up at work this week where we discussed potentially having to run our own URL Shortener because the mechanism (in iOS 9 and above) at Universal Links requires a JSON manifest https://domain.com/apple-app-site-association Since the OS doesn’t follow redirects this manifest has to be hosted on the URL shortener’s root domain. Owing to a limitation on it’s currently when you have configured for your app. Whilst we can switch to another vendor it means more work for our (already stretched) client devs and we really like ‘s support for attributions. AppsFlyer not able to shorten links Universal Links AppsFlyer Which brings us back to the question “should we build a URL shortener?” swiftly followed by “how hard can it be to build a scalable URL shortener in 2017?” Well, turns out it wasn’t hard at all as it took me less than 2 hours to have everything implemented, tested and deployed! Lambda FTW For this URL shortener we’ll need several things: a endpoint that will redirect you to the original URL GET /{shortUrl} a endpoint that will accept an original URL and return the shortened URL POST / an page where someone can easily create short URLs index.html a endpoint that serves a static JSON response GET /apple-app-site-association all of which can be accomplished with . API Gateway + Lambda Overall, this is the project structure I ended up with: using the framework’s template Serverless aws-nodejs each of the above endpoint have a corresponding handler function the file is in the folder index.html static the test cases are written in such a way that they can be used both as integration as well as acceptance tests there’s a script which facilitates running… build.sh integration tests, eg ./build.sh int-test {env} {region} {aws_profile} acceptance tests, eg ./build.sh acceptance-test {env} {region} {aws_profile} deployment, eg ./build.sh deploy {env} {region} {aws_profile} Get /apple-app-site-association endpoint Seeing as this is a static JSON blob, it makes sense to precompute the HTTP response and return it every time. POST / endpoint For an algorithm to shorten URLs, you can find a very simple and elegant on StackOverflow. All you need is an auto-incremented ID, like the ones you normally get with RDBMS. solution However, I find a more appropriate DB choice here because: DynamoDB it’s a managed service, so no infrastructure for me to worry about OPEX over CAPEX, man! I can scale reads & writes throughput elastically to match utilization level and handle any spikes in traffic but, has no such concept as an auto-incremented ID which the algorithm needs. Instead, you can use an to simulate an auto-incremented ID (at the expense of an extra write-unit per request). DynamoDB atomic counter GET /{shortUrl} endpoint Once we have the mapping in a table, the endpoint is a simple matter of fetching the original URL and returning it as part of the header. DynamoDB redirect Location Oh, and don’t forget to return the appropriate HTTP status code, in this case a . 308 Permanent Redirect GET / index page Finally, for the index page, we’ll need to return some HTML instead (and a different content-type to go with the HTML). I decided to put the HTML file in a folder, which is loaded and cached the first time the function is invoked. static Getting ready for production Fortunately I have had plenty of practice , and for this URL shortener we will need to: getting Lambda functions to production readiness configure auto-scaling parameters for the table (which we have an internal system for managing the auto-scaling side of things) DynamoDB turn on caching in for the production stage API Gateway Future Improvements If you put in the same URL multiple times you’ll get back different short-urls, one optimization (for storage and caching) would be to return the same short-url instead. To accomplish this, you can: add GSI to the table on the attribute to support efficient reverse lookup DynamoDB longUrl in the function, perform a GET with the GSI to find existing short url(s) shortenUrl I think it’s better to add a GSI than to create a new table here because it avoids having “transactions” that span across multiple tables. Useful Links Breaking down iOS 9 universal links slides for my talk on getting production ready with AWS Lambda my series on migrating Yubl to a serverless architecture SO : algorithm for shortening URLs Like what you’re reading but want more help? I’m happy to offer my services as an and help you with your serverless project — architecture reviews, code reviews, building proof-of-concepts, or offer advice on leading practices and tools. independent consultant I’m based in and currently the only UK-based . I have nearly of with running production workloads in AWS at scale. I operate predominantly in the UK but I’m open to travelling for engagements that are longer than a week. To see how we might be able to work together, tell me more about the problems you are trying to solve . London, UK AWS Serverless Hero 10 years experience here I can also run an to help you get with your serverless architecture. You can find out more about the two-day workshop , which takes you from the basics of AWS Lambda all the way through to common operational patterns for log aggregation, distribution tracing and security best practices. in-house workshops production-ready here If you prefer to study at your own pace, then you can also find all the same content of the workshop as a I have produced for Manning. We will cover topics including: video course authentication authorization with API Gateway Cognito & & testing running functions locally & CI/CD log aggregation monitoring best practices distributed tracing with X-Ray tracking correlation IDs performance cost optimization & error handling config management canary deployment VPC security leading practices for Lambda, Kinesis, and API Gateway You can also get the face price with the code . Hurry though, this discount is only available while we’re in Manning’s Early Access Program (MEAP). 40% off ytcui