Creating a form forwarding service for AWS Lambda

Since originally publishing this blog post I’ve [written an updated article](https://hackernoon.com/introducing-formplug-v1-a-form-forwarding-service-for-aws-lambda-2c125dfe608e). I’d recommend that you read that instead of continuing with this as some of the content below is now out of dat**e**. [**Introducing Formplug v1, a form forwarding service for AWS Lambda** _It’s estimated that approximately 269 billion emails are sent in a single day. Over 10 million were sent as you read…_hackernoon.com](https://hackernoon.com/introducing-formplug-v1-a-form-forwarding-service-for-aws-lambda-2c125dfe608e "https://hackernoon.com/introducing-formplug-v1-a-form-forwarding-service-for-aws-lambda-2c125dfe608e")[](https://hackernoon.com/introducing-formplug-v1-a-form-forwarding-service-for-aws-lambda-2c125dfe608e) In this post I’ll be introducing a service I’ve created that makes setting up email form forwarding easy. It’s built in [Javascript](https://hackernoon.com/tagged/javascript) using Node.js and the Serverless Framework. It deploys to [AWS](https://hackernoon.com/tagged/aws) Lambda and uses Amazon Simple Email Service to send the emails. To check out the code see the Github repository below. Keep reading for an explanation of the problem and a walk-through of the technical solution. [**danielireson/formplug-serverless** _formplug-serverless - Form forwarding service for AWS Lambda_github.com](https://github.com/danielireson/formplug-serverless "https://github.com/danielireson/formplug-serverless")[](https://github.com/danielireson/formplug-serverless) ### Problem Recently I’ve been finding myself taking a preference towards static site hosting for my side projects. Github Pages works well because like most people I use Github to host my git repositories anyway. To deploy a site it’s as easy as a `git push` to the _gh-pages_ branch and in a few short minutes the website will be live. In many cases static hosting is perfectly fine but on a couple of occasions now I’ve struggled to set up a simple email-based contact form — something which wouldn’t usually be considered too challenging of a task. There’s a couple of ways of handling this. #### Option 1: no contact form The obvious first option is to avoid contact forms altogether and instead just display an email address. This is the option that I’ve taken on many occasions because it’s the easiest but has a few downsides. It pushes extra work onto the users — they have to leave the site and open their email client. It’s also hard to specify what information is required when visitors first contact you, you might require qualifying information for a quote for example. You can have a message asking users for specific information but there’s no validation that can be done to confirm people are adhering to this request. #### _Option 2: mailto links_ One solution for sending forms by email is to set the form’s action attribute to a _mailto_ link set to an email address. When the user submits the form it opens their default email client with the form data as the body, as shown in the screenshot below. The result isn’t a great user experience. Not all users will have a default email client setup, and even when they do they’ll get presented with a draft email. It’s unclear whether the form has be submitted and what actions need to be taken by them. ![](https://hackernoon.com/hn-images/1*If7g5Iiu96xMdOx6mfJaaA.png) #### Option 3: use server-side code As a security feature you can’t send email client-side in the browser. If this were possible website visitors would be highly susceptible to malicious code sending email on their behalf. If you already host your apps on a server it’s painless to setup a contact form. Server-side scripting languages like PHP, Java or ASP have the capabilities that make it easy to send email. In PHP for example it can be achieved in two lines. $message = $\_POST\['message'\]; mail('johndoe@example.com', 'Form submission', $message); If you don’t already have a server however, having to maintain one just for a contact form seems a little wasteful. There’s time involved in maintaining it as well as financial costs, and it introduces other considerations around scalability and security. #### Option 4: use a third-party service Managed form backend services exist that range in functionality and scope. [FormKeep](https://formkeep.com/) for example makes it easy to integrate forms into services such as Trello, Slack or Google Sheets, but can be expensive just for a simple contact form with plans start at $59/month. There’s also always a risk involved when utilising a third-party service; it can go offline at any moment’s notice and you have no control of it. If there service was to shutdown there would be no notification, contact forms would just suddenly stop working. ### Solution A self-hosted form forwarding service is the optimal solution in my opinion. You get the benefits of control as well as no end-user experience degradation. Choosing to develop the backend for AWS Lambda further reduces many of the downsides of self-hosting. > AWS Lambda lets you run code without provisioning or managing servers. You pay only for the compute time you consume — there is no charge when your code is not running. With Lambda, you can run code for virtually any type of application or backend service — all with zero administration. Just upload your code and Lambda takes care of everything required to run and scale your code with high availability. There’s multiple approaches to building applications on AWS Lambda. For simple services you can use the AWS Management Console to create functions and set up triggers. For more complex services it’s better to develop locally and deploy to AWS afterwards, where you can enjoy the familiarities of your favourite code editor. Getting the project structure set up initially can be challenging as there’s lots of things to consider. Which AWS region is the service going to be deployed to? How are AWS services going to be provisioned? How are environment variables going to be managed? How is security going to be managed? Because of these considerations I like to use an opinionated framework called the Serverless Framework when working with AWS Lambda. It provides sensible defaults and allows us to to provision services at a higher and easier to understand abstraction. Using the CLI, deploying and updating a service is as simple as running `serverless deploy`. If you’re looking to learn a bit more about the Serverless Framework I’d recommend you watch the following video below by its creator Austin Collins. The video is a couple of years old now but does a great job of introducing some of the core concepts. I’d also recommend going through the [Serverless docs](https://serverless.com/framework/docs/). ![](https://i.ytimg.com/vi/BKx9vxxuZXY/hqdefault.jpg) ### Architecture overview The form forwarding service is composed of three lambda functions: encrypt, receive, send. The encrypt and receive functions are triggered through the Amazon API Gateway routes _/encrypt_ and _/to_ respectively. (An API Gateway route is essentially a public URL.) The send lambda function is only ever invoked by the receive lambda function. The receive lambda is responsible for handling the form submission and invoking the send lambda to send the email. The encrypt lambda can optionally be used to get an encrypted representation of an email address to use in the receive endpoint URL. This is explained in more detail later. ![](https://hackernoon.com/hn-images/1*K4ZuaZKkmcN3K4v2--6k8Q.png) ### Receive lambda The receive handler is triggered by the _/to_ route which accepts one parameter, the email address to forward the form to. The endpoint is generated on the initial `serverless deploy` and returned in the terminal window. It takes the following structure. https://apigatewayurl/to/johndoe@example.com HTML forms should POST to this endpoint. This will trigger the receive lambda (shown below). It’s responsible for validating the request, getting the form data and invoking the send lambda. module.exports.handle = (event, context, callback) => { let data = receiveRequest.getParams(event) if (receiveRequest.validate(data, callback)) { eventInvoker.send(data) .then(function () { httpRoute.render('receive-success', data, callback) }) .catch(function (error) { httpRoute.render('receive-error', data, callback) }) } } #### Getting the form data The form’s POST request will generate an API Gateway event which is automatically passed to the handler as the first parameter. An example of an event is shown below (with sensitive fields removed). { resource:'/to/{\_to}', path:'/to/johndoe@example.com', httpMethod:'POST', headers: { Accept:'\*/\*', 'Accept-Encoding':'gzip, deflate', 'cache-control':'no-cache', 'CloudFront-Forwarded-Proto':'https', 'CloudFront-Is-Desktop-Viewer':'true', 'CloudFront-Is-Mobile-Viewer':'false', 'CloudFront-Is-SmartTV-Viewer':'false', 'CloudFront-Is-Tablet-Viewer':'false', 'CloudFront-Viewer-Country':'GB', 'content-type':'application/x-www-form-urlencoded', Host:'', 'User-Agent':'', 'X-Amz-Cf-Id':'', 'X-Amzn-Trace-Id':'', 'X-Forwarded-For':'', 'X-Forwarded-Port':'443', 'X-Forwarded-Proto':'https' }, queryStringParameters:null, pathParameters: { \_to:'johndoe@example.com' }, stageVariables:null, requestContext: { path:'/dev/to/johndoe@example.com', accountId:'', resourceId:'', stage:'dev', requestId:'', identity:{ cognitoIdentityPoolId:null, accountId:null, cognitoIdentityId:null, caller:null, apiKey:'', sourceIp:'', accessKey:null, cognitoAuthenticationType:null, cognitoAuthenticationProvider:null, userArn:null, userAgent:'', user:null }, resourcePath:'/to/{\_to}', httpMethod:'POST', apiId:'' }, body:'first=test&second=test&third=test', isBase64Encoded:false } We’re only interested in some of the event data and so we extract this through the _getParams_ function. Specifically we’re interested in getting the form data from _event.body,_ the email address from _event.pathParameters_ and the query string from _event.queryStringParamters_. module.exports.getParams = function (event) { let data = Object.assign({}, querystring.parse(event.body), event.pathParameters, event.queryStringParameters) return data } The node querystring module is used to parse the POST form data from the request body. This takes the string _first=test&second=test&third=test_ and turns it into a JavaScript object. #### Validating the request The data then goes through a validation function which checks that the email is valid and that the honeypot field hasn’t been filled out. module.exports.validate = function (data, callback) { return hasValidEmail(data, callback) && hasNoHoneypot(data, callback) } function hasValidEmail (data, callback) { if (!('\_to' in data)) { httpRoute.render('receive-no-email', data, callback) return false } if ('\_to' in data && !httpValidation.isEmail(data\['\_to'\])) { httpRoute.render('receive-bad-email', data, callback) return false } return true } function hasNoHoneypot (data, callback) { if ('\_honeypot' in data && data\['\_honeypot'\] !== '') { httpRoute.render('receive-honeypot', data, callback) return false } return true } The honeypot field is an optional spam prevention feature. Users can add the field to their form and hide it from visitors. If the form is submitted and the honeypot field isn’t empty the request will be ignored (because a spam bot has tried to enter a value there). #### Invoking the send lambda After validation the send lambda is invoked asynchronously with a payload of the data using the AWS-SDK npm package. module.exports.send = function (data) { let event = { FunctionName: \`formplug-${config.STAGE}-send\`, InvocationType: 'Event', Payload: JSON.stringify(data) } return lambda.invoke(event).promise() } It invokes it asynchronously because we don’t want visitors to be waiting after they’ve submitted the form for Amazon to send the email. Invoking it in this way also means that AWS will [automatically retry it if it fail](http://docs.aws.amazon.com/lambda/latest/dg/retries-on-errors.html)s. #### Generating a HTTP response HTTP responses are made through the handler’s _callback_ function. This is provided by the Serverless Framework. For successful responses the callback function should be called with the first parameter as null and the second parameter as the response object. For example, to generate a successful plain text HTTP response we could do the following. callback(null, { statusCode: 200, headers: { 'Content-Type': 'text/plain' }, body: 'Form submission successfully made' }) But instead of a plain text response we want either a HTML, JSON or URL redirect response (depending on what was requested by the user). The application defaults to a HTML response. ![](https://hackernoon.com/hn-images/1*U0AqbOWXYod4FGF7UGchLQ.png) HTML responses are generated by loading a local template file and replacing a _{{ message }}_ variable with an appropriate message. function buildHtmlResponse (statusCode, message, data) { let response = { statusCode: statusCode, headers: { 'Content-Type': 'text/html' }, body: generateView(message) } if (statusCode === 302) { response.headers.Location = data\['\_redirect'\] } return response } function generateView (message) { try { let template = fs.readFileSync(path.resolve(\_\_dirname, 'template.html')).toString() message = template.replace('{{ message }}', message) } catch (error) { utilityLog.error(error.message) } return message } Users can also request a URL redirect after the form submission. This URL should be supplied as a hidden input in the form itself. If a _\_redirect_ field is supplied then the application will set a 302 HTTP status code and the [_Location_ HTTP header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Location). The visitor’s web browser will pick these up and handle the redirection. Non-redirect response messages are defined in a routes file. Users can define custom messages in their config or use the service defaults. These are called through a route _render_ function. module.exports.render = function (type, data, callback) { let routeDetails = module.exports.getRouteDetails(type, data) callback(null, httpResponse.build(routeDetails.statusCode, routeDetails.message, data)) } module.exports.getRouteDetails = function (type, data) { let statusCode, message switch (type) { case 'encrypt-no-email': statusCode = 422 message = config.MSG\_ENCRYPT\_NO\_EMAIL || 'You need to provide an email address to encrypt.' break case 'encrypt-bad-email': statusCode = 422 message = config.MSG\_ENCRYPT\_BAD\_EMAIL || 'The supplied email address is not valid.' break case 'encrypt-success': statusCode = 200 message = data\['\_encrypted'\] break case 'receive-honeypot': statusCode = 422 message = config.MSG\_RECEIVE\_HONEYPOT || 'You shall not pass.' break case 'receive-no-email': statusCode = 422 message = config.MSG\_RECEIVE\_NO\_EMAIL || 'Form not sent, the admin has not set up a forwarding email address.' break case 'receive-bad-email': statusCode = 422 message = config.MSG\_RECEIVE\_BAD\_EMAIL || 'Form not sent, the admin email address is not valid.' break case 'receive-error': statusCode = 500 message = config.MSG\_RECEIVE\_ERROR || 'Form not sent, an error occurred while sending.' break case 'receive-success': statusCode = httpValidation.hasRedirect(data) ? 302 : 200 message = config.MSG\_RECEIVE\_SUCCESS || 'Form submission successfully made.' break default: statusCode = 500 message = 'An error has occurred.' } return {statusCode, message} } Oftentimes the form forwarding service will be called with JavaScript, in which case a JSON response should be requested. This is done by appending _\_format=json_ to the receive endpoint URL querystring (the underscore is used to adhere to the same naming convention as other private fields). https://apigatewayurl/to/johndoe@example.com?\_format=json This returns the response with a _Access-Control-Allow-Origin_ header set to _\*,_ to denote all domains. This [CORS header](https://developer.mozilla.org/en/docs/Web/HTTP/Access_control_CORS) is required as a security feature for cross-domain JavaScript calls. function buildJsonResponse (statusCode, message, data) { let response = { statusCode: statusCode, headers: { 'Access-Control-Allow-Origin': '\*', 'Content-Type': 'application/json' }, body: JSON.stringify({ statusCode: statusCode, message: message }) } return response } ### Send lambda AWS will trigger the send lambda handler after it receives an event invoked from the receive lambda. module.exports.handle = (event, context, callback) => { let email = mailBuilder.build(event) mailService.send(email) .then(function () { utilityLog.success('Successfully sent email') }) .catch(function (error) { utilityLog.error(\['Error sending email', event, error\], callback) }) } The email body is built by looping over the event’s data which contains all of the user form fields. function buildMessage (data) { let message = '' for (let field in data) { // Don't send private variables prefixed with an underscore if (field.slice(0, 1) !== '\_') { message += field.toUpperCase() + ': ' + data\[field\] + '\\r\\n' } } message += '---' + '\\r\\n' message += 'Sent with Formplug' return message } The above _buildMessage_ function is called in the exported _build_ function which puts together the full email object for the send lambda handler. module.exports.build = function (data) { return { Source: buildSource(), Destination: { ToAddresses: \[ data\['\_to'\] \] }, Message: { Subject: { Data: 'You have a form submission' }, Body: { Text: { Data: buildMessage(data) } } } } } The email object is sent via Amazon Simple Email Service, which has been encapsulated in a separate function (which allows it to be [easily stubbed during testing](https://github.com/danielireson/formplug-serverless/blob/master/test/index.js)). This function is called from the send lambda handler. const aws = require('aws-sdk') const sesClient = new aws.SES() module.exports.send = function (email) { return sesClient.sendEmail(email).promise() } ### Encrypt lambda Usage of encryption is optional; the service can be used without it. It gives users the opportunity to encrypt their email address in the receive endpoint. https://apigatewayurl/to/1974d0cc894607de62f0581ec1334997 The encrypt handler grabs the email address from the URL, validates it and passes it to an encryption function. module.exports.handle = (event, context, callback) => { let data = encryptRequest.getParams(event) if (encryptRequest.validate(data, callback)) { data\['\_encrypted'\] = httpEncryption.encrypt(data\['\_email'\]) httpRoute.render('encrypt-success', data, callback) } } Encryption is done through node’s built-in _crypto_ module, using a cipher that is created using a custom encryption key environment variable. const crypto = require('crypto') const config = require('../../config.json') module.exports.encrypt = function (str) { let cipher = crypto.createCipher('aes-256-ctr', config.ENCRYPTION\_KEY) let crypted = cipher.update(str, 'utf8', 'hex') crypted += cipher.final('hex') return crypted } module.exports.decrypt = function (str) { let decipher = crypto.createDecipher('aes-256-ctr', config.ENCRYPTION\_KEY) let text = decipher.update(str, 'hex', 'utf8') text += decipher.final('utf8') return text } The encrypted email address is returned to the user via the _encrypt-success_ route which generates the HTTP response. ### Wrap-up In this post we went through the creation of a form forwarding service that can be used to handle contact form submissions for both static and dynamic websites. I walked through possible alternative solutions before going on to describe the technical architecture of my solution. The service uses the Serverless Framework to deploy to AWS Lambda, which allows it realise many of the benefits of self-hosting but with reduced complexity. For the complete project code and for installation instructions, view the repository on Github. [**danielireson/formplug-serverless** _formplug-serverless — Formplug is a serverless form backend for AWS Lambda_github.com](https://github.com/danielireson/formplug-serverless "https://github.com/danielireson/formplug-serverless")[](https://github.com/danielireson/formplug-serverless) If you found this post interesting you might enjoy [an article](https://hackernoon.com/i-created-a-word-count-webhook-for-github-45c69ba0e22) I wrote a couple of weeks ago where I walked through another serverless project. I used the Serverless Framework to create a word count webhook for Github. [**Creating a word count webhook for GitHub** _A webhook is not a personal weapon, although it may sound like one. So if you’re looking to learn how to inflict damage…_hackernoon.com](https://hackernoon.com/i-created-a-word-count-webhook-for-github-45c69ba0e22 "https://hackernoon.com/i-created-a-word-count-webhook-for-github-45c69ba0e22")[](https://hackernoon.com/i-created-a-word-count-webhook-for-github-45c69ba0e22)

Creating a form forwarding service for AWS Lambda

Too Long; Didn't Read

Companies Mentioned

Coin Mentioned

@danielireson

RELATED STORIES