Running a scalable & reliable GraphQL endpoint with Serverless

Part 2: AppSync Backend: AWS Managed GraphQL Service

AWS AppSync architecture

Part 1: Introduction: GraphQL endpoints with API Gateway + AWS Lambda Part 2: AppSync Backend: AWS Managed GraphQL Service (this post)Part 3: AppSync Frontend: AWS Managed GraphQL ServicePart 4: Serverless AppSync Plugin: New Features (Latest!!!)

“Powering your GraphQL endpoint with a serverless backend solves scaling and availability concerns outright, and it gives you a big leg up on security. It’s not even that much code or configuration”. - Part 1

Introduction

In this part of the series, we will learn how to build GraphQL endpoints with new AWS service called — AppSync launched at the re: Invent 2017. I will be creating a backend for a mini Twitter App using AppSync with DynamoDB, ElasticSearch, and AWS Lambda integrations and show you how to configure and deploy AppSync using new Serverless-AppSync-Plugin.

Let’s get started! 🏃

Note: If you are new to Serverless or GraphQL, I would recommend grabbing a cup of coffee and going through Part 1 of this series, then come back.

What is AppSync?

AWS AppSync is a fully managed serverless GraphQL service for real-time data queries, synchronization, communications and offline programming features. This blog covers creating a GraphQL API itself, and next part of this series will focus on the AppSync Client and its features (stay tuned! 🔈_)._

“It turns out that Apollo Client 2.0’s modular architecture was a huge win for developers looking to customize their GraphQL client because that’s exactly how the AWS team was able to build AppSync Client!” — by Peggy Rayzis

Note: This post by Nader Dabit is an excellent read for an AppSync primer.

AppSync vs. GraphQL + Lambda

If Serverless + Lambda + GraphQL works so well, then why even bother exploring other solutions?

Because AppSync has a cool piece of functionality that GraphQL + Lambda doesn’t. Namely: real-time subscriptions.

Both have their advantages. But which one is right for your application?

Use Lambda when you want to have more control over what the endpoint does. For example, maybe you want to use your favorite Lambda Server to handle client requests; or perhaps you need a more closed system, and you don’t want to use AWS authentication systems.

Use AppSync when you need support for real-time updates which scales to millions of people. It is much easier to do as compared to implementing real time updates with Lambda and IoT. And of course, it comes with fully AWS managed GraphQL service and enterprise-level security features.

How can I get started with AppSync in Production?

I am glad you asked!

After working with AppSync, you may quickly realize that it is quite time-consuming and error-prone to wire up the entire backend components including GraphQL Schema, Resolver Mapping Templates and Data Sources (as I’ll explain further in a minute).

In a production environment, you would want a piece of code or configuration to make your deployment fully automated along with being fast and scalable.

For the same reason, I am happy to announce brand new Serverless-AppSync-Plugin which allows you to deploy AppSync Components via Serverless CLI and is now open sourced 🎉

My motivation behind writing this blog is to share my learnings about AppSync and provide a smooth and more flexible way to stand up a GraphQL endpoint in production within seconds.

Note: Examples covered in this blog are available on GitHub.

Let’s build a GraphQL Endpoint with AppSync Plugin

Before jumping into this, let me explain AppSync’s four building blocks:

1. 
   

   GraphQL SchemaComplete schema of the mini Twitter app is available here. It includes various types and fields to retrieve user and tweet info.

2. 
   

   GraphQL Resolvers (Mapping Templates)Each field in the schema is resolved using request and response mapping templates (written in Velocity Templating Language). These templates parse the incoming request and interpret responses from your database.

3. Data Sources You get built-in support of DynamoDB, ElasticSearch, and Lambda. In fact, Lambda provides flexibility to add RDS, REST API, MongoDB, Druid, etc.

4. 
   

   **Authentication and IAM Permissions**You can authenticate your API using API KEY, Cognito User Pools along with providing fine-grained access controls using IAM permissions.

Anatomy of the AppSync SDK:

To create the endpoint you basically have two options 1) manually wire all the components together and generate the API in AppSync UI which is time-consuming and error-prone or 2) write code using AWS-SDK which adds more complexity to the entire process.

Here are the series of steps required to have an endpoint up and running:

Anatomy of the AppSync SDK

Serverless AppSync Plugin to the Rescue

And now, look at this 😏

plugins:

• serverless-appsync-plugin

provider:name: awsruntime: nodejs6.10

custom:appSync:name: # Your GraphQL API NameauthenticationType: AMAZON_COGNITO_USER_POOLS | API KEYmappingTemplates:- dataSource: myDynamoDB | myElasticSearch | myLambdatype: # GraphQL Typefield: # Schema Fieldrequest: # Request Mapping Templateresponse: # Response Mapping Templateschema: schema.graphql # Input GraphQL SchemadataSources:- type: AMAZON_DYNAMODB | AMAZON_ELASTICSEARCH | AWS_LAMBDAname: myDynamoDB | myElasticSearch | myLambdaconfig:tableName | endpoint | lambdaFunctionArnserviceRoleArn: IAM ROLE

The Serverless AppSync Plugin allows you to configure all the six steps as a configuration in your serverless.yml. You can basically deploy, update or delete AppSync components using these three commands:

serverless deploy-appsyncserverless update-appsyncserverless delete-appsync

Note: All the configurations in the plugin are self-explanatory, but if you have any questions or want to contribute, please feel free to reach out to me.

Finally, this is how it looks:

AppSync Deployment with Serverless Plugin

Mini Twitter App Architecture

Now, you might be thinking “what is this fuss about mini Twitter App? How does it work? I still don’t get it!”. OK, let me explain:

This app consists of a React Frontend along with an AppSync Client integration. You can further simplify the user authentication flow using AWS Amplify and Cognito User Pool (I’ll be covering frontend architecture in detail in my next blog post).

For the app backend, the GraphQL API is created using the Serverless AppSync Plugin. This API connects to DynamoDB (to get user Info), ElasticSearch (to retrieve user tweets) and Lambda (to fetch any additional user info from the Twitter REST API).

Note: Please follow these instructions to run the app in your local environment.

Mini Twitter App Architecture

Let’s look at some resolver mapping templates:

Example 1: getUserInfo(handle: String!) vs. meInfo

Request Mapping Template for getUserInfo:

{"version" : "2017-02-28","operation" : "Query","query" : {"expression": "handle = :handle","expressionValues" : {":handle" : {"S" : "${context.arguments.handle}"}}}}

Request Mapping Template for meInfo:

{"version" : "2017-02-28","operation" : "Query","query" : {"expression": "handle = :handle","expressionValues" : {":handle" : {"S" : "${context.identity.username}"}}}}

In this example, the request templates are using DynamoDB Query operation to fetch data from the Users table. In meInfo user’s handle is derived from context variable which contains his identity information (parsed through JWT token on the client side). This article explains more about the power of context variable and util functions in AppSync.

Response Mapping Template for both fields:

$util.toJson($context.result.items[0])

Example 2: favourites_count

In this case, we want to fetch the value of a user’s favourite count from Twitter itself. AWS Lambda provides the flexibility to do this by querying the REST API (reference)

exports.graphqlHandler = (event, context, callback) => {switch (event.field) {case 'favourites_count': {const handle = event.arguments.handle ? event.arguments.handle : event.handle;

  _getFavouritesCount_(handle).then(result => {  
    callback(**null**, result);  
  });  

  **break**;  
}  

}};

Example 3: tweets(limit: Int!, nextToken: String)

ElasticSearch provides an immense power of a search engine. In this case, we have indexd all the tweets in ES and the request mapping template below paginates through the user’s tweets:

{"version":"2017-02-28","operation":"GET","path":"/user/twitter/_search","params":{"body":{"from": $context.arguments.nextToken,"size": $context.arguments.limit,"query" : {"bool" : {"must" : [{"match" :{ "handle" : $context.source.handle }}]}}}}}

Response Mapping Template:

{#set($hitcount = $context.result.hits.total)#set($tweetList = [])#set($counter = 0)#if($hitcount > 0)#foreach($entry in $context.result.hits.hits)#set( $element = ${tweetList.add({"tweet" : "$entry.get('_source')['tweet']","tweet_id": "$entry.get('_id')","created_at": $entry.get('_source')['created_at']})})#set ($counter = $counter + 1)#end"items" : $util.toJson($tweetList),"nextToken" : "$counter"#end}

Sample GraphQL Query:

query {meInfo{ # DynamoDBnamedescriptionfavourites_count # Lambdatweets(limit:4, nextToken:"3"){ # ElasticSearchitems{tweettweet_idcreated_at}nextToken}}}

Last but not the least…

The best part? All you need to get subscriptions working in the backend is to extend your GraphQL schema with 4 lines of code:

type Subscription {addTweet: Tweet@aws_subscribe(mutations: [“createTweet”]}

This code sets up a subscription that listens to every new createTweet mutation, and once your client decides to subscribe to this subscription it will get real-time updates (more on this in the next post 😏)

AppSync Limitations

1. 
   

   **Steep learning curve**Working with AppSync requires a good amount of understanding of VTL. For beginners, I would totally recommend this guide. But, the good news is that AWS provides a bunch of helpers and utilities to make our life easier.

2. 
   

   **Missing GraphQL Info Object**AppSync doesn’t provide info object in the context variable for now which limits its functionality regarding integrating with other open source GraphQL frameworks like Prisma.

3. 
   

   **Missing support for DynamoDB Batch operations**As of now, AppSync doesn’t support all DynamoDB API’s for instance — BatchGetItem or BatchPutItem

AppSync is actively adding capabilities to simplify the serverless GraphQL experience, and I’m looking forward to it.

What Next?

In the next part of this series, I will explain mini Twitter App’s FrontEnd components in detail including AppSync Client, AWS Amplify, React Components, Mutations with Optimistic Response and Offline Support, Subscriptions, Conflict Resolution etc (stay tuned! 🔈_)._

The App will look like this:

Special Thanks

First of all, BIG THANKS to Nik Graf for collaborating on this project, being an excellent mentor along with helping with code reviews and implementation.

Thanks, Philipp, Jon, and LolCoolKat for your hard work on AppSync Plugin.

AWS Mobile team (Richard, Rohan, Nader, Manuel, and Michael) for helping out with questions and working closely to resolve issues and bugs.

Last but not the least, thanks everyone who is reading this post or encouraged me to write more. Your support drives me to contribute more 😃 😍

Serverless and GraphQL Meetup at Glassdoor, San Francisco attended by Graphcool, Serverless, Danielle, Nik, Rohan, Michael

I would like to end this blog with one of my favourite quotes —

“The important thing is not to stop questioning. Curiosity has its own reason for existing.” — Albert Einstein