Ampsy

ampsy

In this blog post series, I would like to discuss best practices for building multi-tenant services in . Existing literature on how to build multi-tenant services is usually aimed at SaaS applications with hundreds of customers ( e.g. ). AWS Building a Multi-Tenant SaaS Solution Using AWS Serverless Services The main rationale for this series is to focus on building multi-tenant services for use cases with fewer clients that are all deployed to AWS accounts. Usually, this would apply to scenarios when you build a multi-tenant service for internal use. I will split the series of blog posts into three parts for each type of service-to-service integration: and synchronous, asynchronous, batch integration. Part 1 will discuss multi-tenant architecture for two AWS services: API Gateway and AppSync. Throughout the article, I refer to the code from the sample application app built for this article in Typescript and AWS CDK: . https://github.com/filletofish/aws-cdk-multi-tenant-api-example/tree/main Content Overview Multi-tenancy for internal services 1.1. Tenant isolation 1.2. Multi-tenant monitoring 1.3. Scaling Multi-tenancy for internal services 2.1. Tenant-isolation - acess-control 2.2 Tenant-isolation - noisy neighbor problem 2.3 Multi-tenant monitoring 2.4 Metrics, Alarms, Dashboards 2.5 Onboarding and offboarding API clients Multi-tenancy with AWS AppSync Conclusion 1. Multi-tenancy for internal services Multi-tenancy is the ability of software to serve multiple customers or tenants with a single instance of the software. Once you allow more than one team to call your service , your service becomes multi-tenant. Multi-tenant architecture introduces additional complexity to your services, such as tenant isolation, tenant-level monitoring, and scaling. API 1.1. Tenant isolation Generally, tenant isolation addresses security concerns by ensuring that tenants are prevented from accessing another tenant’s resources. Also, tenant isolation is implemented to make sure that any failures caused by one tenant don’t impact other tenants of your service. It’s also often referred to as a noisy neighbor problem. See more in the AWS Whitepaper on Tenant Isolation Strategies . https://d1.awsstatic.com/whitepapers/saas-tenant-isolation-strategies.pdf 1.2. Multi-tenant monitoring Once multiple tenants start sharing infrastructure resources, you would need to monitor how each of your tenants uses your system. It usually means that the tenant name or identifier should be present in your logs, metrics, and dashboards. Multi-tenant monitoring could be useful for several reasons: Troubleshooting issues: simplifies problem identification and resolution, distinguishing tenant-specific issues from broader ones. Resource allocation and capacity planning. Multi-tenant monitoring can help you track per-tenant resource consumption for resource allocation and capacity planning. Even if your service is serverless, you still need to understand your client resource consumption to understand if you are going to hit any of AWS limits soon (a typical example is a Lambda Function Concurrent Execution limit). SLA Management: Allows tracking of tenant-specific performance against SLAs. Billing. It’s unlikely that you start billing other teams for using your internal service. However, at some scale of the company growth billing other teams could be a good idea to ensure frugal usage of the service. 1.3. Scaling Multi-tenant services are likely more exposed to scaling challenges than single-tenant services. However, scalability is a huge topic and I won’t cover it in this blog post. 2. Multi-tenancy with API Gateway If you are building your AWS web service with , HTTP, or WebSocket API in AWS you are most likely using API Gateway. REST 2.1. Tenant-isolation — access-control AWS recommends deploying each service in its own AWS account(s) to isolate the service’s resources and data, easier cost-management, and separation between test and production environments (see details in ). AWS Whitepaper Organizing Your AWS Environment Using Multiple Accounts If your company services are deployed in AWS then the most obvious solution for managing access to your API Gateway is AWS IAM. AWS Cognito is another option for managing access to multi-tenant API (see , ). Throttling a tiered, multi-tenant REST API at scale using API Gateway The case for and against Amazon Cognito Comparison between AWS IAM and AWS Cognito deserves a separate deep-dive. But for this article, I would stick with AWS IAM as it’s the simplest way to manage access when your company services are in AWS. Once you enable AWS IAM authorization for the API Gateway Method (see ), all API requests for this method should be signed with credentials of IAM identity allowed to call your API Gateway. CFN By default, no access is allowed between AWS accounts. For example, invoking your API Gateway with credentials of another AWS account will fail. To integrate your customers with your API you need to set up cross-account access. For granting cross-account access to your API Gateway you can use two methods: resource-based authorization (not available for API Gateway HTTP API) and identity-based authorization (see more at ): https://repost.aws/knowledge-center/access-api-gateway-account . For resource-based access, you need to update the API Gateway Resource Policy and add the AWS Account of your client. The main disadvantage of this method is that once you update the resource policy, the API Gateway stage needs to be redeployed for changes to take effect (see AWS docs and ). However, if you use CDK you can automate the deployment of new stages (see ). Another disadvantage is the limit for the maximum length of resource policy. Onboarding a client with resource-based authorization [1] [2] AWS CDK Docs for Api Gateway . For identity-based access control, you need to create an IAM role for the client and allow the client to assume it by updating the role’s resource policy (trusted relationships). You could use IAM users, but IAM roles are better from the security point of view. Roles allow authentication with temporary credentials and do not require storing IAM user credentials. There is a limit of 1,000 roles per account, but this limit is adjustable. Plus, another disadvantage of the role-based method for getting cross-account access to your API is that you need to create an IAM role for every new API client. However, role management can be automated with CDK (see ). Onboarding a client with identity-based authorization code sample from provided CDK app IAM authorization only allows you to control access to the API Gateway (using IAM policy you can specify what AWS account can call what API Gateway endpoints). It’s your responsibility to implement control access to the data and other underlying resources of your service. Within your service, you can use the AWS IAM ARN of the caller that is passed with API Gateway Request for further access control: AWS export const handler = async (event: APIGatewayEvent, context: Context): Promise<APIGatewayProxyResult> => {
  // IAM Principal ARN of the api caller
  const callerArn = event.requestContext.identity.userArn!;

  // .. business logic based on caller
  return {
    statusCode: 200,
    body: JSON.stringify({
      message: `Received API Call from ${callerArn}`,
    })
  };
}; 2.2. Tenant-isolation — noisy neighbour problem The default API Gateway limit is 10,000 TPS ( ). However, due to your downstream dependencies, your service might require a lower TPS limit. To avoid an overload of API requests from a single tenant that will impact the availability of the whole system you should implement per-tenant API rate limiting (also referred to as “throttling” or “admission control”). API Gateway Quotas and Limits You can use API Gateway API Usage Plans and Keys to configure limits for each client separately (for details see AWS documentation [1], [2], and [3]) 2.3. Multi-tenant Monitoring API Gateway has two types of logs: API Gateway Execution Logs: contains data such as request or response parameter values, what API keys are required, whether usage plans are enabled, and so on. Not enabled by default, but can be configured. API Gateway Access Logs feature: allows you to log who has accessed your API, how it was accessed, what endpoint was accessed, and the result of the API call. You can provide your log format and choose what to log with context variables (see docs, in CDK). To monitor the requests of your API clients, I would recommend enabling access logging. You can log at very least AWS IAM ARN of the caller ( ), the request path ( ) , your service response status code and API call latency ( ). $context.identity.userArn $context.path $context.status $context.responseLatency Personally, for a service with AWS IAM Auth and Lambda function as compute I found this API Gateway Access Logging configuration useful: const formatObject = {
  requestId: '$context.requestId',
  extendedRequestId: '$context.extendedRequestId',
  apiId: '$context.apiId',
  resourceId: '$context.resourceId',
  domainName: '$context.domainName',
  stage: '$context.stage',
  path: '$context.path',
  resourcePath: '$context.resourcePath',
  httpMethod: '$context.httpMethod',
  protocol: '$context.protocol',
  accountId: '$context.identity.accountId',
  sourceIp: '$context.identity.sourceIp',
  user: '$context.identity.user',
  userAgent: '$context.identity.userAgent',
  userArn: '$context.identity.userArn',
  caller: '$context.identity.caller',
  cognitoIdentityId: '$context.identity.cognitoIdentityId',
  status: '$context.status',
  integration: {
    // The status code returned from an integration. For Lambda proxy integrations, this is the status code that your Lambda function code returns.
    status: '$context.integration.status',
    // For Lambda proxy integration, the status code returned from AWS Lambda, not from the backend Lambda function code.
    integrationStatus: '$context.integration.integrationStatus',
    // The error message returned from an integration
    // A string that contains an integration error message.
    error: '$context.integration.error',
    latency: '$context.integration.latency',
  },
  error: {
    responseType: '$context.error.responseType',
    message: '$context.error.message',
  },
  requestTime: '$context.requestTime',
  responseLength: '$context.responseLength',
  responseLatency: '$context.responseLatency',
};

const accessLogFormatString = JSON.stringify(formatObject);
const accessLogFormat = apigw.AccessLogFormat.custom(accessLogFormatString); Once logging is enabled, you can use CloudWatch Insights to easily get the latest calls from a chosen API client with: fields @timestamp, path, status, responseLatency, userArn
| sort @timestamp desc
| filter userArn like 'payment-service'
| limit 20 2.4. Metrics, Alarms, Dashboards CloudWatch Metrics supported by API Gateway by default are aggregated for all requests. But you can parse API Gateway access logs to publish custom CloudWatch metrics with an additional dimension of your client name to be able to monitor client (tenant) usage of your API. At the very minimum, I would recommend publishing per-client CloudWatch metrics Count, 4xx, 5xx, Latency split by . You could also add dimensions like status code and API path. Dimension=${Client} 2.4.1. Using metric log filters for publishing per-client metrics CloudWatch Metric Log Filters (see docs) allow you to provide a custom filter and extract metric values from API Gateway Access Logs (see example below). Metric Log Filters also allow extracting value for custom metrics dimensions from logs. For multi-tenancy monitoring, the dimension Client could be the IAM ARN of the caller. The main advantages of Metric Log Filters are (1) no compute to manage (2) it’s simple & cheap. But you cannot do any data modifications (e.g. set more readable client names instead of IAM ARNs) and there is a limit of 100 metric filters per single log group (docs). Example of CloudWatch Metric Log Filter to Publish with dimension and Count Client Path new logs.MetricFilter(this, 'MultiTenantApiCountMetricFilter', {
  logGroup: accessLogsGroup,
  filterPattern: logs.FilterPattern.exists('$.userArn'),
  metricNamespace: metricNamespace,
  metricName: 'Count',
  metricValue: '1',
  unit: cloudwatch.Unit.COUNT,
  dimensions: {
    client: '$.userArn',
    method: '$.httpMethod',
    path: '$.path',},});
}); See . all metric filters for 4xx, 5xx error, and latency metrics at the provided sample CDK application 2.4.2. Using Lambda function for publishing per-client metrics The alternative option is to create a Lambda function to parse the logs, extract metrics and publish them. This allows you do more custom stuff like filtering out unknown clients or extract client name from the userArn. With just a couple of lines of CDK code to subscribe Lambda function to API Gateway Access Logs: const logProcessingFunction = new lambda.NodejsFunction(
  this, 
  'log-processor-function',
  {
    functionName: 'multi-tenant-api-log-processor-function',
  }
);

new logs.SubscriptionFilter(this, 'MultiTenantApiLogSubscriptionFilter', {
  logGroup: accessLogsGroup,
  destination: new logsd.LambdaDestination(logProcessingFunction),
  filterPattern: logs.FilterPattern.allEvents(),
}); See full example in as well as implementation of . code Log Processor Lambda Function Once you have started publishing API Gateway metrics that are split by Client, you can now create CloudWatch Dashboards and CloudWatch Alarms for each client separately. 2.5. Onboarding and offboarding API clients Your CDK app could be an easy solution to store a config with client names, their AWS accounts, requested TPS limits, and other metadata. To onboard a new API client you would need to add it to the config managed in code: interface ApiClientConfig {
  name: string;
  awsAccounts: string[];
  rateLimit: number;
  burstLimit: number;
}

const apiClients: ApiClientConfig[] = [
  {
    name: 'payment-service',
    awsAccounts: ['111122223333','444455556666'],
    rateLimit: 10,
    burstLimit: 2,
  },
  {
    name: 'order-service',
    awsAccounts: ['777788889999'],
    rateLimit: 1,
    burstLimit: 1,
   },
]; Using this config the CDK app can then create an IAM role, API Gateway Usage Key, and pass the name of the client to Lambda Function that parses access logs (see it in the sample application code). 3. Multi-tenancy with AWS AppSync If your service has a API you probably use AppSync. Similarly to API Gateway, you can use IAM Auth to authorize AppSync requests. AppSync does not have a resource policy (see ), so you can only use a role-based authorization for setting up access control to AppSync API. Similarly to API Gateway, you would create a separate IAM role for every new tenant of your service. GraphQL GH issue Unfortunately, AppSync has limited support for per-client throttling that we need for tenant isolation and monitoring. While you can set up TPS limits for AppSync with WAF, you cannot create separate per-client limits to isolate your service tenants. Similarly, AppSync does not provide access logs as API Gateway does. Solution? You can add API Gateway as a proxy to your AppSync and use all the above described API Gateway features to implement multi-tenancy requirements like tenant isolation and monitoring. On top of it, you can use other API Gateway features like Lambda Authorizers, Custom Domain, and API lifecycle management that do not yet exist in AppSync. The disadvantage is a slight additional latency for your requests. 4. Conclusion That’s it. If you have any questions or ideas, let me know in the comments or contact me directly. In the next part of this series, I will review best practices for asynchronous internal integration with AWS Event Bridge and AWS SQS / SNS. If you want to dive deep into the topic of building multi-tenant services on top of AWS I found these resources useful: AWS Whitepaper on SaaS Tenant Isolation Strategies Fairness in multi-tenant systems AWS re:Invent 2021 — SaaS architecture patterns: From concept to implementation Also published here.

How to Build Multi-Tenant Internal Services in AWS and CDK (Part 1): API Gateway and AppSync

AWS এবং CDK-এ মাল্টি-টেন্যান্ট অভ্যন্তরীণ পরিষেবাগুলি কীভাবে তৈরি করবেন (পর্ব 1): API গেটওয়ে এবং অ্যাপসিঙ্ক

So erstellen Sie mehrinstanzenfähige interne Dienste in AWS und CDK (Teil 1): API Gateway und AppSync

Cómo crear servicios internos multiinquilino en AWS y CDK (Parte 1): API Gateway y AppSync

Comment créer des services internes multi-locataires dans AWS et CDK (partie 1) : API Gateway et AppSync

AWS और CDK में मल्टी-टेनेंट आंतरिक सेवाएँ कैसे बनाएँ (भाग 1): API गेटवे और AppSync

AWS と CDK でマルチテナントの内部サービスを構築する方法 (パート 1): API ゲートウェイと AppSync

AWS 및 CDK에서 다중 테넌트 내부 서비스를 구축하는 방법(1부): API Gateway 및 AppSync

Como construir serviços internos multilocatários em AWS e CDK (Parte 1): API Gateway e AppSync

Как создать мультитенантные внутренние сервисы в AWS и CDK (часть 1): API Gateway и AppSync

AWS ve CDK'da Çok Kiracılı Dahili Hizmetler Nasıl Oluşturulur (Bölüm 1): API Ağ Geçidi ve AppSync

Cách xây dựng dịch vụ nội bộ nhiều bên thuê trong AWS và CDK (Phần 1): API Gateway và AppSync

如何在 AWS 和 CDK 中构建多租户内部服务（第 1 部分）：API Gateway 和 AppSync

We have another exciting update for the HackerNoon community! HackerNoon has partnered with Jobbio's Amply network to launch a specialized exclusively curated for the IT community. This strategic partnership marks a significant step forward in connecting talented IT professionals with innovative companies seeking top-tier tech talent. job board By partnering with Jobbio's Amply network, HackerNoon aims to provide a dedicated space where tech talent can find opportunities that truly resonate with their skills. Simply click ‘Jobs’ on the Top Nav Green Bar on , and explore thousands of tech jobs from about every top tech company worldwide! HackerNoon The collaboration merges Amply Network's expertise in seamless job matching with hackernoon.com's influence in the tech industry, creating a platform that caters to the evolving needs of the IT sector. Leveraging the strengths of both entities, this partnership represents a unique experience for the IT community in the form of a platform that showcases job opportunities and fosters a community of knowledge sharing and growth within the IT domain. "We believe that this partnership will revolutionize the way IT professionals seek job opportunities and companies discover exceptional tech talent. Our mission has always been to connect people with meaningful work, and this collaboration with Hackernoon aligns perfectly with that goal," - says Stephen Quinn, CEO of Jobbio. How the Works HackerNoon Job Board The curated jobs board features positions ranging from about every field of IT. Be it software development  and data science to cybersecurity and artificial intelligence. Tailored to cater to the IT community, the platform will ensure job seekers can explore roles aligning with their expertise and aspirations. Furthermore, companies posting on the board will benefit from access to a pool of engaged candidates, simplifying the often time-consuming recruitment process. It’s a win-win for everyone! Are you finding a new job in tech? The HackerNoon Jobs board gives you a unique and immersive experience to find your ideal match. 1. You can Filter Jobs by Title and Area You can filter your job search according to your expertise and preferred locality. You can also find thousands of remote work opportunities! 🎉 2. Search Job Openings in Your Favorite Companies Finding your dream job in your ideal firm is now super easy. Search by company and explore hundreds and thousands of job openings there. 3. Looking for the Hottest Tech Jobs in the Market? Check Out the Featured Jobs Section 4. Add Your Company to HackerNoon Job Board Do you want to hire top tech talent? List your company on the by clicking Add Your Job in the top nav. HackerNoon Job Board 5. Receive Top Tech Content and the Hottest Job Openings in our Daily Noonifications Newsletter What’s better than receiving the hottest jobs in the market alongside HackerNoon's top stories daily at noon in your inbox? ☕ Subscribe to the Noonfications Newsletters today! This partnership addresses the current demand for tech talent and signifies a broader shift in how industries collaborate to shape the future. The HackerNoon jobs board sets a new precedent for meaningful partnerships that amplify collective strengths and drive positive change.

HackerNoon & Jobbio's Amply Network Introduce a Job Board Exclusive for Tech Folks 

HackerNoon এবং Jobbio's Amply নেটওয়ার্ক প্রযুক্তির লোকদের জন্য একচেটিয়া একটি চাকরির বোর্ড চালু করেছে

HackerNoon und Jobbios Amply Network stellen eine Jobbörse exklusiv für Technikleute vor

HackerNoon y Amply Network de Jobbio presentan una bolsa de trabajo exclusiva para gente tecnológica

HackerNoon et le réseau Amply de Jobbio présentent un tableau des emplois exclusif pour les gens de la technologie

हैकरनून और जॉबियो के एम्प्ली नेटवर्क ने टेक लोगों के लिए विशेष जॉब बोर्ड पेश किया है

HackerNoon と Jobbio の Amply ネットワークが技術者限定の求人掲示板を導入

HackerNoon과 Jobbio의 Amply Network, 기술 전문가 전용 채용 게시판 소개

HackerNoon e Amply Network de Jobbio apresentam um quadro de empregos exclusivo para o pessoal da tecnologia

HackerNoon ve Jobbio'nun Amply Ağı, Teknoloji Çalışanlarına Özel Bir İş Panosu Sunuyor

HackerNoon & Jobbio's Amply Network giới thiệu bảng công việc dành riêng cho dân công nghệ

HackerNoon 和 Jobbio 的 Amply Network 推出专为科技人士打造的招聘板

Welcome to HackerNoon’s interview series. This section is curated by our Editorial team to spotlight bold and disruptive startups across the globe. If you’ve been nominated, and answer these questions Startups-Of-The-Year create a brand profile here. Hey Hackers, Apsy has been nominated in HackerNoon's annual Startup of the Year awards in Los Angeles, US. Please vote for us . here Read more about us below to understand why we deserve your vote. Meet Apsy Our start-up is called , a cool name rooted in Apps, but to us, it means “making app-building a breeze!” We are the first to introduce a fully AI-powered app builder. Our users speak their minds as our AI builds them. Apsy What makes us a different startup from others who offer tools to build apps is that we're all about letting your ideas flow while our AI does the heavy lifting. You dream it, speak it, and our AI builds it. Unlike traditional DIY tools, we take care of all the techy stuff, so our users don't have to. With Apsy, your creativity is the only limit. My Role Tooraj, a double Ph.D. dynamo in Economics and Computer Science, is a seasoned tech exec with a penchant for AI algorithms. A Y-Combinator and Techstars alumni is now driving his second startup, Apsy, towards reimagining software development. With a vision to make it simpler and cost-effective through AI, Tooraj is steering tech's future. How We're Disrupting/Improving the app development industry Building an app with traditional methods feels like a tough mountain to climb, right? But at Apsy, we're turning that hike into a fun walk in the park! Just chat with our AI about your brilliant app idea, answer a few clever questions it throws at you, and ta-da! Your app is up and running in a snap - we're talking hours, not months! We're shaking up the app-making game, making it a piece of cake for everyone, not just the coding wizards. Apsy isn't just transforming app development but the whole software landscape and tech entrepreneurship. Exciting times are ahead! Standing Out from The Crowd Do you remember how difficult building an app was in the good old days? Costly developers, scary DIY journeys, and months of learning only to end up with an 'okay-ish' app. Well, not anymore! We Apsy all you need to do is share your big idea, answer a few questions from our smarty-pants AI, and boom! It starts crafting screen options for you. And the best part? You can add your finishing touches and have a fabulous, tailor-made app ready to shine in just a few hours. And all this without burning a hole in your pocket! Building an app has never been this fun and easy, right? Our Predictions/Thoughts on the App Development Industry in 2023 The tech industry, specifically app development, is experiencing a significant shift towards no-code and low-code solutions. However, their complexity often presents challenges. Here, AI-driven platforms like Apsy come into play, simplifying app creation without needing users to be coding experts. As we move further into 2023 and beyond, we predict a surge in AI integration into app development owing to the rising demand for rapid, cost-effective digital solutions. The growth of AI sophistication will enable a better understanding of user needs and the delivery of high-quality, personalized apps. This AI infusion will transform the technological landscape and democratize the digital space, fostering innovation and promoting tech entrepreneurship. As Apsy, we're excited to pioneer this transformative journey, making app development more accessible and enjoyable. What word defines the state of app development in 2023? If you're asking about the vibe of app development in 2023, it's all 'Mind-to-Machine'! Just picture this - you come up with a killer app idea, chat about it, and voila, our AI buddy gets it done. No fuss, no code, just pure creativity! This exciting 'Mind-to-Machine' era is giving everyone the power to turn their app dreams into reality. It's like having a chat with a friend who happens to be a coding genius. So, buckle up, folks! We're on a fun-filled, game-changing journey that's turning the tech world on its head! Why we decided to participate in HackerNoon's Startup of the Year awards We jumped into HackerNoon's Startup of the Year awards because we're shaking things up with our 'Mind-to-Machine' tech at Apsy and wanted to share this cool innovation with fellow tech enthusiasts. Plus, who doesn't love a good challenge? We saw it as a great chance to learn, grow, and have some fun while competing with other innovative startups. So, here we are, all set to dazzle at the awards. Fingers crossed, folks! Final Thoughts At Apsy, we're on an exciting mission in 2023, transforming app development and making it accessible for all. We're thrilled to bring our innovation to HackerNoon's Startup of the Year awards, showcasing our approach, learning from peers, and embracing challenges. So, we invite you to on this thrilling journey, where we'll turn dreams into digital realities together! join us Don’t forget to ! Vote for us today

Vote for Apsy in HackerNoon's Startups of the Year 2023

Since Facebook released its previously internally used query language GraphQL in 2016, it has seen an outstanding increase in adoptions for all kinds of applications. So it didn’t come as a surprise when AWS built a managed solution called AppSync and released it in 2021. AppSync is becoming more and more popular and is already being used by large companies, including . BMW and Sky AppSync for GraphQL You may already be thinking that building and operating infrastructure for a GraphQL endpoint are tedious tasks, especially if it needs to be scalable for increasing or ever-changing workloads. This is where AWS AppSync steps in by providing a fully managed service and a GraphQL interface to build your APIs. It’s feature-rich, as it allows you to easily connect resolvers to Lambda, OpenSearch, and Aurora or even directly attach DynamoDB tables via VTL templates. Essentially, you’ll get a managed microservice architecture that connects to and serves data from multiple sources with just a few clicks or lines of infrastructure code. AWS will manage your infrastructure, and you can focus on the implementation of resolvers. Monitoring your endpoints Sadly, the “managed” in “managed service” doesn’t release you from duties. There’s no need to monitor infrastructure, as all infrastructure components are AWS’s responsibility, but you still need to monitor your GraphQL endpoints to identify errors and performance bottlenecks. all AppSync integrates with CloudWatch natively and forwards a lot of metrics about your AppSync endpoints without the need for further configuration. Metrics The metrics reported by AppSync by CloudWatch are: Number of 4xx errors Number of 5xx errors Request latency – how long it takes to fully process a request, including possible response template mappings Sample count statistics – the number of API requests If caching is enabled for your AppSync endpoint, the cache statistics include: Hits Misses Current items Evictions Reclaimed This provides a good overview of how your API is doing, but no detailed view per resolver is possible because everything is aggregated. You can’t tell which resolver is the issuer of an HTTP 500 error. Furthermore, there’s no insight about any timings about the resolvers involved in a request. A query could be resolved via a lot of nested resolvers, which may result in high latencies for just a subset or a single slow resolver. This is where logging comes in to help. Logging As previously mentioned, it’s important to know all details about AppSync’s processing steps for each request to gather insights about failing queries and other errors. Additionally, you require in-depth information about the performance of your GraphQL APIs and your resolvers. This can be achieved by enabling logging for your APIs. AppSync allows you to configure this via multiple settings within the console, including: – tells AppSync to start recording logs. It will generate execution and request summary logs that contain useful information we can use to monitor performance and errors. Enable Logs – extends logs to include more details, including request and response headers, context, and evaluated mapping templates. Include verbose content – set this to or to get further enhanced tracing logs that contain details about how long queries took to resolve . Field resolver log level Error All per attribute Even when verbose content is not enabled for your logs, the request and execution summary logs already show a lot of useful information. This includes how long the parsing and validation steps took, as well as how long it took to complete the whole request. With this configuration, we can now monitor the error rate, the errors themselves, and the latency of the top-level resolvers but nothing about nested resolvers. Field resolver log-level settings are required to get all the information to debug nested resolvers’ performances. With this, we’ll also get tracing messages per resolved attribute. This will enable us to analyze query performance in a very fine-grained manner, as we can see how long each resolver takes to collect the result of a field. Additionally, we’ll get and logs for each (nested) resolver. By also enabling verbose content, these logs will be enhanced with the GraphQL request and both request and response headers, as well as the context object itself. This means that if we’re investigating a DynamoDB resolver, we can see the mapping that was done by our VTL template and identify issues quickly. RequestMapping ResponseMapping 💡 Important detail about having field resolvers’ log-level settings set to : the number of generated logs will not increase slightly but by an order of magnitude. For a frequently used AppSync endpoint, this will drastically increase costs due to the log ingestion as well as storage costs at CloudWatch. A great mitigation strategy to avoid exploding costs is to set proper log retention and have only small windows of detailed field resolver logs. All The latter can, for example, be achieved by using scheduled EventBridge rules and Lambda, which will switch between resolver log-level configurations of “Error” and “All” regularly. Depending on the schedule, you’ll end up with only a fraction of the logs and, therefore, costs without losing insights into your AppSync endpoint. Switching between field resolver log-level settings via Lambda + AppSync AWS X-Ray AppSync integrates with X-Ray, allowing you to get trace segment records and a corresponding timeline graph. This helps you visually identify bottlenecks and pinpoint their resolvers at fault. Sadly, the subsegments only show rudimentary information about the request and response and don’t provide further details that would help to debug problems. Everything is just collected under the AppSync endpoint. It can be filtered, but it’s not flexible enough to cover the requirements necessary for GraphQL. How to monitor with AWS AppSync Dashbird As with many other services, you’ll face the classic downsides: a lack of visibility, a lot of noise, and a clunky integration with CloudWatch alarms and logs. is here to help you get around these limitations so that instead of fighting with the AWS console. We have just added support for AppSync to help you monitor all of your AppSync endpoints without needing to browse dozens of logs or stumble through traces in the X-Ray UI. Dashbird you can focus more on building a great product There’s no need to set up anything besides our CloudFormation template and the necessary permissions to ingest logs via CloudWatch and X-Ray. Dashbird will then automatically collect all needed information – without any performance impact on your application – and prepare it graphically. At a single glance, you’ll see . Furthermore, all requests will be displayed with their status code, request latency, and flag if there were any resolver errors. By clicking on the request itself, you can drill down into it and look at the query and context. fundamental information about the performance and error rate of your AppSync endpoint This shows all the involved resolvers and how much they contributed to the overall response latency, which is crucial information for finding bottlenecks. There’s also a list of any resolver issues. Clicking on a resolver issue will take you to the error overview, giving you more details about when this error first occurred and how many errors of this type have already appeared. There’s also out-of-the-box alerting via Slack, email, or SNS. Since Dashbird automatically clusters similar errors, noise will be reduced, and there’s no need to configure anything in advance to receive critical information in your inbox or channel without flooding it and causing cognitive overload. Also, metrics provided by AWS are enhanced to enable better filtering. For example, CloudWatch’s requests metric will only give you the number of requests that have processed in a single region, but there’s no way to know how many requests have been made to a single API or endpoint. With Dashbird, you can always pinpoint the number of requests per endpoint for any given period of time. all APIs Key takeaways Monitoring is fundamental to any web application. Even though AppSync offers a high level of abstraction of the underlying infrastructure, keeping your endpoint and its resolvers healthy and maintaining low latency is still your job. CloudWatch and X-Ray offer tools that enable you to get the logs required to achieve detailed observability for your application relying on AWS’s managed GraphQL solution. Dashbird takes this to the next level by offering you a structured overview of all your AppSync endpoints, which contain all the details you need to debug errors, resolve bottlenecks, and give your team more time for application development instead of stumbling through CloudWatch logs and X-Ray traces. This article was first published here.

Monitor Your AppSync GraphQL APIs with Simplicity

I have been working with a US client to build a first-of-its-kind app for managing medical consents. It falls under HIPAA compliance and it’s paramount that we do not allow unauthorized access to user data. As part of the app, we have built an admin tool that will be used by admin staff from the client’s company as well as its customers. Essentially, we have three roles in the admin tool: : these are admin staffs from the client’s company. This group of user have access to all the functionalities, including adding new tenants. Admin : these are admin staffs from the client’s customers which are typically Health Information Exchanges (HIEs). An HIE would collaborate with multiple healthcare providers (e.g. hospitals, pharmacies) to facilitate data exchange between them. An HIE’s admin staffs should therefore have access to information related to their own organization as well as any healthcare providers that it collaborates with. Tenant : these are admin staffs from the healthcare providers that collaborate with the HIEs above. They only have access to information related to their own organization as well as its patients. Service Area This is a drastic oversimplification of the entities that are involved, but it serves to illustrate the different tiers of access that we need to model: users can access everything. Admin users can access its own data plus data that belongs to its subsidiary service areas. Tenant users can access only its own data. Service Area The good news is that AppSync makes it easy to implement group-based authentication with Cognito. Which is why we decided to go with AppSync over API Gateway for this project. one of the reasons However, modelling these overlapping actions in the GraphQL schema was still a big challenge in and of itself. The problem The root of the problem is that there are many overlapping actions that each group can perform. For example, users can add new service areas and associate them with any . A user can also add new service areas, but cannot associate the service area with anyone but itself. Admin Tenant Tenant As such, we need two separate mutations with different inputs and lock down their access to different Cognito groups. As you can see from below, as a user, I only have access to the mutation where I cannot override the the service area is associated with. Tenant addServiceAreaAsTenant tenantId Mutation {
  add : ServiceArea!
  @aws add : ServiceArea!
  @aws } type ServiceAreaAsAdmin( : ID!, : String!) tenantId name _auth( : [ ]) cognito_groups "Admin" ServiceAreaAsTenant( : String!) name _auth( : [ ]) cognito_groups "Tenant" As we add more and more queries and mutations this quickly becomes messy. There are a lot of duplications for the clauses, and it’s not easy to see at a glance what each group of users can do. Also, we have to get increasingly creative about how we name these queries and mutations in order to distinguish them from one another. @aws_auth(cognito_groups: …) Copy-and-paste errors can easily creep in (it almost did), and it’s also confusing for the frontend developers to keep track of which queries and mutations they should use for which users. { : Query : Mutation
} { ( : ID!): !
  @ ( : [ ]) ( : ID!): !
  @ ( : [ ]) : !
  @ ( : [ ]) ( : ID!): !
  @ ( : [ ]) : !
  @ ( : [ ])
} { ( : String!): !
  @ ( : [ ]) ( : ID!, : String!): !
  @ ( : [ ]) ( : String!): !
  @ ( : [ ]) ( : String!): !
  @ ( : [ ]) ( : String!): !
  @ ( : [ ])
} { : ID! : String!
} { : ID! : String! : ID!
} schema query mutation type Query getTenantById id Tenant aws_auth cognito_groups "Admin" getServiceAreaById id ServiceArea aws_auth cognito_groups "Admin" getMyProfileAsTenant Tenant aws_auth cognito_groups "Tenant" getMyServiceArea id ServiceArea aws_auth cognito_groups "Tenant" getMyProfileAsServiceArea ServiceArea aws_auth cognito_groups "ServiceArea" type Mutation addTenant name Tenant aws_auth cognito_groups "Admin" addServiceAreaAsAdmin tenantId name ServiceArea aws_auth cognito_groups "Admin" updateMyProfileAsTenant name Tenant aws_auth cognito_groups "Tenant" addServiceAreaAsTenant name ServiceArea aws_auth cognito_groups "Tenant" updateMyProfileAsServiceArea name ServiceArea aws_auth cognito_groups "ServiceArea" type Tenant id name type ServiceArea id name tenantId Using nested Query/Mutation types To make it easier to scale the complexity of this project, we decided to encapsulate each group into its own and types. Query Mutation As you can see below, each group of users gets its own and types. Access is controlled in a single place, and it’s easy to see everything a group of users can do at a glance. Also, we don’t need to rely on naming conventions on overlapping actions anymore. Query Mutation schema { 
  query: Query
  mutation: Mutation
} Query {
  asAdmin: AdminQuery
  @aws_auth(cognito_groups: [ ])
  
  asTenant: TenantQuery
  @aws_auth(cognito_groups: [ ])

  asServiceArea: ServiceAreaQuery
  @aws_auth(cognito_groups: [ ])
} Mutation {
  asAdmin: AdminMutation
  @aws_auth(cognito_groups: [ ])

  asTenant: TenantMutation
  @aws_auth(cognito_groups: [ ])

  asServiceArea: ServiceAreaMutation
  @aws_auth(cognito_groups: [ ])
} AdminQuery {
  getTenant(id: ID!): Tenant!
  getServiceArea(id: ID!): ServiceArea!
} TenantQuery {
  getMyProfile: Tenant!
  getMyServiceArea(id: ID!): ServiceArea!
} ServiceAreaQuery {
  getMyProfile: ServiceArea!
} AdminMutation {
  addTenant(name: String!): Tenant!
  addServiceArea(tenantId: ID!, name: String!): ServiceArea!
} TenantMutation {
  updateMyProfile(name: String!): Tenant!
  addServiceArea(name: String!): ServiceArea!
} ServiceAreaMutation {
  updateMyProfile(name: String!): ServiceArea!
} Tenant {
  id: ID!
  name: String!
} ServiceArea {
  id: ID!
  name: String!
  tenantId: ID!
} type "Admin" "Tenant" "ServiceArea" type "Admin" "Tenant" "ServiceArea" type type type type type type type type This makes the GraphQL schema much easier to maintain as the project continues to grow, without breaking the existing security model. As an user, I’m able to perform the queries and mutations specified in the and types. Admin AdminQuery AdminMutation As a user, the of my organization is captured as a custom attribute on my Cognito user. Tenant tenantId The queries and mutations for a user never accept as an argument, and it’s always taken from . This ensures tenants can only access its own data, without us having to litter our code with custom validation logic. Tenant tenantId $context.identity And since I don’t belong to the group, I’m not authorized to access its queries and mutations. Admin But I’m able to act on my own data, as well as access data for service areas that are associated with my organization. And the same goes to users, whose access will be restricted to its own data. Service Area In addition to this, HIPAA compliance also requires us to have an audit trail of the data a user has accessed. So we also had to implement full request-response logging, which sadly does not come out-of-the-box with AppSync. We’ll dive into this particular topic in a later post. In the meantime, if you want to see how this hierarchical model works for yourself, then check out and try it out. this repo If you want to learn GraphQL and AppSync with a hands-on tutorial then please check out my where you will build a Twitter clone using a combination of AppSync, Lambda, DynamoDB, Cognito and Vue.js. You will learn about different GraphQL modelling techniques, how to debug resolvers, CI/CD, monitoring and alerting and much more. new AppSync course Previously published at https://theburningmonk.com/2020/08/how-to-model-hierarchical-access-with-appsync/

Modeling Hierarchical Access With AppSync In 3 Simple Steps

One thing that API Gateway supports but you can’t do with AppSync out-of-the-box yet is custom domain names. Your shiny new AppSync API is available at , but you really want people to use your own domain instead because is much more memorable and informative. XYZ.appsync-api.us-east-1.amazonaws.com/graphql dev.example.com/graphql In this post, let’s look at two ways you can do this. I previously wrote about five reasons you should consider AppSync over API Gateway. . Check out the article here 1. CloudFront This is my preferred way. It’s . easy to set up and cheap to run Assuming your AppSync API resource is called (if you use the with the Serverless framework, then this is the logical id it’ll use). This is the CloudFormation resource you need to configure to route traffic to the AppSync API. GraphQlApi serverless-appsync-plugin AppSyncDistribution: Type: AWS::CloudFront::Distribution Properties: DistributionConfig: Origins: - Id: AppSync DomainName: Fn: :Select: - '2' - Fn: :Split: - "/" - Fn: :GetAtt: - GraphQlApi - GraphQLUrl CustomOriginConfig: HTTPPort: 80 HTTPSPort: 443 OriginProtocolPolicy: https-only OriginSSLProtocols: - TLSv1 - TLSv1.1 - TLSv1.2 OriginPath: '' Enabled: true HttpVersion: http2 Comment: CloudFront distribution for the Patient AppSync API Aliases: - <INSERT DOMAIN NAME HERE, e.g. dev.example.com> PriceClass: PriceClass_100 DefaultCacheBehavior: AllowedMethods: - HEAD - OPTIONS - GET - DELETE - POST - PUT - PATCH CachedMethods: - HEAD - OPTIONS - GET ForwardedValues: QueryString: true Headers: - x-api-key - Authorization Cookies: Forward: all MinTTL: 0 DefaultTTL: 0 TargetOriginId: AppSync ViewerProtocolPolicy: redirect-to-https Compress: true CustomErrorResponses: [] ViewerCertificate: AcmCertificateArn: <INSERT ARN HERE> SslSupportMethod: sni-only If you’re using the , then you should also check out the plugin. It can make things a lot easier for you. However, at the time of writing, it makes assumptions about the logical Id of the AppSync API (it expects ) as it’s intended to used alongside the aforementioned . Serverless framework serverless-appsync-cloudfront GraphQlApi serverless-appsync-plugin 2. API Gateway Another way you can do this is through API Gateway. You can set up an HTTP proxy that routes traffic to the AppSync API, and then configure a custom domain name in API Gateway. Compared to using CloudFront, this approach takes more work to set up and incurs higher cost and latency (compared to using CloudFront) because every request has to go through API Gateway. If you’re using the Serverless framework, then another stumbling block is that you can’t declare an API Gateway without any Lambda functions. Luckily, you can work around this by bringing SAM’s macro into the like in . easily AWS::Serverless-2016–10–31 serverless.yml this post This lets you configure API Gateway using SAM’s resource type. AWS::Serverless::Api ApiGateway: Type: AWS::Serverless::Api Properties: Name: AppSyncApiProxy StageName: dev OpenApiVersion: "3.0.1" DefinitionBody: openapi: "3.0.1" paths: /{proxy+}: "x-amazon-apigateway-any-method": consumes: - "application/json" produces: - "application/json" parameters: - name: proxy in: path required: true type: string x-amazon-apigateway-integration: uri: Fn: :Join: - '' - - 'https://' - Fn: :Select: - '2' - Fn: :Split: - "/" - Fn: :GetAtt: - GraphQlApi - GraphQLUrl - '/{proxy}' passthroughBehavior: when_no_match httpMethod: ANY type: http_proxy requestParameters: integration.request.path.proxy : method.request.path.proxy EndpointConfiguration: REGIONAL ApiGatewayDomainName: Type: AWS::ApiGateway::DomainName Properties: DomainName: <INSERT DOMAIN NAME> EndpointConfiguration: Types: - REGIONAL RegionalCertificateArn: <INSERT ARN HERE> ApiGatewayBasePathMapping: Type: AWS::ApiGateway::BasePathMapping DependsOn: - ApiGatewayDomainName Properties: DomainName: <INSERT DOMAIN NAME> RestApiId: !Ref ApiGateway Stage: !Ref ApiGateway.Stage One thing to note here is that, by default, SAM creates an API Gateway stage called . This appears to be a long-standing bug with SAM and the workaround is to set to . Stage OpenApiVersion 3.0.1 Route53 As a final step for both of these approaches, you need to configure a Route53 record for them in your hosted zone. For CloudFront, you need something like this: For API Gateway, you need to capture the for the custom domain (in API Gateway) and configure a Route53 record against it. API Gateway domain name You can also configure these using CloudFormation. Summary With both of these approaches, you can create a custom domain name (such as ) for AppSync. Callers can access your AppSync API via which is much more user-friendly and memorable. dev.example.com dev.example.com/graphql As mentioned above, I prefer the CloudFront approach because it’s easier to configure and has a negligible cost overhead. Previously published at https://theburningmonk.com/2020/09/how-to-set-up-custom-domain-names-for-appsync/

How to Set up Custom Domain Names for AppSync

AppSync has built-in logging integration with CloudWatch Logs (see for more details on the logging options). here You can set the to , or . Field resolver log level NONE ERROR ALL When the field resolver log level is set to or , you don’t get a lot of value from the AppSync logs. For example, this is what I see for a GraphQL request: NONE ERROR You can see the but have no way of figuring out what contributed towards that latency. Really, there’s not a lot you can do with the information here. latency But if you set the field resolver log level to , then you suddenly see EVERYTHING! ALL The request and response mapping for each resolver, and how long each of the fields took to resolve. This is great! But, given the amount of information that is sent to CloudWatch Logs, this has a non-trivial cost implication. As you can see below, CloudWatch Logs charges $0.50 per GB of data ingested. This is equivalent to the cost of 125,000 Query/Mutation requests to AppSync, and I bet those operations would generate a lot more than 1 GB of logs! In practice, you wouldn’t want to leave the at in a production environment. The cost would be too significant. Field resolver log level ALL Unfortunately, there is no built-in sampling support, unlike with X-Ray. Sampling resolver logs via cron jobs One workaround is to use Lambda functions to toggle the between and via a pair of cron jobs. Field resolver log level ALL ERROR This is exactly what I did on a client project recently to keep a healthy balance between cost and having visibility into what my application is doing. To do this, I configured two cron jobs (with Lambda). turnOnResolverLogging: handler: functions/toggle-resolver-log-level.handler events: - schedule: cron(8 * * * ? *) environment: APPSYNC_API_ID: !GetAtt AppsyncapiGraphQlApi.ApiId FIELD_LOG_LEVEL: ALL iamRoleStatements: - Effect: Allow Action: - appsync: GetGraphqlApi - appsync: UpdateGraphqlApi Resource: !Ref AppsyncapiGraphQlApi - Effect: Allow Action: iam:PassRole Resource: !GetAtt AppSyncLoggingServiceRole.Arn turnOffResolverLogging: handler: functions/toggle-resolver-log-level.handler events: - schedule: cron(10 * * * ? *) environment: APPSYNC_API_ID: !GetAtt AppsyncapiGraphQlApi.ApiId FIELD_LOG_LEVEL: ERROR iamRoleStatements: - Effect: Allow Action: - appsync: GetGraphqlApi - appsync: UpdateGraphqlApi Resource: !Ref AppsyncapiGraphQlApi - Effect: Allow Action: iam:PassRole Resource: !GetAtt AppSyncLoggingServiceRole.Arn These cron jobs would run on the 8th and 10th minute of each hour, giving me 2 minutes worth of resolver logs per hour. This gives me resolver logs for roughly 3% of Query/Mutation operations. The function looks like this: AppSync = ( ) appSync = AppSync() { APPSYNC_API_ID, FIELD_LOG_LEVEL } = process.env .exports.handler = (event) => { resp = appSync.getGraphqlApi({ : APPSYNC_API_ID
  }).promise() api = resp.graphqlApi
  api.logConfig.fieldLogLevel = FIELD_LOG_LEVEL api.arn api.uris api.tags appSync.updateGraphqlApi(api).promise()
} const require 'aws-sdk/clients/appsync' const new const module async const await apiId const delete delete delete await To make sure I don’t accidentally change any of the other configurations, I fetch the current API configuration first. But since the request doesn’t accept , nor , I have to remove these or the request will fail validation. updateGraphqlApi arn uris tags But wait! What if I want to keep all the resolver logs in dev and only enable sampling in production? Well, you can use the plugin to conditionally exclude these two functions for the stage, like this: serverless-ifelse dev serverlessIfElse: - If: '"${self:custom.stage}" == "dev"' Exclude: - functions.turnOnResolverLogging - functions.turnOffResolverLogging The approach I outlined above is simple and gives me a sufficient amount of resolver logs without blowing up my CloudWatch cost. However, if you want the sampling to be spread more evenly across the hours then you might consider this alternative—run a cron job every few minutes and have the Lambda function: set the resolver log level to ALL busy wait for, say 10 seconds set the resolver log level back to ERROR That way, you will have resolver logs for Query/Mutation requests for 10 seconds every few minutes instead of 2 minutes every hour. Don’t forget to update log retention Another important thing to remember is to update the log retention policy for your log groups. Because by default, CloudWatch Logs will never expire your logs and you will pay $0.03 per GB per month for an ever-increasing pile of logs that are less useful with every passing minute. The best way to manage the log retention for all your log groups—past, present and future—is to deploy to your region and let it do the rest. this SAR application This SAR application would update all your existing log groups during deployment. And when new log groups are created in the future, their retention policy will also be updated automatically. Master AppSync and GraphQL If you want to learn GraphQL and AppSync with a hands-on tutorial then please check out my where you will build a Twitter clone using a combination of AppSync, Lambda, DynamoDB, Cognito and Vue.js. You will learn about different GraphQL modelling techniques, how to debug resolvers, CI/CD, monitoring and alerting and much more. new AppSync course Previously published at https://theburningmonk.com/2020/09/how-to-sample-appsync-resolver-logs/

How to sample AppSync resolver logs

I have been working on a large AppSync project for a client these past few months. The initial version of the app was built in just a few weeks, but the client has commissioned additional features and the project has kept growing. At the time of writing, this project has over 200 AppSync resolvers and 600 CloudFormation resources. Along the way, I learnt a few things about scaling a large AppSync project like this and hope you find them useful too. But first, let me catch you up on where we are in the project: - I’m the only backend developer on this project. - This project uses 100% serverless components – AppSync, DynamoDB, Lambda, S3, etc. - There is a monorepo for the entire backend for the app. - There is one Serverless framework project (as in, one ) in the repo, and one CI/CD pipeline. serverless.yml - This contains every backend infrastructure – AppSync APIs, DynamoDB tables, S3 buckets, Lambda functions, etc. serverless.yml - The project uses the to configure the AppSync APIs. serverless-appsync-plugin - There are two AppSync APIs – one for the mobile app and one for a browser-based CMS system. - There are two Cognito User Pools – one for the mobile app and one for the CMS. - There are a total of 26 DynamoDB tables. There is one table per entity, with a few exceptions where I applied Single Table Design for practical reasons. This might be very different from how your projects are set up. Every decision I made here is to maximize feature velocity for a small team. The client is a bootstrapped startup and cannot afford a protracted development cycle. If you want to learn more about this project and how I approach it from an architectural point-of-view, please watch . this talk On a high-level, the backend infrastructure consists of these components: - CloudFront and S3 for hosting static assets. - Cognito User Pool for authentication and authorization. - AppSync mostly integrates directly with DynamoDB tables. But more complex operations are moved into Lambda functions instead. - Algolia for search. It’s the closest thing (that I have found) to a “serverless ElasticSearch”. - DynamoDB Streams are used to trigger Lambda functions to perform background tasks such as synching changes to Algolia. - Firehose is used to collect BI (business intelligence) events and stores them in S3. - Athena is used to run reports against these BI events. Even though AppSync integrates with DynamoDB directly in most cases, there are still quite a few Lambda functions in the project. As you can see from the in . System Map Lumigo In the Beginning At first, everything was in one CloudFormation stack and it didn’t take long before I hit CloudFormation’s 200 resources limit per stack. Since I’m using the , I can use the to migrate resources into nested stacks. Serverless framework split-stacks plugin The advantage of this approach is that it doesn’t require any code change to my . Existing references through and still work even when the resources have been moved around. serverless.yml !Ref !GetAtt The plugin converts these references into CloudFormation in the nested stack where the references originate from (let’s call this ): split-stacks Parameters NestedStackA If the referenced resources are defined in the stack, then they are passed into the nested stack as parameters. root If the referenced resources are defined in another nested stack (let’s call this ), then the referenced values are included in the of . The stack would use to pass these outputs as parameters to . NestedStackB Outputs NestedStackB root GetAtt NestedStackA In the generated CloudFormation template, this is what it looks like. Here the stack passes the output from one nested stack as a parameter to another. root The plugin has a number of built-in migration strategies – , and . Per Lambda Per Type Per Lambda Group However, I needed more control of the migration process to avoid circular dependencies. Fortunately, the plugin gives me of the migration process by adding a module at the root of the project. fine-grained control stacks-map.js Beware of Circular References How you organize the resources into nested stacks is important. There are ample opportunities for circular references to happen. This is especially true when you’re working with AppSync as there are quite a few different types of resources involved. For example, references Lambda functions or DynamoDB tables, and also references the for . AWS::AppSync::DataSource AWS::AppSync::GraphQLApi ApiId All and all, these are the resource types I have for the AppSync APIs. The arrows represent the direction of the reference. As the project grew, I had to get creative about how I group the resources into nested stacks. This post describes the 3 stages of that evolution. Stage 1 – group by API As a first attempt, I sliced up the resources based on the AppSync API they belong to. The DynamoDB tables are kept in the stack since they are shared by the two AppSync APIs. Other than that, all the other resources are moved into one of two nested stacks (one for each AppSync API). root Additionally, the plugin automatically puts the resources into its own nested stack. split-stacks AWS::Lambda::Version This was the simplest and safest approach I could think of. There was no chance for circular dependencies since the two nested stacks are independent of each other. Although they both reference the same DynamoDB tables in the stack, there are no cross-references between themselves. root But as you can see from the screenshot above, there are a lot more resources in the than the . Pretty soon, the nested stack for the AppSync API for the mobile app would grow too big. AppSyncNestedStack AppSyncCmsNestedStack Stage 2 – moving Lambda functions out The contains a lot of resources related to its Lambda functions. So the natural thing to do next was to move the Lambda function resources out into their own nested stack. AppSyncNestedStack This made a big difference and allowed the project to grow much further. But soon, I hit another CloudFormation limit – a stack can have only 60 parameters and 60 outputs. This was because the project now has over 60 Lambda functions. So the had over 60 resources, each requiring a reference to the corresponding . Therefore, the stack had over 60 parameters, one for every function. VersionsNestedStack AWS::Lambda::Version AWS::Lambda::Function And so I also had to split the in two. VersionsNestedStack This bought me time, but the project kept growing… As the AppSync API for the mobile app approach 150 resolvers, the hit the 200 resources limit again. AppSyncNestedStack Stage 3 – group by DataSource From here on, I need to find a strategy that minimizes the number of cross-stack references so to not run foul of the 60 parameters limit per stack. Therefore, each nested stack needs to be as self-contained as possible. Looking at the resource graph, I realised that the data sources are at the centre of everything. If I start from the data sources, I can create groups of resources that revolve around a single data source (the orange resources below) and are independent of other such groups. I can pack these mutually-independent groups into nested stacks. Since they don’t reference resources in another group, there’s no chance for circular references. With this strategy, I am now able to split the resources into many more nested stacks. Compared to my earlier attempts, this approach is also very scalable. If need be, I can add as many nested stacks as I want (within reasons). With this change, I was able to add another batch of resolvers to support a new feature. Other considerations As you move resources around, there are several things to keep in mind. No duplicate resource names For instance, if a Lambda function is moved from one nested stack to another, then the deployment will likely fail because “A function with the same name already exists”. This race condition happens because the function’s new stack is deployed before its old stack is updated. The same problem happens with CloudWatch Log Groups as well as IAM roles. To work around this problem, I add a random suffix to the names the Serverless framework generates for them. No duplicate resolvers Similarly, you will run into trouble if an AppSync resolver is moved from one nested stack to another. Because you can’t have more than one resolver with the same and . TypeName FieldName Unfortunately, I haven’t found any way to work around this problem without requiring downtime – to delete existing resolvers, then deploy the new nested stacks. Instead, my strategy is to pin the resolvers to the same nested stack by: 1. use a fixed number of nested stacks 2. hash the logical ID of the data source so they are always deployed to the same nested stack 3. when I need to increase the number of nested stacks in the future, hardcode the nested stack for existing data sources If you can think of a better way to do this, then please let me know! So that’s it on how I scaled an AppSync project to over 200 resolvers, hope you have found this useful! If you want to learn GraphQL and AppSync with a hands-on tutorial then please check out my where you will build a Twitter clone using a combination of AppSync, Lambda, DynamoDB, Cognito and Vue.js. You will learn about different GraphQL modelling techniques, how to debug resolvers, CI/CD, monitoring and alerting and much more. new AppSync course Previously published at https://theburningmonk.com/2020/07/how-i-scaled-an-appsync-project-to-200-resolvers/

6 Tips To Scale an AppSync Project To 200+ Resolvers That Will Blow Your Mind

While using any services from AWS you can do it via their API,Console or CLI tool like amplify, amplify was designed specifically to work with AWS and to make it easier for developers to deploy applications on the cloud just by running some commands, In this blog i will be showing you how can you create an Appsync API with Amplify, It will have auto generated resolvers for all the queries,mutation and DynamoDB as the database.Yes you read that right everything will be created for you by AWS using cloudformation which acts as template for all the services that you will be needing. So before we get started i encourage you to install Amplify CLI on your computer and configure it with your Credentials. Let’s get started now. Step 1: Create a folder in your preferred directory and run the following command amplify init This will initialize the folder as an amplify project and in the left hand side you can see all the boilerplate code already created for you.Configure all the setting according to your need or keep it as default. Step 2: Run the following command to add API to your amplify project. Configure this using the following settings or modify if you wish. amplify add api In the above image you can see that for the question Do you have an annotated GraphQL schema, you can select yes if you have your schema in place. Right now for this demo i am selecting No because then AWS will help me make a schema from scratch. Step 3: Now your GraphQL schema has been generated for your ToDo app you can see it in your folder structure. Once you are satisfied with the schema that has been created you can run amplify push to publish all the changes to the cloud. Now run the following command: amplify push Now grab a cup of coffee and wait because it is going to take some time for your database,API and resolvers to be created. Next choose if you want the resolvers to be created (Resolvers are used to tell the API how and where to fetch the data from) Step 4: This is not technically a step, here i will be discussing the folder structure and where you will find everything: 1. Resolvers for Query and mutation: amplify>backend>api> >resolvers 'api name' 2. Query,mutation that will be sent from client side to the API: src>graphql>mutation/queries Now you are all set to do query in your appsync console: Conclusion: So as you saw an API was created in just few steps, AWS Amplify CLI makes it easier to develop your backend. Not only API but you can even add authentication and much more to your API using simple commands provided by Amplify CLI.

Creating Appsync API using Amplify CLI

AWS AppSync within Lambda How to call AppSync queries and mutations within AWS Lambda and Cognito Pools I’m using AWS AppSync to build . 90% of the time it’s great, but there are still some pretty glaring feature gaps. One of the biggest is not being able to use a private key to access GraphQL operations through AWSAppSyncClient (for example, when doing backend operations with Lambda). It seems like there should be some kind of out-of-the-box admin access that allows you to mutate and query the database without needing to authenticate using AWS Cognito. Floom If you’re using AppSync, you’ve probably configured your GraphQL API to use AWS Cognito pools (which is the default). Using Cognito pools is incredibly convenient when for controlling access for different users on the client side. It allows you to use the @auth decorator in your schema.graphql to define read (list, get) and write (create, update, delete) access points. If this is the case, and you want to access the GraphQL operations on your own backend servers (or on AWS Lambda), you are basically limited to two options: You can create an admin user in your Cognito Pool and then use that user to get access credentials for the AWSAppSyncClient SDK. There is a workaround using AWS IAM roles which is . This involves creating authenticated and unauthenticated roles, and then creating an Amazon Cognito identity pool and linking the roles you just created to the identity pool. This will also involve changing the authentication mechanism within your client applications. You will no longer be able to use the @auth decorator within schema.graphql, unless you write your own resolvers. outlined here Here, I’ll explain how to move forward with option #1. A bird’s eye view of the process looks like this: Set up an admin user in AWS Cognito Use the admin’s login credentials to get a session jwtToken from AWS Cognito Use this jwtToken to get an instance of AWSAppSyncClient Now you can use AppSync GraphQL queries and mutations just like you do on the front end. Creating an Admin User I create a user with username admin and add this user to a group that’s also called admin. Once you’ve done this, make sure that this user will have access to the GraphQL operations you’ll need within your Lambda functions. For example, at , we need to a Transaction once the order has been fulfilled by the seller, so I add an @auth rule which allows users in the admin group complete read/write access to the Transaction model. Floom <a href="https://medium.com/media/70ebc4384cdd214733edf6d287f13e4f/href">https://medium.com/media/70ebc4384cdd214733edf6d287f13e4f/href</a> Getting A Session Token in Lambda To access AppSync’s GraphQL operations within Lambda, without needing to switch to AWS_IAM authentication, we’ll need to fetch session tokens using our admin user. We can do this through the CognitoIdentityServiceProvider module of the aws-sdk: <a href="https://medium.com/media/4977617ab3bde579281c5e1194700bfa/href">https://medium.com/media/4977617ab3bde579281c5e1194700bfa/href</a> Instantiating AWSAppSyncClient Now that we have Cognito credentials, we can use them to instantiate an instance of the AWSAppSyncClient object, which will allow us to use the AppSync GraphQL queries and mutations. <a href="https://medium.com/media/4c342d5f71f30d71d331e1b8f61f8ff6/href">https://medium.com/media/4c342d5f71f30d71d331e1b8f61f8ff6/href</a> Using AWSAppSyncClient When you amplify push your schema.graphql changes, Amplify will provide you with a queries.js and a mutations.js with GraphQL operations that look like this: <a href="https://medium.com/media/9be572942aed8a184223de8c515da296/href">https://medium.com/media/9be572942aed8a184223de8c515da296/href</a> You can either import this whole file to use in your Lambda function, or simply copy over the ones that you’ll need (remember to update them if you make changes to your schema.graphql. <a href="https://medium.com/media/fe5e9b01777c15a3680d1f7eb0f5d197/href">https://medium.com/media/fe5e9b01777c15a3680d1f7eb0f5d197/href</a> Now you can use AppSync’s GraphQL operations in a Lambda setting while still using Cognito Authentication (without AWS_IAM roles). When you call mutations, the Amplify GraphQL clients can even subscribe to these mutations using the subscriptions provided in subscriptions.js, like onUpdateTransaction in this case.

Stories, News & Mentions about AMPSY

Ampsy’s Stories on HackerNoon