Welcome to Cloud-Native Development with AWS Cloud9 & AWS CodeStar ⚡🎉

I have been experimenting with [AWS Cloud9](https://aws.amazon.com/cloud9/) since Werner Vogels announced it a few weeks ago at AWS re:Invent 2017 ([keynote video here](https://www.youtube.com/watch?v=fwFoU_Wb-fU)). This article is the paraphrased version of my talk [AWS Cloud9 & CodeStar for Serverless Apps](https://clda.co/aws-cloud9-codestar), given at the [AWS User Group in Milan](https://www.meetup.com/AWSusergroupItaly) on Jan 10th. I am going to skip the “_Serverless Disclaimer_” section of my deck. If you are not familiar with Serverless, please have a look [here](https://medium.com/@PaulDJohnston/a-simple-definition-of-serverless-8492adfb175a), or [here](https://martinfowler.com/articles/serverless.html), or [here](https://auth0.com/blog/what-is-serverless/), or [here](https://en.wikipedia.org/wiki/Serverless_computing), or [here](https://devops.stackexchange.com/questions/61/what-is-serverless), or [here](https://hackernoon.com/a-crash-course-on-serverless-with-node-js-632b37d58b44), or [here](https://hackernoon.com/how-serverless-computing-will-change-the-world-in-2018-7818fc06b447). If you are familiar with Serverless and you don’t like it, you may still enjoy this article and the benefits of Cloud9 and CodeStar. Just make sure you mentally replace “_FaaS_” with “_Container_”, and “_SAM_” with “_CloudFormation_” :) ![](https://pbs.twimg.com/media/DTM-WGcX0AItXxr.jpg:large) ### What is AWS Cloud9? AWS Cloud 9 is a “_cloud IDE_” for writing, running, and debugging code. I’d start saying that most IDEs are fantastic tools to boost your productivity and the quality of your code if you can use them and invested a few months/years in learning them properly. That being said, some IDEs offer more advanced and useful features than others (_please don’t take it personally, unless you code in Word or Notepad_). ![](https://hackernoon.com/hn-images/1*KYVscEHbDKXtY3GJz6l0eg.jpeg) [AWS acquired Cloud9](https://techcrunch.com/2016/07/14/amazons-aws-buys-cloud9-to-add-more-development-tools-to-its-web-services-stack/) in July 2016, which has now been rebranded as AWS Cloud9. Even though it looked brilliant on Werner’s browser, my first impression during the keynote was something along the lines of “**_Why should I pay to write code?_**”, immediately followed by “**_Does that mean I cannot code when I’m offline?_**”. Like most software engineers, I’ve used many IDEs in the last ten years, for free, and I’m used to coding a lot while I’m traveling. Apparently, I was not alone, and many developers asked the same questions during my presentation. So let me briefly recap my arguments. Regarding the cost, I believe it’s pretty much negligible for most organizations that already use AWS heavily (less than _$2 per month_ for a _t2.micro_ environment, 8 hours a day, 20 days a month). And without considering AWS Free Tier and automatic cost-saving settings (hibernation after 30min). The “_no coding offline_” drawback is much harder to defend, but let me try. Unless you are going for a hardcore debugging session over a well-established project, can you really code for more than 30min when you are offline? Can you `git clone` or `git pull` anything useful? Can you `npm install` or `pip install` the modules you need? Can you mock or ignore all the third-party services and APIs your application consumes? Can you avoid googling around your favorite framework’s documentation? Sure, you could prepare for a 12h flight and download/install everything you need in advance. But how often does that happen? Simply put, I’ve seen the best developers and engineers give up and take a break when the network goes down. On the other hand, AWS Cloud9 offers you a better alternative for when your machine gives up :) I could throw my dev machine out of the window right now, switch to my colleague’s notebook, login into my AWS Account and keep working on the very same Cloud9 session (which is saved and stored server-side). That means you could as well use a much cheaper machine such as a Chromebook or a tablet (or a phone?). Well, you could be 100% operational using the random machine of an internet cafe, or your grandmother’s computer :) Of course, there are always exceptions, and I’ll make sure I’m ready to use my local IDE when Cloud9 is not an option. In the meantime, I hope AWS will work on some sort of client-side offline support (and maybe turn Cloud9 into a progressive web app?). #### … and why does it matter for developers? I think AWS Cloud9 solves a bunch of problems for the many organizations currently trying to set up elaborate stacks of tools on every developer’s machine, especially if the team is heterogeneous and/or distributed. Let’s recap some of its features: * It’s a **full-fledged IDE** (based on the [Ace open-source editor](https://github.com/ajaxorg/ace/)) * It comes with an **integrated web terminal**. It is a real ssh session directly your browser, without the need of managing and storing ssh keys or IAM credentials on your local machine. * This web terminal can run on a new EC2 instance managed by AWS Cloud9 (EC2 env.), or you can bring your own instance (SSH env.) * EC2 environments offer quite a handy **cost-saving functionality**, which means you can optionally configure them to **hibernate** after 30 minutes of inactivity (or more) * EC2 environments are based on [this Amazon Machine Image](http://docs.aws.amazon.com/cloud9/latest/user-guide/ami-contents.html), which includes **at least 90% of the dev tooling you need**. For example, the _AWS CLI, sam-local, git, gcc, c++, Docker, node.js, npm, nvm, CoffeeScript, Python, virtualenv, pip, pylint, boto3, PHP, MySQL, Apache, Ruby, Rails, Go, Java_, etc. In case anything is missing, you can always install it :) * AWS Cloud9 comes with the **live debugging capabilities** you’d expect from a modern IDE (currently only for Node.js, though) * It enables **collaborative coding and debugging sessions**. You can share a Cloud9 environment with other _IAM users_ and invite them with _read-only_ or _read-write_ permissions. * It comes with built-in support for **AWS Lambda**. This greatly simplifies the process of creating new Lambda Functions, updating and testing their code locally, deploying new versions, etc. * Plus, **AWS SAM** is part of the team too. [AWS SAM](https://github.com/awslabs/serverless-application-model) — or Serverless Application Model — is an open specification that allows you to define serverless applications and related resources with a simplified CloudFormation syntax. Cloud9 natively integrates some of the functionalities offered by [SAM Local](https://github.com/awslabs/aws-sam-local), an open-source CLI tool written in Go by AWS to simplify local development and testing of Serverless applications. For example, you can invoke Functions locally and emulate API Gateway endpoints too. I took the following screenshot during a live-debugging session of a very simple Lambda Function (**_note_**: I also spent 3 minutes customising theme & layout, according to my taste and needs). [![](https://hackernoon.com/hn-images/1*3YYIOcwPXfYySnv6X2Tpzg.png)](https://aws.amazon.com/cloud9/) AWS Cloud9 in action during a live-debugging session with Node.js #### AWS Cloud9 Limitations and my personal “wishes” I do have a few wishes for AWS Cloud9, and I’ve shared a few of them on Twitter (tweets below). Let me discuss a few of them: * The “_numeric limitations_” are pretty reasonable, in my opinion. You can create up to 20 environments per user (max 10 open concurrently), 100 per account, and you can invite up to 8 members in each environment. These are reasonable numbers since **you can’t share environment across AWS accounts** (yet?) * **Live debugging** is terrific, but **only available for Node.js**. I’m looking forward to built-in support for Python (and all the others, of course). * The same holds for some **runtime customizations** currently not supported. For example, I couldn’t find a way to change the **default linting and code completion** behavior for Python (only _pylint_ arguments are supported). Without custom linting, Python3 developers cannot benefit from [Python’s type hints capabilities](https://twitter.com/alex_casalboni/status/949685627456573440) ([_mypy_](http://mypy-lang.org/) is required). * As discussed above, there is **no support for “offline” development**. Working offline may become a critical need in some scenarios/teams, although you can always download the whole environment to your local machine with one click (_File > Download Project_) and keep working locally. * I think Cloud9 could be much **better integrated with the AWS Console**, especially with services such as Lambda and API Gateway. For example, there is no easy way to jump to the Lambda Console of a given Function (or API Gateway). * The **built-in Lambda integration** is excellent, but it’s still **not as productive as the native Lambda Console**. For example, you can test a Function locally, but you can’t quickly pick the test event from a list of templates. For now, you can workaround this by running [sam local generate-event](https://github.com/awslabs/aws-sam-local#generate-sample-event-source-payloads) in the terminal. ![](https://pbs.twimg.com/profile_images/912702398967271425/z09NNbJ0_400x400.jpg) ![](https://pbs.twimg.com/profile_images/912702398967271425/z09NNbJ0_400x400.jpg) ![](https://pbs.twimg.com/profile_images/912702398967271425/z09NNbJ0_400x400.jpg) ![](https://pbs.twimg.com/profile_images/912702398967271425/z09NNbJ0_400x400.jpg) ### What is AWS CodeStar? AWS CodeStar (aka **Code-\***) is a sort of “_catch-all service_” for the ever-expanding suite of tools for developers. It is a free service that lets you manage and link together services such as _CodeCommit_, _CodeBuild_, _CodePipeline_, _CodeDeploy_, _Lambda_, _EC2_, _Elastic Beanstalk_, _CloudFormation_, _Cloud9_, etc. One of my 2018 new year resolutions is to use more memes in my decks (until someone decides to stop me, for some reason), so here’s how I presented some of the pain points that CodeStar can solve. [![](https://hackernoon.com/hn-images/1*8wcrKrzgaEJwNKam8LCYCw.jpeg)](https://en.wikipedia.org/wiki/CI/CD) Too many projects tend to **postpone** [**CI/CD**](https://en.wikipedia.org/wiki/CI/CD) **until it’s way too late** and their **productivity level is too low**, just because it sounds hard. But it doesn’t have to be that hard, right? ![](https://hackernoon.com/hn-images/1*dq5xLtg9blGXdxnvLveMnQ.png) Nobody likes **building and maintaining real-time dashboards**. But they are vital from day one, for a manager to assess the status of a project, as well as for a developer to monitor how the system is behaving. [![](https://hackernoon.com/hn-images/1*GwoUrficuzqGSxXD-gztEQ.jpeg)](https://git-scm.com/docs/git-push) This is what most developers want to do all day. **Just write some code and push to master**. How can you provide such a simple and frictionless experience without impacting quality and ownership? [![](https://hackernoon.com/hn-images/1*5-f6rY2CN4DDBBg_c4Jn8Q.jpeg)](https://www.atlassian.com/software/jira) Issue tracking can be a very **frustrating experience**, especially if not well integrated with source control, access control, team management, monitoring, etc. **_Data-driven parenthesis_**: I can statistically confirm that the JIRA meme generated 42% more laughs than all others combined. #### … and why does it matter for organizations? CodeStar may not be the best fit for every project/organization, especially the most experienced and advanced ones, but it definitely provides some very good defaults to get started with. It’s worth noting that CodeStar is 100% free, and you only pay for the resources it will spin up on your behalf. Let’s recap its features: * CodeStar offers the concept of “**_project templates_**”. Each template represents a complete stack and includes a sample app, with a given backend, programming language, and framework. * It supports three compute layers: **_EC2_**, **_Elastic Beanstalk_**, and **_Lambda_**. * It supports six programming languages: **_C#_**, **_Java_**, **_Node.js_**, **_Python_**, **_PHP_**, and **_Ruby_** (plus plain HTML apps). * It supports plenty of frameworks: **_Express_**, **_Spring_**, **_Django_**, **_Flask_**, **_ASP.NET Core_**, **_Laravel_**, etc. **_Note_**: AWS Lambda projects only support Express (Node.js) and Spring (Java), plus a few sample projects for **_Alexa Skills_**. * Depending on the project template of your choice, CodeStar will spin up a **_CI/CD pipeline_** by linking together CodePipeline, CodeBuild, and CodeDeploy. * Every CodeStar project starts with **source control** (**_git_**), either on **_AWS CodeCommit or GitHub_**. CodeStar will create the git repository for you (via OAuth in case of GitHub) and take care of triggers/hooks for CI/CD. * You can optionally **configure your own coding tools** to work with CodeStar. Currently, only Cloud9, Eclipse, and VSCode are natively supported (plus the regular AWS CLI). You’ll read more on the Cloud9 integration later in this article. * Even though many projects start as a one-person effort over the weekend, good projects tend to evolve and **onboard more people quickly**. CodeStar allows you to invite IAM users to your project with one of these roles: **Owner** (“God Mode”), **Contributor** (everything but team management), or **Viewer** (read-only dashboard access). More technical info [here](https://docs.aws.amazon.com/codestar/latest/userguide/access-permissions.html). * As mentioned above, **issue tracking** is probably the most frustrating part of many projects, as it brings context switching, visibility, and misunderstanding issues into the equation. CodeStar allows you to **integrate JIRA or GitHub Issues** into your project dashboard, which might help in reducing the context switch and centralizing all the information. * CodeStar provides a **customisable app dashboard** (screenshot below), which includes a **_project wiki section_**, a **_CloudWatch Metrics_** section, your project’s git history, API endpoints, open issues, Cloud9 environments, etc. You can **drag-and-drop sections around** as you wish and use this dashboard to check the overall status of your project quickly and effectively. [![](https://hackernoon.com/hn-images/1*2b2S6gIk-vnnkaMxyfsrAw.png)](https://aws.amazon.com/codestar/) A typical CI/CD pipeline managed by CodeStar (AWS Lambda project) [![](https://hackernoon.com/hn-images/1*7HL7wuxIcKApvRJBzuQsLw.png)](https://docs.aws.amazon.com/codestar/latest/userguide/how-to-customize.html) A brand-new CodeStar dashboard (AWS Lambda project) [![](https://hackernoon.com/hn-images/1*jMJTU4ho6PbaZlSbp3vHQg.jpeg)](https://docs.aws.amazon.com/codestar/latest/userguide/how-to-manage-team-permissions.html) CodeStar’s team management and user roles #### AWS CodeStar Limitations and a few “_gotchas_” CodeStar can look like magic if you’ve never played with CodePipeline and CodeBuild, but unfortunately it’s not perfect yet. I’ve shared a few “wishes” on Twitter too (tweets below), and here’s a quick recap of what I’ve found. * You can create up to **_333 projects per account_** (it looks like the number came out of some kind of thoughtful calculation, right?), but you can only have **_10 projects per user_** and **100 users per project**. As for Cloud9, these seem like reasonable numbers, especially since CodeStar **does not support federated users** or temporary access credentials, which means you’ll need to create a lot of users if a few developers start collaborating across AWS Accounts. * Although the default user roles seem to cover most use cases, you are stuck with the owner/contributor/viewer permissions as **you can’t create custom roles**. For example, you may want to have a “_project manager_” role with view-only and team-management permissions. * Remember that **CodeStar permissions are not related to Cloud9 permissions**. If you want other owners or contributors to join your Cloud9 environment, you’ll have to invite them explicitly. Which makes sense, but I believe it could be improved/automated. * The most critical limitation I’ve found is that **there is no way to customise project templates**. As I’ve mentioned already, CodeStar provides pretty good defaults for a lot of things, but you’ll probably need to change a few details here and there. For example, you may want to add a testing step to CodePipeline, add custom permissions to the default IAM role, edit the default build file, change the default API Gateway stage, etc. And once you’ve done that, **there is no way to save your edits into a new custom project template** so that your team will start from there. Which means you’ll have to apply your customizations to each new project. And you only have two options: 1) _applying changes manually_, or 2) _automatically applying Change Sets to your CloudFormation Stacks_ (please note that every project will create one or more CloudFormation Stack and that the original templates are not versioned anywhere, so good luck with that!). Plus, see the next bullet point :) * Some of the CodeStar functionalities are based on **mysterious and undocumented CloudFormation magic** such as the`AWS::CodeStar::SyncResources` resource and the `AWS::CodeStar` transform. They both sound pretty powerful, but there is no easy way to know what they are used for (or how we could use them). My current understanding is that `AWS::CodeStar::SyncResources` will wait for all the other resources to be deployed (i.e. _DependsOn_) and then make sure everything is okay (e.g. IAM permissions, project id, etc.). `AWS::CodeStar` seems to simply to inject `SyncResources` into the processed template so that we don’t have to. ![](https://pbs.twimg.com/profile_images/912702398967271425/z09NNbJ0_400x400.jpg) ### What about AWS CodeStar + AWS Cloud9? Cloud9 and CodeStar are pretty cool services on their own, and I was excited to see how they’ve been integrated. Or, better, how Cloud9 has been integrated into CodeStar. You can associate AWS Cloud9 Environments to your CodeStar project natively. In case multiple developers are working on the same project, you can create and assign a Cloud9 Environment to each developer (eventually, they’ll collaborate and invite each other, if needed). Once you open Cloud9, you’ll find your IAM credentials integrated with git ([which does require some work](https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-gc.html)) and your CodeCommit repository already cloned for you. Unfortunately, this **magic doesn’t happen if you choose GitHub** (for now?). As a couple of friends and colleagues pointed out, it’s not such a critical or technical complex integration, in the sense that you could have taken care of it yourself (as you’d do on your local machine). But I think it’s a great way to **streamline the development experience** and **reduce the margin for error**, especially when you work on **_multiple projects_** and **_multiple accounts_**. For example, most developers make great use of **_AWS profiles_** when working on their local machine, and some of them also manage to remember which profile can do what, in which account, etc. With CodeStar+Cloud9 **you won’t care anymore about profiles or local credentials** since every Cloud9 environment is bound to a specific project and account. Also, since CI/CD is enabled by default, most of the time you will just **_write code, test with sam-local_** and `git push` 💛 Of course, you may also have a generic Cloud9 Environment (i.e. not related to a specific project) and use it with multiple profiles to manage unique resources or prototype new stuff. ### Let me open a parenthesis: AWS SAM I decided to conclude my presentation with a brief parenthesis about AWS SAM, which got a few mentions and therefore deserves some context. **_\*\* Serverless alert \*\*_** [![](https://hackernoon.com/hn-images/1*HuQE7zDSI-PC0nfdiMNA4g.jpeg)](https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md) Meet SAM! _SAM_ stands for **Serverless Application Model**, and it’s an [open specification](https://github.com/awslabs/serverless-application-model) whose goal is to offer a standard way to define serverless applications. Technically speaking, it’s a [CloudFormation Transform](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/transform-section-structure.html) named `AWS::Serverless` that will convert special Serverless resources such as `AWS::Serverless::Function` into standard CloudFormation syntax. You can think of Transforms as a way to **augment the expressiveness of CloudFormation templates** so that you can define complex resources and their relationships in a much more concise way. If you are familiar with other tools such as the Serverless Framework, you’ll notice that the syntax is quite similar (there is even a [plugin to convert your templates to SAM](https://github.com/SAPessi/serverless-sam)). In fact, you can deploy SAM templates with [**AWS SAM Local**](https://github.com/awslabs/aws-sam-local), a CLI tool for local development written in Go and officially released by AWS. You can use AWS SAM Local to **test your Lambda Functions locally** and **emulate API Gateway endpoints** too. The CLI tool is available by default on every Cloud9 EC2 Environment, and the UI already supports some of its functionalities. #### A few SAM examples [![](https://hackernoon.com/hn-images/1*FeGmaUApk8LvcpCzKQRiUA.png)](https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction) A simple serverless Function (additional properties are available for IAM policies, VCP config, DLQ, tracing, etc.) [![](https://hackernoon.com/hn-images/1*_hBUzGDh9eJrFUUMTA1TNw.png)](https://github.com/awslabs/serverless-application-model/blob/master/docs/safe_lambda_deployments.rst#safe-lambda-deployments) The same Function defined above, plus CodeDeploy Traffic Shifting (10% every 10min) [![](https://hackernoon.com/hn-images/1*QMPIugOO1lPs-vqob9Ckiw.png)](https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#event-source-object) The same Function defined above, plus an API Gateway endpoint (nothing else needs to be defined) [![](https://hackernoon.com/hn-images/1*RhJo8YTcXoGeAo7aujOUWw.png)](https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlesssimpletable) A simplified DynamoDB definition (only primary key and throughput) [![](https://hackernoon.com/hn-images/1*iqwAkcRn7d__gwwuAftKZA.png)](https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessapi) An API Gateway defined in swagger format (sam-local will package it to S3 before deploying!) #### My personal “wishes” for AWS SAM I have only one wish for AWS SAM: I would love to see more transparency and documentation related to the `AWS::Serverless` Transform. **Apr 2018 Update:** AWS SAM is now [open-source on GitHub](https://github.com/awslabs/serverless-application-model/tree/develop/samtranslator)! And since I like dreaming, why not allowing **custom CloudFormation Transforms too**? I am almost ready to bet they are implemented with some kind of Lambda hook, and I can’t even start to imagine how many great things the community might be able to develop and share that way. ![](https://pbs.twimg.com/profile_images/912702398967271425/z09NNbJ0_400x400.jpg) I hope you learned something new about AWS Cloud9 and CodeStar (please don’t confuse them and create weird hybrids such as “**CloudStar**” as I did a few times). I would recommend building a simple prototype or a sample project on CodeStar asap. You can get started [here](https://console.aws.amazon.com/codestar)! If you got this far, you probably enjoyed the article or feel like sharing your thoughts. Either way, don’t forget to recommend & share, and please do not hesitate to give feedback & share your ideas =)

Welcome to Cloud-Native Development with AWS Cloud9 & AWS CodeStar ⚡🎉

Too Long; Didn't Read

Companies Mentioned

Coin Mentioned

@alexcasalboni

RELATED STORIES