If you are familiar with Serverless and you don’t like it, you may still enjoy this article and the benefits of Cloud9 and CodeStar. Just make sure you mentally replace “FaaS” with “Container”, and “SAM” with “CloudFormation” :)
AWS Cloud 9 is a “cloud IDE” for writing, running, and debugging code.
I’d start saying that most IDEs are fantastic tools to boost your productivity and the quality of your code if you can use them and invested a few months/years in learning them properly.
That being said, some IDEs offer more advanced and useful features than others (please don’t take it personally, unless you code in Word or Notepad).
AWS acquired Cloud9 in July 2016, which has now been rebranded as AWS Cloud9. Even though it looked brilliant on Werner’s browser, my first impression during the keynote was something along the lines of “Why should I pay to write code?”, immediately followed by “Does that mean I cannot code when I’m offline?”. Like most software engineers, I’ve used many IDEs in the last ten years, for free, and I’m used to coding a lot while I’m traveling.
Apparently, I was not alone, and many developers asked the same questions during my presentation. So let me briefly recap my arguments.
Regarding the cost, I believe it’s pretty much negligible for most organizations that already use AWS heavily (less than $2 per month for a t2.micro environment, 8 hours a day, 20 days a month). And without considering AWS Free Tier and automatic cost-saving settings (hibernation after 30min).
The “no coding offline” drawback is much harder to defend, but let me try.
Unless you are going for a hardcore debugging session over a well-established project, can you really code for more than 30min when you are offline? Can you
git clone or
git pull anything useful? Can you
npm install or
pip install the modules you need? Can you mock or ignore all the third-party services and APIs your application consumes? Can you avoid googling around your favorite framework’s documentation?
Sure, you could prepare for a 12h flight and download/install everything you need in advance. But how often does that happen? Simply put, I’ve seen the best developers and engineers give up and take a break when the network goes down.
On the other hand, AWS Cloud9 offers you a better alternative for when your machine gives up :) I could throw my dev machine out of the window right now, switch to my colleague’s notebook, login into my AWS Account and keep working on the very same Cloud9 session (which is saved and stored server-side). That means you could as well use a much cheaper machine such as a Chromebook or a tablet (or a phone?). Well, you could be 100% operational using the random machine of an internet cafe, or your grandmother’s computer :)
Of course, there are always exceptions, and I’ll make sure I’m ready to use my local IDE when Cloud9 is not an option. In the meantime, I hope AWS will work on some sort of client-side offline support (and maybe turn Cloud9 into a progressive web app?).
I think AWS Cloud9 solves a bunch of problems for the many organizations currently trying to set up elaborate stacks of tools on every developer’s machine, especially if the team is heterogeneous and/or distributed.
Let’s recap some of its features:
It comes with an integrated web terminal.It is a real ssh session directly your browser, without the need of managing and storing ssh keys or IAM credentials on your local machine.
EC2 environments are based on this Amazon Machine Image, which includes at least 90% of the dev tooling you need.For example, the AWS CLI, sam-local, git, gcc, c++, Docker, node.js, npm, nvm, CoffeeScript, Python, virtualenv, pip, pylint, boto3, PHP, MySQL, Apache, Ruby, Rails, Go, Java, etc.In case anything is missing, you can always install it :)
It enables collaborative coding and debugging sessions.You can share a Cloud9 environment with other IAM users and invite them with read-only or read-write permissions.
It comes with built-in support for AWS Lambda.This greatly simplifies the process of creating new Lambda Functions, updating and testing their code locally, deploying new versions, etc.
Plus, AWS SAM is part of the team too.AWS SAM — or Serverless Application Model — is an open specification that allows you to define serverless applications and related resources with a simplified CloudFormation syntax. Cloud9 natively integrates some of the functionalities offered by SAM Local, an open-source CLI tool written in Go by AWS to simplify local development and testing of Serverless applications. For example, you can invoke Functions locally and emulate API Gateway endpoints too.
I took the following screenshot during a live-debugging session of a very simple Lambda Function (note: I also spent 3 minutes customising theme & layout, according to my taste and needs).
AWS Cloud9 in action during a live-debugging session with Node.js
I do have a few wishes for AWS Cloud9, and I’ve shared a few of them on Twitter (tweets below).
Let me discuss a few of them:
The “numeric limitations” are pretty reasonable, in my opinion. You can create up to 20 environments per user (max 10 open concurrently), 100 per account, and you can invite up to 8 members in each environment.These are reasonable numbers since you can’t share environment across AWS accounts (yet?)
As discussed above, there is no support for “offline” development.Working offline may become a critical need in some scenarios/teams, although you can always download the whole environment to your local machine with one click (File > Download Project) and keep working locally.
AWS CodeStar (aka Code-*) is a sort of “catch-all service” for the ever-expanding suite of tools for developers.It is a free service that lets you manage and link together services such as CodeCommit, CodeBuild, CodePipeline, CodeDeploy, Lambda, EC2, Elastic Beanstalk, CloudFormation, Cloud9, etc.
One of my 2018 new year resolutions is to use more memes in my decks (until someone decides to stop me, for some reason), so here’s how I presented some of the pain points that CodeStar can solve.
Too many projects tend to postpone CI/CD until it’s way too late and their productivity level is too low, just because it sounds hard. But it doesn’t have to be that hard, right?
Nobody likes building and maintaining real-time dashboards. But they are vital from day one, for a manager to assess the status of a project, as well as for a developer to monitor how the system is behaving.
This is what most developers want to do all day. Just write some code and push to master. How can you provide such a simple and frictionless experience without impacting quality and ownership?
Issue tracking can be a very frustrating experience, especially if not well integrated with source control, access control, team management, monitoring, etc.
Data-driven parenthesis: I can statistically confirm that the JIRA meme generated 42% more laughs than all others combined.
CodeStar may not be the best fit for every project/organization, especially the most experienced and advanced ones, but it definitely provides some very good defaults to get started with. It’s worth noting that CodeStar is 100% free, and you only pay for the resources it will spin up on your behalf.
Let’s recap its features:
CodeStar offers the concept of “project templates”.Each template represents a complete stack and includes a sample app, with a given backend, programming language, and framework.
It supports plenty of frameworks: Express, Spring, Django, Flask, ASP.NET Core, Laravel, etc.Note: AWS Lambda projects only support Express (Node.js) and Spring (Java), plus a few sample projects for Alexa Skills.
CodeStar provides a customisable app dashboard (screenshot below), which includes a project wiki section, a CloudWatch Metrics section, your project’s git history, API endpoints, open issues, Cloud9 environments, etc.You can drag-and-drop sections around as you wish and use this dashboard to check the overall status of your project quickly and effectively.
A typical CI/CD pipeline managed by CodeStar (AWS Lambda project)
A brand-new CodeStar dashboard (AWS Lambda project)
CodeStar’s team management and user roles
CodeStar can look like magic if you’ve never played with CodePipeline and CodeBuild, but unfortunately it’s not perfect yet. I’ve shared a few “wishes” on Twitter too (tweets below), and here’s a quick recap of what I’ve found.
You can create up to 333 projects per account (it looks like the number came out of some kind of thoughtful calculation, right?), but you can only have 10 projects per user and 100 users per project.As for Cloud9, these seem like reasonable numbers, especially since CodeStar does not support federated users or temporary access credentials, which means you’ll need to create a lot of users if a few developers start collaborating across AWS Accounts.
Some of the CodeStar functionalities are based on mysterious and undocumented CloudFormation magic such as the
AWS::CodeStar::SyncResources resource and the
AWS::CodeStar transform. They both sound pretty powerful, but there is no easy way to know what they are used for (or how we could use them).My current understanding is that
AWS::CodeStar::SyncResources will wait for all the other resources to be deployed (i.e. DependsOn) and then make sure everything is okay (e.g. IAM permissions, project id, etc.).
AWS::CodeStar seems to simply to inject
SyncResources into the processed template so that we don’t have to.
Cloud9 and CodeStar are pretty cool services on their own, and I was excited to see how they’ve been integrated. Or, better, how Cloud9 has been integrated into CodeStar.
You can associate AWS Cloud9 Environments to your CodeStar project natively. In case multiple developers are working on the same project, you can create and assign a Cloud9 Environment to each developer (eventually, they’ll collaborate and invite each other, if needed).
Once you open Cloud9, you’ll find your IAM credentials integrated with git (which does require some work) and your CodeCommit repository already cloned for you.
Unfortunately, this magic doesn’t happen if you choose GitHub (for now?).
As a couple of friends and colleagues pointed out, it’s not such a critical or technical complex integration, in the sense that you could have taken care of it yourself (as you’d do on your local machine). But I think it’s a great way to streamline the development experience and reduce the margin for error, especially when you work on multiple projects and multiple accounts.
For example, most developers make great use of AWS profiles when working on their local machine, and some of them also manage to remember which profile can do what, in which account, etc. With CodeStar+Cloud9 you won’t care anymore about profiles or local credentials since every Cloud9 environment is bound to a specific project and account. Also, since CI/CD is enabled by default, most of the time you will just write code, test with sam-local and
git push 💛
Of course, you may also have a generic Cloud9 Environment (i.e. not related to a specific project) and use it with multiple profiles to manage unique resources or prototype new stuff.
I decided to conclude my presentation with a brief parenthesis about AWS SAM, which got a few mentions and therefore deserves some context.
** Serverless alert **
SAM stands for Serverless Application Model, and it’s an open specification whose goal is to offer a standard way to define serverless applications.
Technically speaking, it’s a CloudFormation Transform named
AWS::Serverless that will convert special Serverless resources such as
AWS::Serverless::Function into standard CloudFormation syntax.
You can think of Transforms as a way to augment the expressiveness of CloudFormation templates so that you can define complex resources and their relationships in a much more concise way.
If you are familiar with other tools such as the Serverless Framework, you’ll notice that the syntax is quite similar (there is even a plugin to convert your templates to SAM).
In fact, you can deploy SAM templates with AWS SAM Local, a CLI tool for local development written in Go and officially released by AWS.
You can use AWS SAM Local to test your Lambda Functions locally and emulate API Gateway endpoints too. The CLI tool is available by default on every Cloud9 EC2 Environment, and the UI already supports some of its functionalities.
A simple serverless Function (additional properties are available for IAM policies, VCP config, DLQ, tracing, etc.)
The same Function defined above, plus CodeDeploy Traffic Shifting (10% every 10min)
The same Function defined above, plus an API Gateway endpoint (nothing else needs to be defined)
A simplified DynamoDB definition (only primary key and throughput)
An API Gateway defined in swagger format (sam-local will package it to S3 before deploying!)
I have only one wish for AWS SAM: I would love to see more transparency and documentation related to the
Apr 2018 Update: AWS SAM is now open-source on GitHub!
And since I like dreaming, why not allowing custom CloudFormation Transforms too? I am almost ready to bet they are implemented with some kind of Lambda hook, and I can’t even start to imagine how many great things the community might be able to develop and share that way.
I hope you learned something new about AWS Cloud9 and CodeStar (please don’t confuse them and create weird hybrids such as “CloudStar” as I did a few times). I would recommend building a simple prototype or a sample project on CodeStar asap. You can get started here!
If you got this far, you probably enjoyed the article or feel like sharing your thoughts. Either way, don’t forget to recommend & share, and please do not hesitate to give feedback & share your ideas =)