An account for every color of theĀ rainbow Having seperate AWS accounts is the latests Dev Ops craze to grace our toolkit (not including kubernetes of course) and with good reason! Seperate AWS accounts to manage environments, or even products is a great idea. If done right itās something else that will help you sleep at night. The benefits are: Separation of concerns across AWS accounts means completely different set of IAM Users, Roles & API Keys. Now your Production account will finally, not be exposed to every single one of your Developers. Security! Developers will only be scared of touching things in the Production account. Meaning less bloat in your other accounts, whilst being able to tear other environments apart. This is underrated, developers will be able to play & build more! Development Speed, Developer ease! Make Dev Ops For Everyone! . Youāll be able to stop paying 2% of your AWS bill, for dissecting your bills. You can use inbuilt , to definitively understand the financial aspect of every environment. Billing stax consolidated billing Iām going to show you how Avoss tackled their multi account setup, with a large focus on . deployments, security & developer access But be warned! The silver bullet doesnāt exist and multi accounts comes with its own trade offs! Learning is going to take time, migrating to multi accounts will be a headache & managing everything is going to take careful consideration. IAM Running a multi AWS account setup would be very hard & tedious, if you are not using some sort of Infrastructure as Code solution. In this blog post & at Avoss we are using . If you want to start investigating Infrastructure as Code, I covered a comparison in my last blog post . Side note* CloudFormation CloudFormation Vs Terraform IAM & Deployments Who has access to what? How does everything get deployed? How can we reduce access! ARHH DONāT COMMIT THE ACCESS KEYS (yes we have all done it)! Itās overwhelming. Your first issue is going to be IAM. The accounts are: Goals for our Multi environment There should be for your Production Account. no loose API keys Developers should still be able to have full to production, but changes should take place through your Continuous Integration. read access Or at the very least only two API keys, if your continuous integration is running outside of AWS infrastructure. We can achieve both. Lets getĀ started I know this is going to make you cringe, but first we are going to have to create a User or Role manually. Unless there is something I donāt know about there has to be some manual User or Role creation, to create that first initial User or Role that will kick of your deployments. The only other scenario is using your root credentials, and that is definitely a no go ā ļø. This will have to be done over each environment. The manually created User or Role, is going to be used to create the rest of the IAM Roles & Permissions for all of your deployments in each environment. This gives us a few advantages, all of your IAM roles & permissions, will be easily visible & subject to code review. And coupled with continuous integration, the roles & permissions will roll across all off your accounts nicely. But , giving one manually created User or Role permission to execute all the permissions & roles for your whole account! Wow that is scary, but we can take the appropriate steps to make sure these set of permissions are locked down as tight as possible. Secondly if you are using a continuous integration solution inside of AWS, you can have an the deployment Role & have the zero API keys I promised for your Production account. Making your deployments incredibly secure. this may sound scary EC2 instance assume Below is the manually created User or Roleās attached policy. You will notice that It , that you specifically name can only execute a specific stack It can that you specify only create a specific User & specific Roles This means that even if this Role or User was compromised that attacker would need to know your , and/or . If you wanted to be extremely hardcore, you could go as far as to salt these names with a bit of random junk. exact stack name roles IAM User Now thats secure! Okay now we have a user, that can create more roles, where are we going? To a place where API keys donāt exist & we have maximum security. Nirvana. This scenario repeats over every AWS account you wish to use for deployments At the heart of this diagram is the which you can checkout . This is where your rest of your IAM roles & permissions come from. ${env}-roles-stack.yml here This creates a specific deployment User or Role & specific roles, that can only execute against specific stacks. This is very similar to our manually created User or Role, except each future deployment role also specifies what exactly can assume it, whether itās CloudFormation or the Deployment User or Role. This works incredibly well, when your infrastructure is decomposed. ${env}-roles-stack.yaml You will notice the relationship between the Role & the User is two ways. The Role is explicitly told it can only be assumed by the User. Whilst we tell the User it is allowed to assume the role & also allowed to execute the specific CloudFormation stack, that requires the role. The alternative to assuming the role from the User, is to pass the role to the CloudFormation stack & specifying on the Role only CloudFormation is allowed to assume that role. This means that if an attacker compromises our deployment CI at this point they again they would need the with & whether that role was being . exact name of a Role, the exact stack name passed to a certain service or assumed Assuming the deployment role, when passing the role isnāt available Passing roles to CloudFormation via command line when available This is great & a very secure deployment, but what happened to the multiple AWS accounts! The deployment to Multiple AWS Accounts or Environments The deployment of each one of these pipelines is replicated over your AWS environments. For each AWS account their is a set of deployment Roles & a shared CloudFormation template. Your continuous integration should handle the input parameters of deployments like account details, environment name, region etc. And there is no reason why this diagram canāt involve as many accounts as you wish! This is easy to scale, as per each environment there is only one deployment user. of this deployment. If you are passing roles to CloudFormation stacks to be assumed when executed, you MUST make sure that the roles exist for any future modifications of the said stacks. Including deletion, if it does not exist CloudFormation doesnāt have a role to assume to execute the stack! Another caveat Tell me again why all this effort is worthĀ it? Firstly your first line of defence must be compromised whether that is the manually created User/Role, or the deployment User/Role. Then the attacker, must figure out the permissions are CloudFormation related and can only execute for certain stacks, against certain roles. And if they happen to know the stack name, & role the deployment role should be locked down so the blast radius only affects that piece of infrastructure. Itās very secure! Now if you used only roles during this process, you will have the deployment process. The only attack could from someone taking over your compromised EC2 instance that inherits the role. This is unlikely, especially if you are taking advantage of Virtual Private Networks and protecting your instance! literally zero API keys Can I access production though? You can access it like normal through the ! And you guessed it, you grant access by in your production account from your development account. Yes! web console & command line [switching roles](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_permissions-to-switch.html) In your production account you create a role with all the permissions your heart so desires (but keep it strict! this is production remember!). Then you must edit the to allow your Development account to assume the role. In your Development account you create and attach a policy to your desired Users or Groups. This policy will only have the function of being able to assume the role, in your production account. Trust Relationship Then you can actually fill this handy link out, that will prompt to a switch role screen . After you have switched roles, you will be able to browse the UI like normal, granted you have the permissions. https://signin.aws.amazon.com/switchrole?roleName=<role_name>&account=<target_account> Console UI after switching roles This is seamless & itās easy to switch back and forth. Except for itās easier, you didnāt even have to type another password in! Better yet, you can use the CLI to and use the command line like normal too! assume-role Consolidated Billing Last but not least, for those who havenāt seen it. If you set up your accounts as an organisation, you get free access to . Consolidated billing offers a per account price split, which gives you a clear indication of your financial AWS health! Not only that but the cost explorer, also understands each one of your linked accounts & you can apply this to your filters, rather than just having to rely on tags. consolidated billing An example of what consolidated billing looksĀ like Thanks for reading! If you got this far, be sure to leave a clap!