Written by Jim Lamb, Head of Engineering at Airwalk Reply.
Delete everything from an AWS account except your Single Sign-On infrastructure.
We have a large number of AWS accounts within our various AWS Organisations. We also use Single Sign-On (SSO), both via AWS SSO and SAML identity federation, to allow centralised control of user access and permissions.
Historically, we have created a new AWS account whenever a new person or project needed one, but as we grew we were creating so many new accounts it was all getting a bit out of control.
It was a non-zero effort to add new accounts and get them hooked into the SAML identity federation, so we asked why we don’t just re-use the accounts we no longer use? The main answer is because it’s a bit of a faff finding and deleting all the AWS resources that somebody has previously created. There’s no “factory reset” button for an AWS account.
It’s easy enough to find the resources that are costing you money; we’re pretty good at removing anything that we no longer need which is adding to the monthly bill. The problem is the cost-free resources that are hiding and might cause us problems later on if we try to re-use the account, particularly if we’re installing a newer version of the same app. Resources with duplicate names, IAM policies with slightly different permissions, configurations of singleton resources, etc.
Nuke the site from orbit. It’s the only way to be sure.
In order to make sure we’ve deleted everything, we use aws-nuke. aws-nuke is designed to remove all resources from an AWS account. There are plenty of articles around on how to use this tool, so I won’t repeat them here except to say we are playing with a flamethrower here, so make sure you’ve at least read the manual. Use at your own risk.
It was all fun and games using aws-nuke the first time until I realised I’d destroyed our ability to login. The account was beautifully empty, but we were locked out of it. Sad face.
I will now describe how you can use aws-nuke to remove everything except your base infrastructure — those things you want to exist in every account and not have to recreate once you’ve performed the aws-nuke, specifically SSO.
If you’ve not used aws-nuke before, be aware that it produces quite a bit of output, a lot of it due to differences between the services available in different AWS regions, but also because some services are no longer available or not enabled for your account.
My Config
I’m going to explain the aws-nuke config I created to help me.
TL;DR You can find the whole config file at the bottom of this article.
Regions
There is currently no shorthand way to tell aws-nuke to cover all regions (though it has been suggested) so you have to list them all out. You could find this list with a command like:
You also need to give aws-nuke the special region name “global” to cover items such as IAM. This gives us the first part of the config:
regions:
- "global" # This is for all global resource types e.g. IAM
- "us-east-2"
- "us-east-1"
- "us-west-1"
- "us-west-2"
- "af-south-1"
- "ap-east-1"
- "ap-south-1"
- "ap-northeast-3"
- "ap-northeast-2"
- "ap-southeast-1"
- "ap-southeast-2"
- "ap-northeast-1"
- "ca-central-1"
- "cn-north-1"
- "cn-northwest-1"
- "eu-central-1"
- "eu-west-1"
- "eu-west-2"
- "eu-south-1"
- "eu-west-3"
- "eu-north-1"
- "me-south-1"
- "sa-east-1"
Account blacklist
Next comes the account blacklist. It’s optional and lets you specify some accounts never to touch. I’d advise you list your production accounts here.
account-blocklist:
# Accounts to never touch, e.g. production
- 666666666666
- 555555555555
Then comes the useful stuff. In order to make the config more manageable, I’ve taken advantage of the “presets” notion that exists in aws-nuke. This allows us to group resource definitions together, then apply those groups to different accounts.
Preset 1: AWS SSO
My first preset is for AWS SSO, allowing me to specify filters on those AWS resources created by AWS SSO when provisioning accounts. By using these filters, any resource matching these expressions will never be touched when aws-nuke is run. We ensure that the IAM SAML provider, the IAM roles and IAM policy attachments created during AWS SSO provisioning are left intact.
My next preset is for our SAML identity provider. Your requirements may vary, but we originally used the method of Azure AD SAML federation described in this AWS blog post. You likely won’t need this, but I’m including it here in case somebody out there finds it useful. If you don’t want it, just don’t include this preset in your configuration.
The preset filters out the IAM SAML provider, the IAM roles, IAM policies, IAM policy attachments and the CloudFormation stacks that created them.
Nothing to do with SSO, but you might use an Organization-wide CloudTrail that you want to leave intact and this is achieved simply as below, specifying your trail name.
Having created our presets, we need to associate them with accounts. This is as simple as this, where 111111111111 and 222222222222 are our account IDs:
The --no-dry-run option still requires you to type the account name in twice before it will delete anything, so I don’t consider it dangerous leaving it in here.
Some problems/workarounds
Break it down
In order to not completely drown in output, I find it easier to run aws-nuke a number of times, with slightly different config, in this order:
Specifying only regions where you know you have resources, potentially further breaking it down by adding one known region at a time.
Specifying all regions except “global” to cover anything else you didn’t know about. Perhaps there is something created in a region you don’t normally use.
Specifying all regions including “global”, to cover IAM, etc.
In reality, this means starting with a config file which includes the list of all regions as above, then commenting most of them out and uncommenting some of them after each individual run of aws-nuke until they’re all uncommented. The final run will include every region and you can be confident everything has been deleted.
Bombing out
Sometimes aws-nuke does bail out. Even when it doesn’t, with a large set of resources I get a lot of “too many open files” errors. I tend not to worry and either split up the regions (see Break it down, above) or keep running the same aws-nuke command a few times and eventually it deletes everything.
S3 sprawl
S3 buckets with very large numbers of objects can also be a problem. Again, one option is to just keep re-running, though sometimes it’s just easier to go and delete the bucket and all its objects via the AWS Console.
To be sure
Even if it exits successfully, with so much output, often containing HTTP 4xx errors, I like to re-run until I see there is nothing left that aws-nuke considers worth destroying:
No resource to delete.
How to create filters
In order to come up with working filters, including those I list in this article, I found it best to be very specific about the types of resource I was looking for, then add resource patterns to filters in small batches. This made it easier to find what I needed in the huge amount of output.
For example, to find some specific IAM Roles in an account, I would use resource-types -> targets in my configuration file so that aws-nuke would only consider IAM Roles.
resource-types:
targets:
- IAMRole
Then I would run aws-nuke and look in the output for those roles I wanted to preserve:
Running aws-nuke again with the filter included in my configuration file confirmed that these resources have now changed from “would remove” to “filtered by config”.
IAM Roles filtered out by our configuration file
If you’re looking for a quick regex for an AWS account number for those resources that contain an account number as part of the ID, here is an example:
If you’ve got this far, thank you for reading! Nothing I’ve written here can’t be found elsewhere or figured out from trial and error, but if you’ve got exactly the same requirement to nuke most of the things, perhaps you’ve saved a few minutes of your life.
