Hi 👋🏽
This week, we're diving headfirst into the deep end of AWS's pool of innovations with the IAC Generator — a tool that's been causing quite the buzz.
The promise AWS gives us is quite bold:
Generate AWS CloudFormation templates and AWS CDK apps for existing AWS resources in minutes.
But does it really work? Let's find out together.
This will be a hands-on 👷🏽♀️ issue.
For the impatient, TL;DR, in a nutshell, 🥜: Good for a start, but far away from usable in production.
🛠 The Experiment
We took the Terraform-powered web app we built for the CloudWatch Book 📕 for a spin with the IAC Generator, aiming to seamlessly migrate it into a CDK app.
We want to offer the code in CDK & Terraform, and CDK needs yet to be developed. That sounds like the perfect approach! 🏗️
The architecture of that project looks like this:
Not overly complex, but also not simple.
Let's see how that goes. 👀
🪄 The IAC Generator
The IAC Generator is responsible for importing AWS resources into CloudFormation.
The process of generating CloudFormation based on your account looks like this:
- Scan all resources 👀
- Create a template 🏗️
- Add resources to the template ✚
- Import the resources managed by CloudFormation or move to CDK. 🔀
Step 1: Generate CloudFormation 🧙🏽
Go over to the CloudFormation console in your desired region and open the IAC Generator Tab.
Scan your resources
Hit the button Start a new scan. Your account will now be scanned for resources and their relationships. This scan can take a while.
The scan in our account ran for about 5 minutes and found 1345 resources.
Create a template
Next, we create a new template called cw-book-example-app.
Selecting Resources
We can see all resources in the account now.
One problem is that the CloudFormation importer uses the Resource Identifier which is an ID. This is not always obvious. Look at API Gateway for example. The Resource Identifier doesn't give you a clue which API it is exactly.
Whereas, the name does it. Serverless Hero Yan already pointed that out in his blog post about the IAC Generator.
We don't want to copy the whole AWS account into one template, as we have development and production resources in the same account (shame on us).
That is why we need a way to choose resources that should be added to the template.
This is where Tags come in. As you will learn on your AWS journey, tags play a crucial role in multiple services on AWS. We can filter the resources by tags. Let's do that.
All our resources have the following tags:
That should be sufficient enough to distinguish.
We can't search by multiple tags, but since this is a dedicated account only for this one application it is okay to add the filter for:
- Tag key:
Environment - Tag value:
dev
We add all 44 selected resources to our CloudFormation template, including all related resources.
Template is generated
The template is generated 🎉
Step 2: Migrating to CDK 🚀
With our CloudFormation template as our blueprint, we go into CDK territory. We download the template and execute the command:
cdk migrate --stack-name cw-book-example-app --from-path ./cw-book-example-app-1707551700519.yaml --language typescript
The migration is quite fast. But there is a warning about missing resources. After checking the README.md file you can see a few errors on resource options.
From that point, we are not 100% sure what is going wrong but it looks quite right, so we continue.
The CDK Project
Let's check the CDK project. This is the file structure of the new project:
It created a CDK App in bin/cw-book-example-app.ts
It also created one file which includes all stacks in lib/cw-book-example-app-stack.ts.
The process of the migration is done. We haven't deployed it because we knew we wouldn't continue using it for our actual production system.
The Good, The Bad, and The Techy ✨
Pros 🟢
Good to get started
If you have never worked with Infrastructure as Code and built your AWS infrastructure in a Click-Ops way it can be quite intimidating to get started.
With this approach, you get an idea of your resources and how to provision them with CloudFormation and CDK.
Finding resources in your account
With code, it can be easier to understand which resources you have. You can also use the resource manager for that (which is much easier IMO) but this could be a benefit.
Cons 🔴
There are some parts we don't like and think can be improved.
Naming
Naming is hard; we get that. But after migrating we have several constructs with these names:
- lambdaFunction00cwhotfdev...
- apiGatewayRestApi00sq9n...
- codeStarConnections...
It is not possible to work with that.
One Stack
The migration command created one stack. Designing stacks while developing a new application is not easy. If you want to separate stacks you should have thought about that while importing the resources into CloudFormation.
For customers using this feature, this will not be feasible.
One File
The migrate command created one file with 2393 lines of code.
That's too much and very unclear.
L1 Constructs
The beauty in CDK comes with L2 constructs. Not caring about IAM, and having additional APIs like grantRead makes working with CDK so much easier.
The migrate command migrates on the lowest level possible and gives us L1 constructs which are CloudFormation constructs. Just look at my Step Function definition. 🥲
This is probably not something that is easily maintained.
Lambda Code
We rely a lot on Lambda functions. We were already curious about how the IAC generator will get the Lambda execution code. But it simply doesn't. Which makes sense, but also defeats the purpose a bit.
Once you want to deploy you need to define the buckets where the code resides.
For each Lambda function, you need to declare the bucket.
Summary
In summary, migrating your resources from your AWS account to CloudFormation and then to CDK is possible.
What's not possible: using the generated code without making adaptions. At least not for a serious, long-term project.
We think it can be a great starting point. But it can also be quite complex to iterate from that point on.
The AWS Post mentions VPCs a lot. We do understand that understanding all resources that were manually created within that VPC is hard.
It is a great starting point. However, we won't use it for creating the CDK code for the CloudWatch Book. We will create it manually with all best practices in mind.
Since you are still dependent on CloudFormation, designing stacks is not possible with the automated approach.
We do understand that building such a kind of feature can be hard to get right for everybody. We don't see a lot of use cases here right at this moment.
Sprinkling in some LLMs together with natural language generation could improve a lot of things here! Maybe that is for the future. 🔮
Have you used it?
What's your take on the IAC Generator? Have you encountered similar challenges, or did you find a workaround that made the process smoother?
Share your experiences, tips, or questions with us.
Your insights not only enrich our community but also pave the way for more informed, practical explorations in the future.
Until our next adventure, keep experimenting, learning, and sharing within the AWS cosmos. See you in two weeks for another dose of cloud wizardry! 🌈
Best,
Sandro & Tobi
P.S. Curious about another AWS feature? Let us know, and it might be the star of our next hands-on guide! ✌️
| | Tobi & Sandroour goal is to simplify AWS & Cloud Learning for everybody. You don't need expensive certifications to build on AWS! |
