On the Reverse IaC

5 min readMar 9, 2021

Introduction

I want to give some thoughts on the topic of creating code from the infrastructure — let’s call it Reverse IaC, as I wasn’t able to find any term that is used to describe this approach. This thread hopes to address the following areas:

Areas for optimization
Risks of usage
Popular use cases

Examples in this article use terraformer, a tool to build Terraform files from existing infrastructure. At the time of writing this article, the tool supports almost 30 cloud providers and other products with in total over 600 resource types.

A word of warning

This article tries to address the approach that is nowadays being used at many companies, from small product companies to bigger ones, from small consulting companies to big multi-international ones.

It is a fact the approach helps saving money companies by automation rather than manual creation of code from an infrastructure. When talking with people that use terraformer, I hear about days, weeks or even months saved with it. I will try to even go further and say manual processes are prone to error, so should not be done by hand. A specialized set of tools should be used instead.

The author is a strong believer of IaC, been applying the IaC practices for many years, starting with Chef and later used CloudFormation, Ansible, Terraform, Terragrunt, CDK and similar for creating successful solutions, and always been believing in automation as the only way to move forward.

Discovery and review of the infrastructure

Let’s say you have a new account of a cloud provider to manage. Modern cloud providers can consist of 100+ services, which means it’s likely possible to miss resources configured in a service that can be critical for a system to run.

Sample visualization of cloud resources to show configuration easily can get complicated — here is an image of AWS IAM, undoubtedly one of most complicated AWS services (ref)

An engineer can be tempted to invoke a command like one below:

terraformer import aws --resources="*"

To have a good overview of how the system works, you can import all of the resources and review them in a single file. In that way, you will have high chances of understanding how all of the resources in an account are configured.

Moving infrastructure provisioning to code

Almost all of the cloud providers have a UI that allows manual provisioning of resources. The UI allows for an easy overview of resources but allows modification of the resources as well. That ease of modification allows individuals to learn to use cloud or organizations that are yet to grow in maturity of management operate their resources.

It may happen a system will be provisioned manually. It results in certain disadvantages, which most notably are: higher cost due to human capital to manage it or inconsistencies due to human error. It also reduces agility and compliance with standards and good practices.

It may also happen resources are provisioned manually through a CLI. In either case, once resources are provisioned, there should always be a way to provide a proper level of management or to improve it. Usually, that would require manual mapping of all resources to a code e.g. to Terraform.

While contributing to the terraformer project, I saw a lot of people use the tool to quickly create code of the infrastructure that once was not provisioned with one code. Resources could be provisioned manually because there was no time to configure it through code or there was no way to configure a resource through code. It is sometimes a case in Terraform as support for specific resource type can be delivered after release from a cloud provider (example). It may also be a case when a provider was added at later stage (example).

Automation of the migration of the provisioning can save up time spent to perform such an operation and depending on the size of an infrastructure, those can be hours, days or even months.

There are of course risks there — as with any automation, the imperfections in the tool can omit some of the resources or it may be impossible for a tool to import all of the resources to a local machine (usually all resources must be dumped to a memory, which is sometimes a problem in accounts that have thousands of resources).

The results of the automated import should be reviewed manually and the tooling should be adjusted to cover more resources.

Continuous Reverse IaC

Let’s now assume you use the import tool to perform an import not just as a one-time action but run it on scheduled basis e.g. once a day. With that, you can centralize configuration of the infrastructure that may be provisioned manually or automatically through many different tools (after all, bigger organizations can write their own provisioning tool or have many that aren’t standardized).

Let’s look at the below simple example of setting up such a scheduled import:

terraformer import aws --resources="*"
git add generated/*
git commit -m "Changes in `date +"%m-%d-%y"`"
git push origin main

The script above can be run on scheduled basis to run a daily import, add and commit the changes and push it to a remote git repository e.g. GitHub. In that way, it’s easy to monitor and review git diffs done on an infrastructure regardless whether they were done manually or though any automated provisioning tools.

When setting up DR

Disaster Recovery plans are often neglected when setting up cloud resources and usually done after a system is created. Companies with lower organizational maturity see low value in maintaining procedures in case of a disaster. In the first case, infrastructure is often provisioned manually as well.

To address such cases and reduce immanent risk of a destructive event that may put a company to being unable to recover fast enough, so that will face financial losses, reverse IaC can be used as a first step of onboarding to applying IaC practices and to have code from which an infrastructure can be recreated.

The plan can be quickly created and engineers can start performing game days, when a simulations of disastrous failures of a system are tested to check how systems, processes, and team response to the events.

What tools to use?

Here’s a list of more notable Reverse IaC tools:

Each tool has it pros and cons — terrafomer supports well most of the cloud providers and wide range or resources although doesn’t have 100% resource type coverage. All commands are issues though a CLI interface.

Former2 on the other hand, provides very well support of the AWS and CloudFormation resource types. It comes with a UI in a form of a web browser plugin.