Having spent the weekend building infrastructure scripts, I can now say that I like Terraform. My initial foray into the Infrastructure-as-Code (IaC) arena was using Pulumi. This appealed to me because it supported declarations in C#. After not very long, however, I found that Pulumi’s C# support was rather lacking, and put IaC on the shelf.

IaC was resurrected when my boss told me that all infrastructure I built for this project would have to be scripted. One of my team members dove into Terraform, and recently gave me a crash course. The language itself is easy, but I think the development environment itself is very good as well.

As I posted earlier, I’m using Terraform Cloud both as a state provider for my development workspace, and as CI/CD for my dev/test and production workspaces. This produces a very nice development workflow.

1. Create a feature branch
2. Make the feature branch work
3. PR to QA branch
4. Terraform Cloud (TC) plans the updated code
5. Confirm plan in TC
6. PR to …
7 …

Infrastructure workflows are the same as Git workflows! With all of the features that Azure DevOps provides, including work item association and approvals. The pipeline itself is triggered upon push. But that’s ok. The code review process was required first, since the merge to the QA branch required approval. Then, TC will not automatically apply changes unless instructed to. Making infrastructure changes requires confirmation from a user with the appropriate access.

That’s the development workflow, and it works very well, and was not that difficult to set up given the extensive Terraform documentation. Personally I run it on WSL2 using VS Code with the Terraform extension. Infrastructure dev environments should probably run some flavor of Unix (WSL2, MacOS, Linux) and nothing more than VS Code is required.

Code organization is very important. Terraform is a declarative language with very few opportunities for reuse and almost no control structures. This makes for the possibility of unreadable code very quickly. Organizing into workspaces and modules is the only way to keep larger projects under control. It’s probably easiest to explain with an example, so let’s put down some requirements.

The project in question has 4 total environments, 3 non-prod and 1 prod. The 3 non-prod environments share a single Azure Dev/Test subscription, and the production environment has its own subscription. Costs should be minimized without sacrificing code readability. Development should plan and apply from the CLI or Web UI only. QA should plan and apply upon push to qa branch, and UAT should plan and apply upon push to uat branch. Finally, production should plan and apply upon push to main branch.

The first thing to note from these requirements is the cost minimization. It’s always easiest to duplicate, and it would be much easier to simply build three copies of all infrastructure in dev/test. But that’s expensive: the search service alone costs $300/mo per instance. And having multiple key vaults when one will suffice is kind of annoying. And you don’t need multiple storage accounts, multiple containers is fine. Upon analyzing the infrastructure requirements, it was clear that the Redis cache, the Key Vault, the Storage Account and the Container Repository could all be shared among the three non-prod projects.

So how to achieve this? My first attempt simply included the same module in multiple projects. Alas, this simply led to cycles of create/recreate as updates were made to the various branches. Creating and recreating the core infrastructure is absolutely the opposite of what we want. So, the first step was to split out the shared infrastructure into its own folder, and added the Terraform files. Then, the shared infrastructure modules moved, and the root Terraform definition updated. Finally, the shared infrastructure main.tf and variables.tf are updated.

Now, the workspace layout. First, we need a workspace to hold the shared infrastructure. I created a single workspace with a -devtest suffix and initialized it. From the shared directory, run terraform plan and terraform apply. This will create the shared infrastructure in a separate workspace so that there are no conflicts between the environments. This workspace should be CLI-driven. The development environment should always be CLI-driven, and the shared infrastructure for development also belongs to QA and UAT. Therefore, no other workspace is needed for Dev/Test core infrastructure. Note the need to create a corresponding production workspace later.

Next, we need workspaces and variable sets for each environment. The dev workspace will be marked as CLI-driven, and the other two (QA and UAT) will be attached to Azure DevOps Git and trigger runs upon push. The variable sets will be created and associated accordingly.

That’s the gist of it. Terraform really saves a great deal of effort and reduces mistakes, as well as providing a lot more process around infrastructure development. Good use of workspaces and modules can make for reasonably organized layouts for even more complex infrastructure requirements.