Terraform modules are a great way to simplify your Terraform code by writing it once and then reusing the modules across your Terraform workspaces. Terraform modules enable you to encapsulate and organize your infrastructure code into smaller, modular components. This modular approach helps in managing and scaling complex infrastructure deployments.
The creation of modules is relatively simple and there are a ton of resources to help you get started whether you write the module from scratch, pull it from the public Terraform module registry, or use a provider like Hestio to manage them for you. In this blog, we aren’t going to talk about creating the modules, but rather about scaling the usage of Terraform modules.
When scaling your module usage, there are a number of factors to consider:
The more Terraform modules you use and the more developers you have, the bigger the problem becomes. Users will always push for more modules and more services, but in the current state, it will be hard to keep track of what Terraform modules you have and who is using it.
If these challenges apply to you then you would likely benefit from a private module registry from a Terraform Automation & Collaboration (TACO) platform like Scalr or Terraform Cloud. Terraform private module registries allow you to create a curated list of modules to share with your organization. You’ll find the following common functionality across most private Terraform registries:
As mentioned, the list above is common across most of the Terraform module registries, but you’ll still have scaling issues within larger organizations around things like RBAC, reporting, and sharing of the modules.
Scalr has a few key differentiators in the way the module registry has been implemented:
Scalr created the module registry with hierarchical inheritance in mind to save administrators from an operational nightmare. The last thing you want is for teams to step on each other's feet when deploying, in Scalr you can create environments to give each team or app its own dedicated space to manage their deployments or workspaces. It would be an operational mess to have to go into every single environment and import the modules, which is where the inheritance model comes into play:
A module registry can be created at each level in the diagram above:
Create and maintain the module in one place and share with many organizations, teams, and environments. This allows you to create organizational standards, but also lets your individual teams add more modules at their respective scope if their permissions allow for it, but not change or remove modules from a higher scope.
Terraform code can be deployed into workspaces by using the Terraform CLI, through PR automation with a VCS provider, or through the Scalr private module registry referred to as “no-code deployments”. With this method, your users who are not as familiar with Terraform can create a workspace directly from the module registry and will be prompted to fill in any required inputs that do not already have a value. This greatly simplifies the overall experience, but the users will still see the core Terraform workflow in action while the resources are being created.
Once you have figured out how to distribute the Terraform module code and have your users deploy it in their workspaces, you’ll want to understand the overall usage. Just because you have told your users to use the module registry doesn't mean they are doing that. The Scalr Terraform reports feature gives you insights into the following:
That last point is critical as you’ll be able to identify if anyone is working around controls to pull Terraform modules from a different module registry or even from a Github repository that is not approved. The reports not only help you understand the module usage, but also ensure compliance.
As mentioned, you’ll need to make sure you are not a bottleneck for your development teams. Hestio, a Scalr partner, created low code modules that the most advanced or novice Terraform users can deploy. The modules can be shared with your organization and Hestio will take care of the maintenance for you.
With the Scalr module registry, you can easily and safely manage Terraform modules for any size organization. The key areas to think about when creating your module strategy is to:
Try it out in Scalr today, everything listed here is included in Scalr’s free tier.