Best Practices
Best Practices
January 2, 2024

Getting Started with Terraform Providers

By
Brendan Thompson

Today we are going to be looking into Terraform Providers, the absolute foundation to Terraforms success and ease of use. The topics being covered are:

  • What are they
  • How to use them
  • Tips/Tricks

What are they

Terraform Providers can be thought of as a plugin for Terraform that enables extra functionality, in a similar way that extensions in Visual Studio Code provide extra functionality to the tool. These Providers allow us to interface with essentially anything that has an API which is why Terraform is so powerful it gives a consistent language and style to interact with almost anything we can dream of.

Providers come in three different flavours (or Tiers):

  • Official — owned and maintained by HashiCorp (e.g. vault)
  • Partner — owned and maintained by a technology company that is a HashiCorp partner (e.g. azuredevops)
  • Community — published and maintained by individual contributors or groups within the Terraform ecosystem (e.g. scratch)

Let's step through the process of the provider with the below diagram.

Communication flow
  1. HCL — the Terraform code we write to provision resources on a platform or service
  2. Provider — the plugin for Terraform that takes the HCL, transforms it into the format required by the destination API
  3. SDK — the client library used by the Provider to communicate with the API in Golang
  4. API — the destination API that we want to communicate with

Using this pattern means that it is extremely easy for us as engineer to communicate with an API using Terraform, it also means that creating Providers for new APIs is also a relatively simple endeavour. HashiCorp provides a few methods for Provider development.

Another extremely import piece of the Provider plugin is the documentation, documentation about the Provider its resources and data sources live along side the Provider code itself and webpages can be easily automatically generated. Having these two things side-by-side enables extremely easy uptake from consuming engineers of the Provider itself.

Providers are generally sourced from a registry, either the one that comes from your TACO or from the public HashiCorp registry, however it is also possible to source from the local filesystem. This is fantastic when you're developing providers locally or you have providers that are internal and cannot be hosted externally for security reasons.

How to use them

Now we know what a Provider is and what it does we will look into how to actually use a Provider in our own Terraform code. There are some basics that are the same for every single provider however the attributes required vary from provider to provider, this is why it is extremely important to review the providers documentation. Lets first take a look at the basics and then we will look through some examples.

provider "some_provider" {}

Any provider declaration requires a call to the provider block with the name of the provide as the ID, the block however is not actually always required. If a provider does not require configuration (e.g. scratch, random) then no provider block is required. There are also scenarios wherein the provider itself has sensible defaults or is able to source required configuration items from environment variables that mean the declaration of a block is not required an example of this would be the google provider which if nothing is defined it will look for the configuration in environment variables or the gcloud cli configuration file(s).

It is good practice to have a dedicated providers.tf file, this ensures that identifying all in-use providers is extremely easy from a code perspective.

\

Let's dive into an example with the azurerm provider for Microsoft Azure. This provider, for the most part, will try to ingest its configuration from environment variables however this particular provider MUST always have the block defined due to the features block it requires. We will look at two examples of declaring this provider, one with the minimum requirements and one with everything needed to get going when not sourcing from the environment.

This first example shows the simplest form of declaration for this provider, it simply requires the existence of the features {} block. Personally I think this requirement is ridiculous as it doesn't provide any value and should be defaulted however currently it is a requirement.

 provider "azurerm" {
	features {}
}
The next example shows what is required to be passed into the provider in order for it to be able to authenticate and access Azure resources. In this instance we need a Service Principal client_id as well as a client_secret alongside these attributes we also need to know what tenant and subscription to operate within.

provider "azurerm" {
	features {}

	client_id       = "00000000-0000-0000-0000-000000000000"
	client_secret   = ""
	tenant_id       = "10000000-0000-0000-0000-000000000000"
	subscription_id = "20000000-0000-0000-0000-000000000000"
}

Next up let's look at the exact same scenario but for AWS. Thankfully AWS doesn't have something silly like the required features {} block 😃, we just need to pass an empty provider block and everything else will be sourced from the environment.

provider "aws" {}

The environment variables that are required are; AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_REGION with those defined it is able to authenticate and communicate with AWS. Next we will look at the provider block with those properties defined with Terraform.

provider "aws" {
	region     = "ap-southeast-2"
	access_key = ""
	secret_key = ""
}

In both examples we can see that the configuration required for providers is not overly cumbersome, for the most part it is about how to authenticate to a given platform or service. An important part of using a provider is ensuring that it is both up to date and sourced from the correct location. The below is what would be defined within the terraform.tf file and allows us to constrain the provider.

terraform {
	required_providers {
		azurerm = {
			source  = "hashicorp/azurerm"
			version = "~> 3.0"
		}

		aws = {
			source  = "registry.trust.me/cloud/aws"
			version = "=5.31.0"
		}
	}
}

In the above we are using the azurerm provider from the official HashiCorp registry and ensuring that we have the latest 3.x.x version of the provider. For AWS we are sourcing from third-party trusted registry and we are explicitly asking for a version of the provider.

Tips/Tricks

The biggest, and the most important tip with providers is the ability to have aliased version of the provider. This means you can have a different version of the provider, different credentials, different... anything! This becomes super powerful when you're managing infrastructure that spans multiple accounts etc. Let's dive into an example of having two provider definitions for azurerm.

provider "azurerm" {
	features {}
}

provider "azurerm" {
	alias = "network"

	features {}
}

Now we have our providers declared we can also ensure that they are constrained correctly too, with the following.

terraform {
	required_providers {
		azurerm = {
			source                = "hashicorp/azurerm"
			version               = "~> 3.0"
			configuration_aliases = [ azurerm.network ]
		}
	}
}

In the below example we are going to get a data source for a resource group using each provider. In a real world scenario on Azure it would be completed normal to have your networking live in a different place under a different service account, using provider aliases allows us to easily access both.

data "azurerm_resource_group" "app" {
	name = "rg-aue-prd-app"
}

data "azurerm_resource_group" "network" {
	provider = azurerm.network

	name = "rg-aue-prd-network"
}

Closing Out

Today we have learned that a provider is an easy way for Terraform and by extension us as engineers to communicate with other platforms/services APIs using a language we are all familiar with. We have gone through how to actually use a provider as well as the most important tip of all, provider aliases which allow us to have multiple connections to a single platform/service with different credentials or a different scope.

Start using the Terraform platform of the future.

A screenshot of the modules page in the Scalr Platform