Policy-Driven Terraform

Overview

When it comes to building repeatable, code-driven infrastructure in the cloud, one tool that everyone should investigate using is Terraform. By utilising Terraform, organisations can leverage the ability to create versioned, parameter-driven infrastructure and configuration across a wide variety of cloud providers, applications and business systems. To put it simply, the creative power you can achieve is vast. However, as somebody once mused: With great power comes great responsibility! How do you ensure that the infrastructure your engineers are creating is secure and compliant within your organisation’s guidelines or standards? Can you stop bad configuration before it has a chance to become a problem? Enter: Sentinel.

Sentinel is a Policy-as-Code framework developed by Hashicorp, the creators of Terraform. This framework gives platforms the power to create embedded, logic-driven, finely-grained policies to enforce conditions and decisions.

“Great, so how can I use this with Terraform?” you might be asking yourself! Well to answer that question, Sentinel support is built directly into Terraform Cloud, and by extension, Terraform Enterprise (the private, self-managed version of Cloud). Hashicorp’s official documentation describes Terraform Cloud like so:

Terraform Cloud is an application that helps teams use Terraform together. It manages Terraform runs in a consistent and reliable environment, and includes easy access to shared state and secret data, access controls for approving changes to infrastructure, a private registry for sharing Terraform modules, detailed policy controls for governing the contents of Terraform configurations, and more.

To paraphrase the important part in that extract — by utilising Terraform Cloud as the platform for executing your Terraform code, you gain the ability to enforce guardrails and policies on your infrastructure, allowing you to ensure the things you create are within the expected boundaries, and block their creation if they are not.

So Where Do My Policies Fit In?

The standard Terraform core workflow is typically broken down as follows:

1. Write — Infrastructure as Code is authored using the Terraform language.
2. Plan — Terraform reads the authored code and inspects the target environment, creating a preview of any planned changes before applying.
3. Apply — Terraform orchestrates the target environment, provisioning infrastructure and configuration as necessary.

Terraform Cloud further extends this workflow by interjecting a Policy Check stage before the Apply stage:

By introducing this stage, Terraform Cloud effectively creates a gated condition through which your code must pass before any target infrastructure is created. The level of enforcement is controlled here on a policy-by-policy basis through the use of enforcement levels, which control how Terraform will proceed in the case of policy breaches:

• Hard-mandatory requires that the policy passes. If a policy fails, the run is halted and may not be applied until the failure is resolved.
• Soft-mandatory is similar to hard-mandatory, but allows an administrator to override policy failures on a case-by-case basis.
• Advisory will never interrupt the run, and instead will only surface policy failures as an informational message to the user.

The Anatomy Of A Sentinel Policy

Before we look at a real world example, let’s first take a quick run-through the basics of a Sentinel Policy definition.

Sentinel Policies are written in a subset of the Hashicorp Configuration Language, or HCL for short. At the most abstract level, a Sentinel Policy looks like the following:

import "..."main = rule {
   ...  
}

import — The import statement allows a policy to access reusable libraries and data sources available within the policy execution environment. The following imports are available within Terraform Cloud:

• The standard Sentinel Imports — These allow access to common functions such as string manipulation, math functions, HTTP client creation, etc.
• tfconfig — Allows you to access information about the Terraform code being executed.
• tfrun — Allows you to access information about the run-time environment for the current Terraform execution.
• tfplan — Allows you to access information about the planned changes within a Terraform execution.
• tfstate — Allows you to access information about the stored state associated with an applied Terraform run.

main — Every policy must have a rule named main. This rule acts as the entry point for the policy execution.

rule — Rule blocks contain the conditions and evaluation logic which form the tests within a policy. Rules must return a boolean value depending on whether the rule is successful (true), or is disallowed (false).

Individual policies are stored within distinct files, usually with a .sentinel extension.

A very basic, working policy might look like the following:

import "tfconfig"

main = rule {
  tfconfig.modules.mymodule.version is "~> 1.0"
}

Here you can see that we are importing the tfconfig library, giving us the ability to inspect the Terraform code we are applying. Within our main rule we are specifying a check that our mymodule instance is referencing an allowed version. Again, if any code is encountered which contradicts this rule, then the policy is marked as failed.

Once you have defined your policies, you must also create a Policy Set. A Policy Set is a HCL file defining the policies to be evaluated in order. Each policy is referenced within a policy block containing a relative link to the policy file, along with the enforcement level for the policy. For example:

policy "my-policy-1" {
  source = "./policies/policy-1.sentinel"
  enforcement_level = "advisory"
}

policy "my-policy-2" {
  source = "./policies/policy-2.sentinel"
  enforcement_level = "hard-mandatory"
}

policy "my-policy-3" {
  source = "./policies/policy-3.sentinel"
  enforcement_level = "soft-mandatory"
}

A Policy Set file must be named sentinel.hcl. When stored together, the directory structure associated with the example Policy Set above might look like the following:

/
├─ policies/
│  ├─ policy-1.sentinel
│  ├─ policy-2.sentinel
│  ├─ policy-3.sentinel
├─ sentinel.hcl

To help accelerate consumers in building their policy libraries, Hashicorp provide examples of the recommended directory structure, alongside useful reusable policy functions and sample policies in their terraform-guidelines repository, here.

Creating Our Code

In order to demonstrate a policy in action, we’re going to follow a very simple example whereby we will use Terraform to create an S3 bucket within AWS. We’re going to imagine that we have a hard requirement within our organisation that any buckets we create must not be publicly available; any buckets which do not follow this requirement will be prevented from being created.

The Terraform code we’re going to use to achieve this is as follows:

provider "aws" {
  region = "eu-west-1"
}

variable "acl" {
  description = "Bucket ACL - public-read, private, etc"
}

resource "aws_s3_bucket" "this" {
  bucket_prefix = "example-bucket"
  acl           = var.acl
}

As you can see, we are using the Terraform AWS provider to create our resources. We have used a variable for the bucket acl property as we will be varying the value in our workflow to demonstrate our policy being applied.

In order to apply our control, our policy will be written as follows:

import "tfplan/v2" as tfplanbuckets = filter tfplan.resource_changes as _, rc {
  rc.mode is "managed" and
  rc.type is "aws_s3_bucket"
}

main = rule {
  all buckets as _, bucket {
    bucket.change.after.acl is "private"
  }
}

Here you will see the following sections within the policy:

• First we import the Terraform plan information with the import statement. This allows us to reference the resources which will be modified as a part of our Terraform execution.
• Next we filter the planned actions to target only the AWS S3 bucket resources within our code. We assign the return value of this call to the buckets variable.
• Finally we use the main rule to loop over the items within our buckets collection, asserting whether the acl value for each is private. When the assertion encounters a non-private value, it will return false, causing the main rule to fail.

In order to enforce this policy, our Policy Set definition is created as follows:

policy "s3-bucket-acl" {
    source = "./s3-bucket-acl.sentinel"
    enforcement_level = "hard-mandatory"
}

Here, you can see that we have used the hard-mandatory enforcement level to ensure that any policy failures prevent infrastructure changes from taking place as per our requirement.

For the sake of this example, both our terraform code and our sentinel policies are saved to separate Github repositories, to which Terraform Cloud is given read access.

Joining The Dots

Now that we have created both our Terraform infrastructure code and our Sentinel Policy Set, we can configure our Terraform Cloud environment.

First of all, we must link our Terraform Cloud Organisation to our Policy Set. By doing so, we ensure that all infrastructure created within our organisation is evaluated against our policies.

Once our Policy Set is linked, Terraform Cloud continues to monitor our policy repository and automatically updates its internal references whenever changes are made, ensuring that our latest policy code is used to evalute any future Terraform executions.

Next we create a Terraform Workspace to contain our Terraform run. Here, we are going to create a Version Control Workflow workspace. This type of workspace allows Terraform Cloud to directly checkout and read our Terraform code, in addition to allowing us the ability to queue Terraform executions directly within the console.

Once we have followed all the UI prompts and have successfully linked our workspace to our code, we are given access to modify the new workspace. As we earlier defined the bucket ACL property as a variable within our Terraform code, we are prompted to configure our workspace variables.

Upon executing our run via the “Queue plan” button, the Terraform Cloud workflow highlighted earlier will execute, performing the following actions:

• Our infrastructure code and policy definitions will be checked out into our workspace.
• A terraform plan will be executed.
• The output of the plan will be evaluated against our policies.
• Depending on whether our policies are passed or not, our execution will either be failed outright, or we will be prompted to authorise the terraform apply stage.

Testing Failure

For our first run through, we are going to simulate a blocked resource by setting the acl variable to public-read.

When we execute the plan, you can see Terraform Cloud performs the expected workflow highlighted above, failing the run and preventing any infrastructure from being created:

Testing Success

Now we have proven that our policy is blocking non-compliant infrastructure as expected, we can simulate an allowed resource by setting the acl variable to private.

When we execute the plan this time, you can see that Terraform Cloud performs the same expected workflow, but in this case proceeds through the policy check stage with a successful outcome, before finally prompting us to continue with the terraform apply stage:

We can now manually confirm the planned infrastructure changes and allow Terraform to continue creating our desired resources.

In Conclusion

In this article we have introduced you to the Hashicorp Sentinel Framework, how Sentinel policies are defined, and through a real example, shown Terraform Cloud actively enforcing policies during infrastructure creation.

As with any introduction however, the information presented here only touches the surface of what you can achieve by using Sentinel in conjunction with Terraform Cloud. What we hope you will take away is the value you unlock through the use of the tools described, and how you can achieve the ability to ensure the creation of compliant, regulated resources first time throughout your infrastructure lifecycle.

If you’ve been inspired to continue the journey of enforcing your policies as code within Terraform, here are some helpful resources to get you started:

• Terraform Cloud Homepage: https://www.terraform.io/cloud
• Terraform Cloud Sentinel Documentation: https://www.terraform.io/docs/cloud/sentinel/index.html
• Hashicorp Sentinel Guidelines Repository: https://github.com/hashicorp/terraform-guides/tree/master/governance/third-generation