Pre-deployment Policy Checks for Terraform using OPA

Overview

To achieve cloud compliance and improve security, many organizations depended on policies and procedures. Review processes have been defined to approve and review proposed changes to the infrastructure for better governance and audit.

The change review process used to be manual, requiring a substantial time investment before the advent of automation, scanning, and collaborative tools. One of the current consequences is the risk of extended delays between the change request and implementation of a change.

There are many tools and platforms available in the market that can help organizations to speed up the process through automation and help teams to lessen the review process time. These tool range from Open-source solutions to Enterprise level SaaS based applications. OPA is the open source Policy as Code testing tool, which helps in speeding up the process by evaluating the infrastructure code before it goes to production against the rules or policies that you defined and make you organization secure.

In this blog article, I’ll teach you how to use the Open Policy Agent to begin testing your Terraform code.

Infrastructure-As-Code:

Infrastructure-as-code is a programmatic way of defining and provisioning cloud resources. Hashicorp Terraform is one of the infrastructure provisioning tools that allows you to specify and describe the infrastructure you want and automatically creates, deletes, and modifies your existing infrastructure.

OPA (Open Policy Agent):

OPA states the following on their website:

“Stop using a different policy language, policy model, and policy API for every product and service you use. Use OPA for a unified toolset and framework for policy across the cloud native stack.
Whether for one service or for all your services, use OPA to decouple policy from the service’s code so you can release, analyze, and review policies (which security and compliance teams love) without sacrificing availability or performance.”

There are numerous option available to get started playing with OPA.

First, OPA has a CLI that you can easily interact with it that you can easily download it from here.

If you are interested in Docker. The latest stable image tag is openpolicyagent/opa:latest

What is Rego:

Rego is the language used to write the OPA policy. Rego is declarative so policy authors can focus on what queries should return rather than how queries should be executed. These queries are simpler and more concise than the equivalent in an imperative language.

Let’s have a look at some of the commands accessible in the CLI.

eval → for evaluating Rego expressions and policies

run → Start OPA in interactive or server mode, interactive mode which is read–eval–print loop (REPL) is the default mode when no flags provided and the other one is the server mode which is activated with -s or --server flag.

Using OPA to test your Terraform Configuration:

OPA acts as a gate before it actually provisions any infrastructure, So it will be very easy for teams to identify compliance issues at the very earliest opportunity.

Here’s a workflow that uses OPA to determine whether the Terraform (TF) code given is valid or not in a pipeline:

The more detailed Workflow of how you can integrate the process with AWS Services and use git to store policies:

Step by Step guide to test sample terraform configuration:

1- Lets take a simple terraform configuration file

For example:

resource "aws_instance" "my_web_instance" {
 instance_type = "t2.nano"
 ami = "ami-09b4b74c"
 tags = {
  "team" = "Digital",
  "service" = "tomcat",
  "env" = "Dev"
 }
}

2- After initializing, create terraform plan with output in binary:

terraform init
terraform plan --out tfplan.binary

3- Convert Terraform Plan to json :

tfjson tfplan.binary > tfplan.json

Here, I have used this JSON converter

4- Now, You need OPA Policy (You can see policy examples here)

In the below policy, we are checking if the resource contain mandatory tags which is in our case are team, service and env . Also we are validating the env value should be from the following dict {`qa`, `stb`, `prod`}

Here, I have divided the policy in two files, which is very easy to manage when we have multiple policies. In the main.rego which is the parent policy file, I am calling the child policy file tags.rego . If any change is required in the child policy it will not impact the parent policy, and like this you can attach as many child policies as you require to the parent policy.

main.rego
package policy

default all_policies = false

import input as params

all_policies {
 #loop for each resource
 checkPoliciesForResource(params.changedResources[_])
}

checkPoliciesForResource(resource) {
 checkTags(resource)
}

tags.rego
# package needs to be same as that of the main file,
# else these components will not be included

package policy

# the allowed values for the tag 'env'
allowedEnvTags = {
 "prod",
 "stb",
 "qa"
}

#entry point for tags enforcement 
checkTags(resource) {
 # read the tags based on the resource type
 tags = readTags(resource.type, resource)
 # check for the tag enforcement
 ensureMandatoryTags(tags)
}

# every resource to be evaluated will have a 'readTags' function for
# itself the returned document should resemble the below structure
# {'tag-name': {value: 'tag-value'}}
readTags("aws_instance", resource) = tags {
 tags = resource.changedAttributes.tags
}

#check if all the mandatory tags are available & have allowed values
ensureMandatoryTags(tags) {
 # check if the team name is provided
 checkTagHasValue(tags["team"])
 #check if the service name is provided
 checkTagHasValue(tags["service"])
 #check if the env is allowed value
 allowedEnvTags[tags["env"].value]
}

checkTagHasValue(tag) {
 re_match("[^\\s]+", tag.value)
}

Evaluate OPA :

opa eval data.policy.all_policies -d infra-policies/ -i tfplan.json -f pretty

Output:

The output will be boolean, because we have set default all_policies = falsein the main.rego

OPA Playground:

OPA also provides the Rego Playground which is online interactive environment, where you can play with buit-in policies or add your own policy to test. Checkout here: https://play.openpolicyagent.org/

Summary

There are many tools available which can provide similar functionality to OPA like Terratest, Kitchen, InSpec but these tools require resource provisioning to perform tests which takes time to provision and adds cost. Hashicorp Sentinel provides Policy as Code testing without any resource provisioning, but Sentinel comes with Terraform Enterprise Solution which is a paid product. Therefore a benefit of OPA is that it is open source and free to use. You can leverage it easily according to your organization’s needs and also its coverage has been increased to Kubernetes, Kafka, HTTP APIs, etc. Also, you can view the OPA ecosystem which is a great place to see integrations, use cases, and related projects.

In addition, OPA has a lot of nice tooling builds, for example: conftest, deprek8ion, kube-mgmt and gatekeeper.

Photo by Caspar Camille Rubin on Unsplash