In the previous; Learn Terraform: The Ultimate terraform tutorial [PART-1], you got a jump start into Terraform world; why not gain a more advanced level of knowledge of Terraform that you need to become a terraform pro.

In this Learn Terraform: The Ultimate terraform tutorial [PART-2] guide, you will learn more advanced level of Terraform concepts such as terraform lifecycle, terraform function, terraform modules, terraform provisioners, terraform init, terraform plan, terraform apply commands and many more.

Without further delay, let’s get into it.

Table of Content

1. What are Terraform modules?
2. Terraform provisioner
3. Terraform Lifecycle
4. Terraform jsonencode example with Terraform json
5. Terraform locals
6. Terraform conditional expression
7. Terraform dynamic block conditional
8. Terraform functions
9. Terraform can function
10. Terraform try function
11. Terraform templatefile function
12. Terraform data source
13. Terraform State file
14. Terraform backend [terraform backend s3]
15. Terraform Command Line or Terraform CLI
16. Quick Glance of Terraform CLI Commands
17. Terraform ec2 instance example (terraform aws ec2)

What are Terraform modules?

Terraform modules contain the terraform configuration files that may be managing a single resource or group of resources. For example, if you are managing a single resource in the single terraform configuration file that is also a Terraform module, or if you wish to manage multiple resources defined different files and later clubbed together in a single file, that is also known as Terraform modules or a root module.

A Terraform root module can have multiple individual child modules, data blocks, resources blocks, and so on. To call a child module, you will need to explicitly define the location of the child module using the source argument as shown below.

• In the below code the location of module EFS is one directory behind the current directory, so you defined the local Path as ./modules/EFS

module "efs" {                            # Module and Label is efs
  source               = "./modules/EFS"  # Define the Path of Child Module                             
  subnets              = var.subnet_ids
  efs_file_system_name = var.efs_file_system_name
  security_groups      = [module.SG.efs_sg_id]
  role_arn             = var.role_arn
}

• In some cases the modules are stored in Terraform Registry, GitHub, Bitbucket, Mercurial Repo, S3 bucket etc and to use these repsoitories as your source you need to declare as shown below.

module "mymodule1" {                              # Local Path located  Module
  source = "./consul"
}

module "mymodule2" {                              # Terraform Registry located Module
  source = ".hasicorp/consul/aws"
  version = "0.1.0"
}

module "mymodule3" {                              # GIT located  Module
  source = "github.com/automateinfra/"
}

module "mymodule4" {                              # Mercurial located  Module
  source = "hg::https://automateinfra.com/vpc.hg"
}

module "mymodule5" {                               # S3 Bucket located  Module
  source = "s3::https://s3-eu-west-1.amazonaws.com/vpc.zip"
}

Terraform provisioner

Do you know what Terraform allows you to perform an action on your local machine or remote machine such as running a command on the local machine, copying files from local to remote machines or vice versa, Passing data into virtual machines, etc. all this can be done by using Terraform provisioner?

Terraform provisioners allow to pass data in any resource that cannot be passed when creating resources. Multiple terraform provisioners can be specified within a resource block and executed in the order they’re defined in the configuration file.

The terraform provisioners interact with remote servers over SSH or WinRM. Most cloud computing platforms provide mechanisms to pass data to instances at the time of their creation such that the data is immediately available on system boot. Still, you can pass the data with Terraform provisioners even after creating the resource.

Terraform provisioners allows you to declare conditions such as when = destroy , on_failure = continue and If you wish to run terraform provisioners that aren’t directly associated with a specific resource, use null_resource.

Let’s look at the example below to declare multiple terraform provisioners.

• Below code creates two resources where resource1 create an AWS EC2 instance and other work with Terraform provisioner and performs action on the AWS EC2 instance such as copying apache installation instrution from local machine to remote machine and then using file installing apache on the AWS EC2 instance.

resource "aws_instance" "resource1" {
  instance_type = "t2.micro"
  ami           = "ami-9876"
  timeouts {                     # Customize your operations longevity
   create = "60m"
   delete = "2h"
   }
}

resource "aws_instance" "resource2" {

  provisioner "local-exec" {
    command = "echo 'Automateinfra.com' >text.txt"
  }
  provisioner "file" {
    source      = "text.txt"
    destination = "/tmp/text.txt"
  }
  provisioner "remote-exec" {
    inline = [
      "apt install apache2 -f /tmp/text.txt",
    ]
  }
}

Terraform Lifecycle

Terraform lifecycle defines the behavior of resources how they should be treated, such as ignoring changes to tags, preventing destroy the infrastructure.

There are mainly three arguments that you can declare within the Terraform lifecycle such as :

1. create_before_destroy: By default Terraform destroys the existing object and then create a new replacement object but with create_before_destroy argument within terraform lifecycle the new replacement object is created first, and then the legacy or prior object is destroyed.
2. prevent_destroy: Terraform skips the destruction of the existing object if you declare prevent_destroy within the terraform lifecycle.
3. ignores-changes: When you execute Terraform commands if there are any differences or changes required in the infrastructure terraform by default informs you however if you need to ignores the changes, then consider using ignore_changes inside the terraform lifecycle.

• In the below code aws_instance will ignore any tag changes for the instance and for azurerm_resource_group new resource group is created first and then destroyed once the new replacement is ready.

resource "aws_instance" "automate" {
  lifecycle {
    ignore_changes = [
      tags,
    ]
  }
}

resource "azurerm_resource_group" "automate" {
  lifecycle {
    create_before_destroy = true
  }
}

Terraform jsonencode example with Terraform json

If you need to encode Json files in your terraform code, consider using terraform jsonencode function. This is a quick section about terraform jsonencode, so let’s look at a basic Terraform jsonencode example with Terraform json.

• The below code creates an IAM role policy in which you are defining the policy statement in json format.

resource "aws_iam_role_policy" "example" {
  name   = "example"
  role   = aws_iam_role.example.name
  policy = jsonencode({
    "Statement" = [{
      # This policy allows software running on the EC2 instance to access the S3 API
      "Action" = "s3:*",
      "Effect" = "Allow",
    }],
  })
}

Terraform locals

Terraform locals are the values that are declared once but can be referred to multiple times in the resource or module block without repeating it.

Terraform locals helps you to decrease the number of code lines and reduce the repetitive code.

locals {                                         # Declaring the set of related locals in a single block
  instance = "t2.micro"
  name     = "myinstance"
}

locals {                                         # Using the Local values 
 common_tags {
  instance_type  = local.instance
  instance_name  = local.name
   }
}

resource "aws_instance" "instance1" {            # Using the newly created Local values
  tags = local.common_tags
}

resource "aws_instance" "instance2" {             # Using the newly created Local values
  tags = local.common_tags
}

Terraform conditional expression

There is multiple time when you will encounter using conditional expressions in Terraform. Let’s look at some important terraform conditional expression examples below, which will forever help you using Terraform. Let’s get into it.

• Below are examples on how to retrieve outputs with different conditions.

aws_instance.myinstance.id      # This will provide you a result with Ec2 Instance details.
aws_instance.myinstance[0].id   # This will provide you a result with first Ec2 Instance details.
aws_instance.myinstance[1].id   # This will provide you a result with second Ec2 Instance details
aws_instance.myinstance.*id     # This will provide you a result with all Ec2 Instance details

• Now, let us see few complex examples where different conditions are applied to retrieve outputs.

[for value in aws_instance.myinstance:value.id]    # Returns all instance values by their ids.
var.a != "auto" ? var.a : "default-a"              # if var.a is auto then use var.a else default-a
[for a in var.list : a.instance[0].name]           # var.list[*].instance[0].name
[for a in var.list: upper(a)]                      # iterates over each item in var.list and lists upper case 
[for a in var.list : a => upper(a)]     # list original objects and corresponding upper case [("a","A"),("c","C")]                                                         


Terraform dynamic block conditional

Terraform dynamic block conditional is used when resource or module block cannot accept the static value of the argument and instead depend on separate objects that are related to, embedded within the other block or outputs.

For example application = "${aws_elastic_beanstalk_application.tftest.name}" .

Also, while creating any resource in the module, you are not allowed to provide the arguments multiple times, such as name and value, so in that case, you can use dynamic settings. Below is a basic example of a dynamic setting.

resource "aws_elastic_beanstalk_environment" "tfenvtest" {
  name                = "tf-test-name"
  application         = "${aws_elastic_beanstalk_application.tftest.name}"
  solution_stack_name = "64bit Amazon Linux 2018.03 v2.11.4 running Go 1.12.6"

  dynamic "setting" {
    for_each = var.settings
    content {
      namespace = setting.value["namespace"]
      name = setting.value["name"]
      value = setting.value["value"]
    }
  }
}

Terraform functions

The Terraform includes multiple terraform functions, also known as built-in functions that you can call from within expressions to transform and combine values. The syntax for function calls is a function name followed by comma-separated arguments in parentheses: min, join, element, jsonencode, etc.

min(2,3,4)                                 # The output of this function is 2

join("","","hello", "Automate", "infra")   # The output of this function is hello, Automate , infra

element(["a", "b", "c"], length(["a", "b", "c"])-1)   # The output of this function is c

lookup({a="ay", b="bee"}, "c", "unknown?")          # The output of this function is unknown

jsonencode({"hello"="Automate"})          # The output of this function is {"hello":"Automate"}

jsondecode("{\"hello\": \"Automate\"}")   # The output of this function is { "hello"="Automate"}                                                           
                                                                 

Terraform can function

Terraform can evaluate the given expression or condition and accordingly returns a boolean value (true if the expression is valid, else false if the result has any errors. This special function can catch errors produced when evaluating its argument.

local.instance {
  myinstance1 = "t2.micro"
  myinstance2 = "t2.medium"
}

can(local.instance.myinstance1) #  This is True
can(local.instance.myinstance3) #  This is False

variable "time" {
  validation {
     condition  = can(formatdate("","var.time"))   # Checking the 2nd argument
  }
}


Terraform try function

Terraform try function evaluates all of its argument expressions in turn and returns the result of the first one that does not produce any errors.

As you can check below, the terraform try function checks the expression and returns the first correct option, t2.micro, in the first and second options in the second case.

local.instance {
  myinstance1 = "t2.micro"
  myinstance2 = "t2.medium"
}

try(local.instance.myinstance1, "second-option") # This is available in local so output is t2.micro
try(local.instance.myinstance3, "second-option") # This is not available in local so output is second-option

Terraform templatefile function

The Terraform templatefile function reads the file at a given directory or path and renders the content present in the file as a template using the template variable.

Syntax: templatefile(path,vars)

• Lets understand the example of Terraform templatefile function with Lists. Given below is the backend.tpl template file with below content. When you execute the templatefile() function it renders the backend.tpl and assigns the address and port to the backend argument.

# backend.tpl

%{for addr in ipaddr ~}      # Condition via Directive
backend ${addr}:${port}      # Prints this      
%{end for ~}                 # Condition via Directive

templatefile("${path.module}/backend.tpl, { port = 8080 , ipaddr =["1.1.1.1","2.2.2.2"]})

backend 1.1.1.1:8080
backend 2.2.2.2:8080

• Lets checkout another example of Terraform templatefile function but this time with maps. When you execute the templatefile() function it renders the backend.tpl and assigns the value of set with each config mentioned in the templatefile (a=automate and i=infra).

# backend.tmpl

%{ for key,value in config }
set ${key} = ${value}
%{ endfor ~}

• Execute the function

templatefile("${path.module}/backend.tmpl,
     { 
        config = {
              "a" = "automate"
              "i" = "infra"
           } 
      })

set a = automate
set i = infra

Terraform data source

Terraform data source allows you to fetch the data defined outside of Terraform, defined by another separate Terraform configuration, or modified by functions. After fetching the data, Terraform data source can use it as input and apply it to other resources.

Let’s learn with a basic example. In the below code, you will notice that using data it is fetching the instance details with a provided instance_id.

data "aws_instance" "my-machine1" {          # Fetching the instance
  instance_id = "i-0a0269cf952a02832"
  }

Terraform State file

The main function of the Terraform state file is to store the terraform state, which contains bindings between objects in remote systems and is defined in your Terraform configuration files. Terraform State file is by default stored locally on your machine where you run the Terraform commands with the name of terraform.tfstate.

The Terraform state is stored in JSON formats. When you run terraform show or terraform output command, it fetches the output in JSON format from the Terraform state file. Also, you can import existing infrastructure which you have created by other means such as manually or using scripts within Terraform state file.

When you are an individual, it is ok to keep the Terraform state file in your local machine but when you work in a team, consider storing it in a repository such as AWS S3, etc. While you write anything on the resource that is Terraform configuration file, then the Terraform state file gets Locked, which prevents someone else from using it simultaneously and avoids it being corrupted.

You can store your remote state file in S3, Terraform Cloud, Hasicorp consul, Google cloud storage, Azure blob storage, etc.

Terraform backend [terraform backend s3]

Terraform backend is a location where terraform state file resides. The Terraform state file contains all the resource details and tracking which were provisioned or will be provisioned with Terraform, such as terraform plan or terraform apply command.

There are two types of Backend; one is local that resides where you run terraform from it could be Linux machine, windows machine or wherever you run it from, and other is remote backend which could be SAAS based URL or storage location such as AWS S3 bucket.

Let’s take a look at how you can configure local backend or remote backend with terraform backend s3

# Local Backend
# whenever statefile is created or updates it is stored in local machine.

terraform {
  backend "local" {
    path = "relative/path/to/terraform.tfstate"
  }
}

# Configuring Terraform to use the remote terraform backend s3.
# whenever statefile is created or updates it is stored in AWS S3 bucket. 

terraform {
  backend "s3" {
    bucket = "mybucket"
    key    = "path/to/my/key"
    region = "us-east-2"
  }
}

Terraform Command Line or Terraform CLI

The Terraform command-line interface or Terraform CLI can be used via terraform command, which accepts a variety of subcommands such as terraform init or terraform plan. Below is the list of all of the supported subcommands.

• terraform init: It initializes the provider, module version requirements, and backend configurations.

• terraform init -input=true ➔ You can need to provide the inputs on the command line else terraform will fail.

• terraform init -lock=false ➔ Disable lock of terraform state file but this is not recommended

• terraform init -upgrade ➔ Upgrades Terraform modules and Terraform plugins

• terraform get: terraform get command downloads and updates modules mentioned in the root module.

• terraform plan: terraform plan command determines the state of all resources and compares them with real or existing infrastructure. It uses terraform state file data to compare and provider API to check.

• terraform plan -compact-warnings ➔ Provides the summary of warnings

• terraform plan -out=path ➔ Saves the execution plan on the specific directory.

• terraform plan -var-file= abc.tfvars ➔ To use specfic terraform.tfvars specified in the directory.

• terraform apply: To apply the changes in a specific cloud such as AWS or Azure.

• terraform apply -backup=path ➔ To backup the Terraform state file

• terraform apply -lock=true ➔ Locks the state file

• terraform apply -state=path ➔ prompts to provide the path to save the state file or use it for later runs.

• terraform apply -var-file= abc.tfvars ➔ Enter the specific terraform.tfvars which contains environment-wise variables.

• terraform apply -auto-approve ➔ This command will not prompt to approve the apply command.

• terraform destroy: It will destroy terraform-managed infrastructure or the existing enviornment created by Terraform.

• terraform destroy -auto-approve ➔ This command will not prompt to approve the destroy command.

• terraform console: Provides interactive console to evaluate the expressions such as join command or split command.

• terraform console -state=path ➔ Path to local state file

• terraform fmt: terraform fmt command formats the configuration files in the proper format.

• terraform fmt -check ➔ Checks the input format

• terraform fmt – recursive ➔ It formats Terraform configuration files stored in subdirectories.

• terraform fmt – diff ➔ displays the difference between the current and previous formats.

• terraform validates: It validates the Terraform configuration files.

• terraform validate -json ➔ Output is in json format

• terraform graph: terraform graph generates a visual representation of the execution plan in graph form.

• terraform graph -draw-cycles

• terraform graph -type=plan

• terraform output: terraform output command extracts the values of an output variable from the state file.

• terraform output -json

• terraform output -state=path

• terraform state list: It lists all the resources present in the state file created or imported by Terraform.

• terraform state list – id=id ➔ This command will search for a particular resource using resource id in Terraform state file.

• terraform state list -state=path ➔ This command will prompt you to provide the path of the state file and then provides the list of all resources in terraform state file.

• terraform state show: It shows attributes of specific resources.

• terraform state show -state=path ➔ This command will prompt you to provide the path and then provide the attributes of specific resources.

• terraform import: This command will import existing resources from infrastructure which was not created using terraform but will be imported in terraform state file and will be included in Terraform next time we run it.

• terraform refresh: It will reconcile the Terraform state file. Whatever resource you created using terraform and if they are manually or by any means modified, the refresh will sync them in the state file.

• terraform state rm: This command will remove the resources from the Terraform state file without actually removing the existing resources.

• terraform state mv: This command moves the resources within the Terraform state file from one location to another

• terraform state pull: This command will manually download the Terraform state file from a remote state in your local machine.

• terraform state push: This command will manually upload the local state file to the remote state.

Quick Glance of Terraform CLI Commands

Terraform ec2 instance example (terraform aws ec2)

Let’s wrap up this ultimate guide with a basic Terraform ec2 instance example or terraform aws ec2.

• Assuming you already have Terraform installed on your machine.
• First create a folder of your choice in any directory and a file named main.tf inside it and copy/paste the below content.

# This is main.tf terraform file.

resource "aws_instance" "my-machine" {
  ami = "ami-0a91cd140a1fc148a"
  for_each  = {
      key1 = "t2.micro"
	  key2 = "t2.medium"
   }
    instance_type      = each.value
	key_name       = each.key
    tags =  {
	   Name  = each.value
	}
}

resource "aws_iam_user" "accounts" {
  for_each = toset( ["Account11", "Account12", "Account13", "Account14"] )
  name     = each.key
}


• Create another file vars.tf inside the same folder and copy/paste the below content.

#  This is var.tf terraform file.

variable "tag_ec2" {
  type = list(string)
  default = ["ec21a","ec21b"]
}

• Finally, create another file output.tf again in the same folder and copy/paste the below content.

# This is  output.tf terraform file

output "aws_instance" {
   value = "${aws_instance.my-machine.*.id}"
}
output "aws_iam_user" {
   value = "${aws_iam_user.accounts.*.name}"
}


Make sure your machine has Terraform role attached or Terraform credentials configured properly before you run the below Terraform commands.

• To find out the Terraform version run terraform -version command.

terraform -version  # It gives Terraform Version information

• Now Initialize the terraform by running the terraform init command in same working directory where you have all the above terraform configuration files.

terraform init   # To initialize the terraform 

• Next run the terraform plan command. This command provides the blueprint of what all resources will be deployed before deploying actually.

terraform plan   

• Validate the terraform configuration files by running the terraform validate command.

terraform validate   # To validate all terraform configuration files.

• Now run the Terraform show command provides the human readable output of state file that gets generated only after terraform plan command.

terraform show   # To provide human-readable output from a state or plan file.

• To list all resources within terraform state file run the terraform state list command.

terraform state list 

• Finally use terraform apply command to provision the resources.

terraform apply  # To Actually apply the resources 

• To provide graphical view of all resources in configuration files run terraform graph command.

terraform graph  

• To Destroy the resources that are provisioned using Terraform run Terraform destroy command.

terraform destroy   # Destroys all your resources or the one which you specified 

Conclusion

Now that you have learned everything you should know about Terraform, you are sure going to be the Terraform leader in your upcoming projects or team or organizations.

So with that, what are you planning to automate using Terraform in your next adventure?