In this blog post you learn the basics of Terraform State Management. In Part 2 of this two-part series you will get a deep dive into working with the state.

In my IaC Starter Guide you already read about the basics of Terraform. You learned how to setup an infrastructure for a simple website using S3. Today we will have a look into Terraform State and State Management.

The State - a place for all your resources

States

With Terraform you define your infrastructure resources using scripts. In our case these resources are deployed to AWS Cloud. But how does Terraform knows, which resources are already there, will be created, deleted or updated?

With this the Terraform State comes into play. The state stores your Terraform-created resources with its configuration. It is stored in the terraform.tfstate file and is JSON strcutured.

The state plays a significant role within your Terraform setup. If no state is in place, Terraform automatically assumes that all resources should be created anew.

Backends

Your state is part of the so-called backend which is used to execute operations in Terraform. It acts as a snapshot of your Terraform resources created. Depending on how the backend is configured you can choose between two type of states.

Types of States

You can choose between two type of states:

  • local - on your local PC and
  • remote - not locally stored, e.g. on a central server or in Terraform Cloud

two types of states

Your local state

If you don’t configure a backend explicitly, Terraform will create a local state automatically. Local states are suitable for beginners. The local state will be stored in your Terraform root directory as terraform.tfstate file.

I recommend to take a look into your state file to understand how Terraform is storing its resources.

Configuring your local state

If needed your can commit a configuration for a local state by configuring a backend. This can be useful if you store your local state at a central place. You can also choose to change the file name of your state. With this you define a local backend explicitly:

1
2
3
4
5

terraform {
  backend "local" {
    path = "relative/path/to/terraform.tfstate"
  }
}

If the file defined for path is not created yet, Terrafrom will create one for you.

Your remote state

A remote state is a Terraform State stored in a central data storage. There a variety of ways to store the remote state: e.g. with AWS S3, Azure Blob Storage, or a REST Interface and much more. To use the remote state your need to define a respective backend. You’ll find an overview about all backends available in the Terraform documentation: Terraform Remote Backends Overview

When you should use a remote state?

When working with others

If more than one people are working on a infrastructure setup it is useful to work with a state stored centrally. Additionally you can use state locking to prevent the state from being corrupted.

To split your setup

Building a large infrastructure setup it can be useful to split it up into smaller components. Thinking of this each component can be maintained by an individual team. With a remote state you can store important data as outputs and pass them on to another remote state.

Outputs coming from a state can be implemented using terraform_remote_state data source. You will learn more about this resource in part 2.

For secure storing and versioning

Depending on where you store your remote state you can make use of additional security or versioning for the state file. For instance if you use AWS S3 as a backend you can use S3 Encryption for encrypting your state file and S3 Versioning to store every version of your state. Thinking of version control system you can use such a system as well to store your state remotely. If your are using Gitlab there can be a way to go.

When using CI/CD

Automating is a must when running a successful software project. With a remote state your setup is not dependend on a specific local storage place. Be sure to you CI/CD pipeline tool access to the state file and you are ready to a automated infrastructure deployment.🚀

I always use a remote state using AWS S3 when working with Terrafrom. With this I ensure that my state is centrally available and can be accessed during a continious deployment process. A Additionally member of my team can maintain the setup, as well.

Create a remote state using S3 backend

💥 Attention: The S3 Bucket must already be created.

To store your state in an AWS S3 bucket, a backend of type s3 needs to be implemented.

1
2
3
4
5
6
7

terraform {
  backend "s3" {
    bucket = "mybucket" # unique S3 bucket name
    key    = "path/to/my/key" #ex: awesome/s3-backend/terraform.tfstate
    region = "eu-west-1"
  }
}

Keep in mind that you must have (programmatic) access to the bucket.

Working with the State

The state not only secures your Terraform resources. You can also extend, modify or lock your state. In part 1 of the series “Terraform State Management like a Boss” I show you the following two ways to work with the state.

Importing resources

You have already created resources like servers, databases etc. at a provider? No problem! With an import you can import already created resources into a state.

This is very easy with terraform import.

Terraform will then manage the resource for you, which is very useful when switching to Infrastructure as Code. Sometimes it happens that resources are created manually for testing, due to lack of time, ignorance or because they are not yet available for Terraform. Importing gives you the security that you can build your infrastructure sustainably in the future.

How can you import resources?

For each resource of a procider there is the import statement under “Import” in the Terraform documentation.

For example, this is how you can import an S3 bucket:

# Syntax: terraform import <provider resource type>.<resource name> <ident of the resource being imported>
terraform import aws_s3_bucket.bucket my-awesome-bucket-name

💡 Know-how: In order to successfully import resources, there are a few things you need to be aware of:

  • The resource you want to import must be defined in Terraform beforehand. It must already have a name that you can reference (see import command above).
  • No configurations will be imported. In the Planning Step you can then see which configuration was used and adjust the Terraform resource accordingly, if necessary.

Some little import helper for your AWS Resourcen

To import AWS resources including their configuration as terraformed resources, Terraforming by Daisuke Fujita can help you. It does not cover all AWS resources, but can be a super support for the most common ones.

State Locking

If several colleagues in the team work on the infrastructure, access the state and change things, locking of the Terraform state is necessary. This prevents the state from being corrupted.

State locking is not available for every backend. With the S3 backend, an AWS DynamoDB can be used to manage the locking state.

1
2
3
4
5
6
7
8

terraform {
  backend "s3" {
    bucket = "mybucket" # unique S3 bucket name
    key    = "path/to/my/key" # ex: awesome/s3-backend/terraform.tfstate
    region = "eu-west-1"
   dynamodb_table = "name_oy_my_dynamodb_table"
  }
}

💥Attention: The used DynamoDB must have the primary key (hash_key) LockID of type string. Otherwise State Locking will not be used.

Prospects: State Manipulation

By manipulating the state, you actively encroach in it. The Terraform CLI offers you the possibility to do this. Resources can be deleted, moved or even completely recreated. In part two of the series Terraform State Management like a Boss you will learn more about manipulating the state.

Tips and tricks

Be careful with sensitive data and use encryption

Be careful when working with sensitive data (e.g. passwords or IDs) in your Terraform script. For local states, Terraform stores everything in plain text in the state file (in terraform.tfstate). For remote states, the state files can be encrypted depending on the backend and configuration used. The state is only in memory when Terraform is using it.

It is recommended to use Terraform Cloud or AWS S3 as backend configuration. With both, you can turn on encryption. For an S3 backend, you turn on encryption via the encrypt property

Translated with www.DeepL.com/Translator (free version)

💥Keep in mind: The S3 bucket must be set up with S3 encryption (SSE-S3).

1
2
3
4
5
6
7
8

terraform {
  backend "s3" {
    bucket = "mybucket" # unique S3 bucket name
    key    = "path/to/my/key" # ex: awesome/s3-backend/terraform.tfstate
    region = "eu-west-1"
   encrypt = true
  }
}

S3 Backend: Turn on Versioning

An AWS S3 bucket as a backend not only offers the possibility of encryption (server-side and at REST), but also versioning. This preserves old versions of your state. With an appropriate lifecycle policy for the bucket, you can then also archive older versions on a regular basis - e.g. save them to another S3 storage class.

💡 Acloudguru has published a great tutorial about the complete setup (incl. state locking): http://aws-cloud.guru/terraform-aws-backend-for-remote-state-files-with-s3-and-dynamodb/

References:

How did you like this post? Feel free to send me your feedback, like or share on Twitter.

Happy Coding and see ya next time!

Nora See ya!