By initiating deployment and provisioning of cloud infrastructure resources using an IaC tool, three entities to be built upon the development and apply. I call them sources of truth that they should be in sync to keep the IaC setup properly. Nowadays IaC solutions like OpenTofu, Terraform, and Pulumi follow this structure and approach. These sources are:

• Upstream Resources: The resources that are created by IaC tools on Cloud providers, such as EC2 instances, RDS databases, VPC networks, and so on.

• State File: A file that stores the state of upstream resources in JSON format. it’s a reference used by the IaC tool to what has been created on the cloud and stores the state, IDs, links, etc.

• Template: A file developed by a human and depends on the IaC tool, in various formats like HCL, Python, Golang, etc. This file defines, the desired state of each resource on the cloud. States like EC2 instance types, S3 bucket name, RDS username, and many other parameters for each resource type.

These three sources form the foundation of an Infrastructure as Code setup and understanding each component is crucial for successfully managing your IaC.

Pushing components out of sync is unavoidable and can occur for various reasons, such as quick response to a service outage or incident by modifying infrastructure/upstream resources or human errors. These events cause IaC to become desynchronized state and understanding how these changes affect your setup is crucial for maintaining your IaC system.

Let’s dive into each source of truth and explore what to expect when any of them is out of sync state. We will investigate each source of truth through three events: Creation, Modification, and Deletion.

Upstream Resource:

This situation occurs during an incident or application outage, and quick actions and manual changes on infrastructure resources are necessary to resolve the issue. Also, It is especially common when someone is unfamiliar with IaC and prefers to make direct manual modifications.

• Deleting and recreating a resource can trigger creation by the IaC apply process. The apply process might fail if the resource is referenced in the state file by its name as its ID (like an S3 bucket)

• Modifying a resource triggers an update during the IaC plan/apply process due to re-align sources of truth. This situation is usually referred to as “Drift” when the actual state of the resource diverges from the desired state defined in the state file and the template.

• Removing a resource can trigger recreation during the IaC plan/apply process, as the IaC tool will attempt to restore it to match the desired configuration defined in the template.

State File:

Changes in state files are rare, especially when stored in remote backends like cloud object storage (S3). However, they can occur during manual maintenance tasks, such as modifying or moving states between multiple state files.

• Creating a state can trigger deletion during the IaC plan/apply process if resources with the same ID already exist upstream. This occurs because the state file prioritizes the template as the primary source of truth and attempts to synchronize upstream resources with the desired state defined in the template.

• Modifying the state file triggers change during the plan/apply process. However, if the template and upstream resources remain unchanged, no actual changes will occur in the infrastructure; the modification will only affect the state file.

• Removing a state triggers creation during the plan/apply process, and the IaC tool attempts to synchronize the resource with the template, which serves as the primary source of truth. Additionally, the apply process might fail if the resource exists upstream and is identified by its name as ID ( like S3 bucket)

Template File:

Finally, any changes made to the template file directly reflect your defined and desired configuration. Creation in the template results in resource creation, modifications trigger changes, and removals result in upstream resource termination. This process is the standard procedure for managing your infrastructure by IaC.

Conclusion:

In IaC changes and misalignments are unavoidable, often requiring manual adjustments to restore synchronization. However, it’s best to make changes to upstream resources using the template files. This way, you can minimize manual changes to the state files, helping to maintain consistency and prevent problems. Understanding the effects of changes on each component is crucial for analyzing the situation and making informed decisions on infrastructure stability.