In this post we are going to cover Terraform state and remote backends. These are 7 real Terraform interview questions for data engineers, asked the way they come up on a real project, and the last one is a scenario: the state file was deleted by accident, the platform is still running, and you have to recover.
State is Terraform's record of what it manages. Every plan starts from it, so when state goes wrong, everything goes wrong: two applies corrupt it, a hand change in the portal drifts away from it, a rename in code makes Terraform want to destroy a container. That is why interviewers spend a whole round on it.
Part 2 of the Real Time Terraform Interview Questions and Answers series on TechBrothersIT.
- The short answer
- Let's set up the project
- Question 1: A teammate asks what the state file is and why Terraform needs it. How do you explain it?
- Question 2: Two engineers ran apply at the same time and the state got corrupted. How do remote state and locking prevent that?
- Question 3: Your manager asks where to store state for an Azure data platform. How do you set up the backend?
- Question 4: Someone changed a resource in the portal and Terraform wants to change it back. How do you handle drift?
- Question 5: A resource was renamed in the code and the plan wants to destroy and recreate it. How do you fix that safely?
- Question 6: Security asks why the state file is treated as sensitive. What is in it, and how do you protect it?
- Scenario 7: The state file was deleted by accident and the platform is still running. How do you recover?
- The full script
The short answer
State maps every address in the code to a real object. Keep it in a remote backend: for Azure, a blob in a dedicated, versioned storage account with Entra ID auth and a lease lock. Handle drift with refresh-only plans, renames with moved blocks, protect state like a secret, and recover a lost state from a blob version.
Let's set up the project
We use a small Terraform 1.17 project. We use local resources: terraform_data stands in for the resource group, and local_file stands in for a lake container and a pipeline config. So the plan is real and nothing is created in Azure. The bootstrap/ folder (the state storage account) and the azure/ folder (the backend block) are real azurerm 5.11 code, checked with terraform validate. Before the first step we run terraform init in the root folder, and init in bootstrap/ and in azure/ (with -backend=false).
terraform {
required_version = "~> 1.17"
required_providers {
local = {
source = "hashicorp/local"
version = "~> 2.9"
}
time = {
source = "hashicorp/time"
version = "~> 0.14"
}
}
}# Local stand-ins: the plan is real, nothing is created in Azure
resource "terraform_data" "resource_group" {
input = "rg-sales-data-dev"
}
resource "local_file" "landing" {
filename = "${path.module}/lake/landing.keep"
content = "container in ${terraform_data.resource_group.output}"
}
resource "local_file" "pipeline" {
filename = "${path.module}/pipeline/nightly.json"
content = jsonencode({ schedule = "02:00", source = "landing" })
}terraform {
backend "azurerm" {
use_oidc = true # ARM_CLIENT_ID, ARM_TENANT_ID
use_azuread_auth = true
storage_account_name = "stsalestfstate"
container_name = "tfstate"
key = "sales-data/dev.terraform.tfstate"
}
}terraform {
required_version = "~> 1.17"
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~> 5.11"
}
}
}
provider "azurerm" {
features {}
}terraform {
required_version = "~> 1.17"
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~> 5.11"
}
}
}
provider "azurerm" {
features {}
}# Bootstrap: created once, keeps its own small state
resource "azurerm_storage_account" "tfstate" {
name = "stsalestfstate"
resource_group_name = "rg-sales-tfstate"
location = "eastus2"
account_tier = "Standard"
account_replication_type = "GRS"
shared_access_key_enabled = false
min_tls_version = "TLS1_2"
blob_properties {
versioning_enabled = true
delete_retention_policy {
days = 30
}
container_delete_retention_policy {
days = 30
}
}
}
resource "azurerm_storage_container" "tfstate" {
name = "tfstate"
storage_account_id = azurerm_storage_account.tfstate.id
}variable "pipeline_principal_id" {
type = string
description = "Object ID of the pipeline identity"
}
resource "azurerm_role_assignment" "pipeline_state" {
scope = azurerm_storage_container.tfstate.id
role_definition_name = "Storage Blob Data Contributor"
principal_id = var.pipeline_principal_id
}Question 1: A teammate asks what the state file is and why Terraform needs it. How do you explain it?
State is Terraform's record of what it manages. It maps each address in the code to a real object, with its ID and attributes. Without it, Terraform can't know that the resource group already exists, so it would try to create it again.
Let's do the first apply with terraform apply -auto-approve, and then list what is in the state with terraform state list.
$ terraform apply -auto-approve
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
$ terraform state list
local_file.landing
local_file.pipeline
terraform_data.resource_groupApply says 3 added, and state list shows the three resources Terraform now owns: local_file.landing, local_file.pipeline and terraform_data.resource_group. Now we print one of them with terraform state show.
$ terraform state show terraform_data.resource_group
# terraform_data.resource_group:
resource "terraform_data" "resource_group" {
id = "<uuid>"
input = "rg-sales-data-dev"
output = "rg-sales-data-dev"
}We get its id, its input and its output. The file itself is JSON, so let's read the top fields with jq.
$ jq '{version, terraform_version, serial, lineage}' terraform.tfstate
{
"version": 4,
"terraform_version": "1.17.0",
"serial": 4,
"lineage": "<uuid>"
}version 4 is the format, then terraform_version 1.17.0 wrote it. serial goes up on every write, and lineage identifies this state's history, so Terraform can refuse an older or unrelated copy.
State also tracks dependencies, so a resource removed from the code is destroyed in the right order, and it caches attributes, which helps plans on big configurations.
Never edit state by hand. Use the state commands, or better, moved, import and removed blocks in code, so every change shows up in a plan.
Question 2: Two engineers ran apply at the same time and the state got corrupted. How do remote state and locking prevent that?
With local state, each laptop has its own copy, so two applies start from different versions, and the last write wins. Remote state gives everyone one shared copy, and the lock lets only one write run at a time.
Let's see the lock in action, on a local state file. We add load.tf with a time_sleep that takes 12 seconds, our stand-in for a slow resource like a Databricks workspace. Engineer A runs apply in the background, and 4 seconds later engineer B runs apply.
# Stand-in for a slow resource, like a Databricks workspace
resource "time_sleep" "workspace" {
create_duration = "12s"
}$ terraform apply -auto-approve > engineer-a.log 2>&1 &
$ sleep 4
$ terraform apply -auto-approve
Error: Error acquiring the state lock
Error message: resource temporarily unavailable
Lock Info:
ID: <uuid>
Path: terraform.tfstate
Operation: OperationTypeApply
Who: root@vm
Version: 1.17.0
Created: <timestamp>
Info:
Terraform acquires a state lock to protect the state from being written
by multiple users at the same time. Please resolve the issue above and try
again. For most commands, you can disable locking with the "-lock=false"
flag, but this is not recommended.Engineer B stops right away with Error: Error acquiring the state lock. The lock info shows who holds it and the operation, OperationTypeApply. Meanwhile, engineer A's apply finishes normally.
$ grep -E 'Creation complete|Apply complete' engineer-a.log
time_sleep.workspace: Creation complete after <time> [id=<timestamp>]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.It ends with Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
On Azure, the azurerm backend locks with a lease on the state blob, so the same protection covers every laptop and pipeline. Never use -lock=false to get around it.
If a crashed pipeline leaves a lock behind, first check that nothing is still running. Then run force-unlock with the lock ID.
# only after checking that no run is still active
terraform force-unlock LOCK_IDUse force-unlock only for your own lock, when automatic unlocking failed.
Question 3: Your manager asks where to store state for an Azure data platform. How do you set up the backend?
Use the azurerm backend: the state is a blob in a storage account that holds only state, not data. The backend block in azure/backend.tf names the account, the container and the key.
Let's go step by step:
- First, create that account once, in a small bootstrap config, because the backend has to exist before anything can use it. In
bootstrap/main.tfwe turn on blob versioning and soft delete for blobs and containers, so old versions can be restored, and we turn shared keys off. - Then authenticate with Microsoft Entra ID:
use_azuread_auth, plus OIDC from the pipeline (use_oidc). No access keys, no SAS tokens. - The pipeline identity gets Storage Blob Data Contributor on the state container, nothing more. That is
bootstrap/access.tf. - Use one key per environment and per stack, like
sales-data/dev.terraform.tfstatehere, and keep production state in its own account, with tighter access.
Now we validate both configs with terraform -chdir=bootstrap validate and terraform -chdir=azure validate.
$ terraform -chdir=bootstrap validate
Success! The configuration is valid.
$ terraform -chdir=azure validate
Success! The configuration is valid.Both print Success! The configuration is valid.
Hard-coded values and -backend-config values are saved in the .terraform folder and in plan files, so pass credentials as ARM_* environment variables. Coming from local state? Add the backend block and run terraform init -migrate-state.
Question 4: Someone changed a resource in the portal and Terraform wants to change it back. How do you handle drift?
Drift means the real resource no longer matches the state and the code. Let's make some. Someone edits the nightly pipeline config by hand with sed, and moves the schedule from 02:00 to 03:00. Then we run terraform plan.
$ sed -i 's/02:00/03:00/' pipeline/nightly.json
$ terraform plan
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# local_file.pipeline will be created
+ resource "local_file" "pipeline" {
+ content = jsonencode(
{
+ schedule = "02:00"
+ source = "landing"
}
)
+ content_base64sha256 = (known after apply)
+ content_base64sha512 = (known after apply)
+ content_md5 = (known after apply)
+ content_sha1 = (known after apply)
+ content_sha256 = (known after apply)
+ content_sha512 = (known after apply)
+ directory_permission = "0777"
+ file_permission = "0777"
+ filename = "./pipeline/nightly.json"
+ id = (known after apply)
}
Plan: 1 to add, 0 to change, 0 to destroy.
─────────────────────────────────────────────────────────────────────────────
Note: You didn't use the -out option to save this plan, so Terraform can't
guarantee to take exactly these actions if you run "terraform apply" now.The plan wants to put it back: the code still says "02:00". The local provider treats an edited file as gone, so it plans a new one, 1 to add. In Azure, you'd usually see an update in place.
To see only what changed outside Terraform, we run terraform plan -refresh-only.
$ terraform plan -refresh-only
Note: Objects have changed outside of Terraform
Terraform detected the following changes made outside of Terraform since the
last "terraform apply" which may have affected this plan:
# local_file.pipeline has been deleted
- resource "local_file" "pipeline" {
- content = jsonencode(
{
- schedule = "02:00"
- source = "landing"
}
) -> null
- content_base64sha256 = "rNn88PNVDz7jLR85Eu5WYt9Hxit154snFUmFfz97ZhQ=" -> null
- content_base64sha512 =
"1OYpwR5BloRp2IJz5szyRR2KA98j6goX++u4HyxhkaH/+GSBH/Nq7kQCe7l0XjvyBA2HpzfkrOVze3IoUm5ydQ==" ->
null
- content_md5 = "0f4b04cbce079e66b2bfdcd5a324f4bf" -> null
- content_sha1 = "d4f8fe6f5bc8209fd226aba09372373c696adf23" -> null
- content_sha256 = "acd9fcf0f3550f3ee32d1f3912ee5662df47c62b75e78b271549857f3f7b6614" ->
null
- content_sha512 =
"d4e629c11e41968469d88273e6ccf2451d8a03df23ea0a17fbebb81f2c6191a1fff864811ff36aee44027bb9745e3bf
2040d87a737e4ace5737b7228526e7275" -> null
- directory_permission = "0777" -> null
- file_permission = "0777" -> null
- filename = "./pipeline/nightly.json" -> null
- id = "d4f8fe6f5bc8209fd226aba09372373c696adf23" -> null
}
This is a refresh-only plan, so Terraform will not take any actions to undo
these. If you were expecting these changes then you can apply this plan to
record the updated values in the Terraform state without changing any remote
objects.
─────────────────────────────────────────────────────────────────────────────
Note: You didn't use the -out option to save this plan, so Terraform can't
guarantee to take exactly these actions if you run "terraform apply" now.It says Objects have changed outside of Terraform, the edited file shows as has been deleted, and it takes no action. Applying it would only record reality in the state.
Then we decide:
- The change was wrong - apply: Terraform puts the code version back
- The change was right - put it in the code, so the next plan is clean
- Another system owns it - lifecycle ignore_changes, like a tag from Azure Policy
Here the change was wrong, so we run terraform apply -auto-approve.
$ terraform apply -auto-approve
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.1 added: the file is back with the code version.
I schedule terraform plan -refresh-only -detailed-exitcode. Exit code 2 means drift, so it raises an alert before the next deployment.
Question 5: A resource was renamed in the code and the plan wants to destroy and recreate it. How do you fix that safely?
Terraform tracks resources by address, not by the real name. Let's rename the label from landing to landing_container in main.tf and run terraform plan.
# Local stand-ins: the plan is real, nothing is created in Azure
resource "terraform_data" "resource_group" {
input = "rg-sales-data-dev"
}
resource "local_file" "landing_container" {
filename = "${path.module}/lake/landing.keep"
content = "container in ${terraform_data.resource_group.output}"
}
resource "local_file" "pipeline" {
filename = "${path.module}/pipeline/nightly.json"
content = jsonencode({ schedule = "02:00", source = "landing" })
}$ terraform plan
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
- destroy
Terraform will perform the following actions:
# local_file.landing will be destroyed
# (because local_file.landing is not in configuration)
- resource "local_file" "landing" {
- content = "container in rg-sales-data-dev" -> null
- content_base64sha256 = "PI0DgnFcVGQejQNMt8TsG6Otujfa8YV2K7R3i2xGeCE=" -> null
- content_base64sha512 =
"0yBBsxqFtM+0Jr+9mlcljdpsb7eKs56UE3yRczmhceiZtxLpYB8l/xuR1fXdxrTHL6aJDLV03nRjzlULmnoNyw==" ->
null
- content_md5 = "04464df93bec308e98d31cfce52d608d" -> null
- content_sha1 = "949c4c50edfe68c0936a216a39ca46ef30858262" -> null
- content_sha256 = "3c8d0382715c54641e8d034cb7c4ec1ba3adba37daf185762bb4778b6c467821" ->
null
- content_sha512 =
"d32041b31a85b4cfb426bfbd9a57258dda6c6fb78ab39e94137c917339a171e899b712e9601f25ff1b91d5f5ddc6b4c
72fa6890cb574de7463ce550b9a7a0dcb" -> null
- directory_permission = "0777" -> null
- file_permission = "0777" -> null
- filename = "./lake/landing.keep" -> null
- id = "949c4c50edfe68c0936a216a39ca46ef30858262" -> null
}
# local_file.landing_container will be created
+ resource "local_file" "landing_container" {
+ content = "container in rg-sales-data-dev"
+ content_base64sha256 = (known after apply)
+ content_base64sha512 = (known after apply)
+ content_md5 = (known after apply)
+ content_sha1 = (known after apply)
+ content_sha256 = (known after apply)
+ content_sha512 = (known after apply)
+ directory_permission = "0777"
+ file_permission = "0777"
+ filename = "./lake/landing.keep"
+ id = (known after apply)
}
Plan: 1 to add, 0 to change, 1 to destroy.
─────────────────────────────────────────────────────────────────────────────
Note: You didn't use the -out option to save this plan, so Terraform can't
guarantee to take exactly these actions if you run "terraform apply" now.The old address is gone from the code. So the plan creates local_file.landing_container, and destroys local_file.landing (because local_file.landing is not in configuration). That is 1 to add, 0 to change, 1 to destroy. For a real container, that deletes the data.
The fix is a moved block: from the old address, to the new one. Let's add it, then run plan, apply and state list.
# Local stand-ins: the plan is real, nothing is created in Azure
resource "terraform_data" "resource_group" {
input = "rg-sales-data-dev"
}
resource "local_file" "landing_container" {
filename = "${path.module}/lake/landing.keep"
content = "container in ${terraform_data.resource_group.output}"
}
moved {
from = local_file.landing
to = local_file.landing_container
}
resource "local_file" "pipeline" {
filename = "${path.module}/pipeline/nightly.json"
content = jsonencode({ schedule = "02:00", source = "landing" })
}$ terraform plan
Terraform will perform the following actions:
# local_file.landing has moved to local_file.landing_container
resource "local_file" "landing_container" {
id = "949c4c50edfe68c0936a216a39ca46ef30858262"
# (10 unchanged attributes hidden)
}
Plan: 0 to add, 0 to change, 0 to destroy.
─────────────────────────────────────────────────────────────────────────────
Note: You didn't use the -out option to save this plan, so Terraform can't
guarantee to take exactly these actions if you run "terraform apply" now.
$ terraform apply -auto-approve
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
$ terraform state list
local_file.landing_container
local_file.pipeline
terraform_data.resource_group
time_sleep.workspaceNow the plan says local_file.landing has moved to local_file.landing_container, and Plan: 0 to add, 0 to change, 0 to destroy. Apply records the new address in the state, and nothing changes in the real world. State list shows the new name.
Why a moved block and not terraform state mv? It's code: it shows up in the plan, it's reviewed in the pull request, and it runs the same in every environment, and for every team using a shared module. Moved blocks need Terraform 1.1 or later.
Keep the moved block until every state has moved.
Question 6: Security asks why the state file is treated as sensitive. What is in it, and how do you protect it?
State holds every attribute Terraform knows, including secrets: database passwords, keys and connection strings. Marking a value sensitive only hides it in the output. It's still stored in plain text.
Let's prove it. In sql.tf a terraform_data stands in for an Azure SQL server. The admin password comes from the environment variable TF_VAR_sql_admin_password, and the variable is sensitive = true. We plan to a file, apply it, then search the state and the plan file for the password.
variable "sql_admin_password" {
type = string
sensitive = true
}
# Stand-in for azurerm_mssql_server and its admin password
resource "terraform_data" "sql_server" {
input = {
name = "sql-sales-dev"
admin_login = "sqladmin"
admin_password = var.sql_admin_password
}
}$ terraform plan -out=tfplan
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# terraform_data.sql_server will be created
+ resource "terraform_data" "sql_server" {
+ id = (known after apply)
+ input = {
+ admin_login = "sqladmin"
+ admin_password = (sensitive value)
+ name = "sql-sales-dev"
}
+ output = (known after apply)
}
Plan: 1 to add, 0 to change, 0 to destroy.
─────────────────────────────────────────────────────────────────────────────
Saved the plan to: tfplan
To perform exactly these actions, run the following command to apply:
terraform apply "tfplan"
$ terraform apply tfplan
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
$ grep -n admin_password terraform.tfstate
122: "admin_password": "Lab-only-Passw0rd",
129: "admin_password": "string",
137: "admin_password": "Lab-only-Passw0rd",
144: "admin_password": "string",
160: "value": "admin_password"
$ terraform show -json tfplan | grep -o '"admin_password":"[^"]*"' | sort -u
"admin_password":"Lab-only-Passw0rd"The plan shows admin_password = (sensitive value). But the state file has "admin_password": "Lab-only-Passw0rd" in plain text, twice, and so does the saved plan file.
To keep secrets out of state, use write-only arguments, from Terraform 1.11, fed by ephemeral values. For example, administrator_login_password_wo on azurerm_mssql_server, read from an ephemeral azurerm_key_vault_secret. Terraform sends it to Azure, and never stores it in state.
And protect state like a secret:
- Private state storage - Entra ID: the pipeline and a few admins only
- No shared keys -
shared_access_key_enabled = false - Versioning and soft delete - on the state account
- Never in Git -
*.tfstate, plan files,.terraform/
Read access to state means read access to every secret in it.
Scenario 7: The state file was deleted by accident and the platform is still running. How do you recover?
First, stop. Freeze the pipeline and don't run apply. Before we try this, the full script copies the state to previous-version.tfstate. That copy stands in for the previous blob version that versioning keeps.
Now let's delete the state, then run terraform state list and terraform plan.
$ rm terraform.tfstate terraform.tfstate.backup
$ terraform state list
No state file was found!
State management commands require a state file. Run this command
in a directory where Terraform has been run or use the -state flag
to point the command to a specific state location.
$ terraform plan
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# local_file.landing_container will be created
+ resource "local_file" "landing_container" {
+ content = (known after apply)
+ content_base64sha256 = (known after apply)
+ content_base64sha512 = (known after apply)
+ content_md5 = (known after apply)
+ content_sha1 = (known after apply)
+ content_sha256 = (known after apply)
+ content_sha512 = (known after apply)
+ directory_permission = "0777"
+ file_permission = "0777"
+ filename = "./lake/landing.keep"
+ id = (known after apply)
}
# local_file.pipeline will be created
+ resource "local_file" "pipeline" {
+ content = jsonencode(
{
+ schedule = "02:00"
+ source = "landing"
}
)
+ content_base64sha256 = (known after apply)
+ content_base64sha512 = (known after apply)
+ content_md5 = (known after apply)
+ content_sha1 = (known after apply)
+ content_sha256 = (known after apply)
+ content_sha512 = (known after apply)
+ directory_permission = "0777"
+ file_permission = "0777"
+ filename = "./pipeline/nightly.json"
+ id = (known after apply)
}
# terraform_data.resource_group will be created
+ resource "terraform_data" "resource_group" {
+ id = (known after apply)
+ input = "rg-sales-data-dev"
+ output = (known after apply)
}
# terraform_data.sql_server will be created
+ resource "terraform_data" "sql_server" {
+ id = (known after apply)
+ input = {
+ admin_login = "sqladmin"
+ admin_password = (sensitive value)
+ name = "sql-sales-dev"
}
+ output = (known after apply)
}
# time_sleep.workspace will be created
+ resource "time_sleep" "workspace" {
+ create_duration = "12s"
+ id = (known after apply)
}
Plan: 5 to add, 0 to change, 0 to destroy.
─────────────────────────────────────────────────────────────────────────────
Note: You didn't use the -out option to save this plan, so Terraform can't
guarantee to take exactly these actions if you run "terraform apply" now.State list says No state file was found!, and the plan says Plan: 5 to add, 0 to change, 0 to destroy. Without state, Terraform thinks nothing exists, so it wants to create all five resources, on top of a running platform.
Because blob versioning is on, we restore the previous version of the state blob, in the portal or with state push. Here we run terraform state push previous-version.tfstate, then plan again.
$ terraform state push previous-version.tfstate
$ terraform plan
No changes. Your infrastructure matches the configuration.
Terraform has compared your real infrastructure against your configuration
and found no differences, so no changes are needed.No changes. Your infrastructure matches the configuration. Terraform matches the platform again.
State push is careful: it refuses a copy with an older serial, or a different lineage, than the state already there. Then the rest of the plan:
- No copy at all? - import blocks, for_each for many resources
- Read every plan - until it shows only imports, no creates
- Fix the cause - no delete rights on the state account for people
- Practice the restore - once, before you need it
Keep soft delete and versioning on the state account, and practice the restore once, before you need it.
Key points to remember
- State maps code addresses to real objects: never edit it by hand
- Remote state + locking: one shared copy, one writer at a time
- Azure: a dedicated, versioned account, Entra ID + OIDC, a lease lock
- Drift: refresh-only plan, then revert, codify or ignore_changes
- moved blocks for renames; state is a secret; restore a blob version
The full script
This is a Python script that writes every file and runs every command above in order: put Terraform 1.17 on your PATH and run it in an empty folder.
Show the full script that writes the files and runs every step
# ========================================================================
# Terraform Interview Q&A #2 - State and remote backends
# TechBrothersIT. Names and sample values are invented for teaching.
# Every step below was run with Terraform 1.17.0; the outputs in the video and the blog post are the real outputs.
# Providers used: hashicorp/random, null, local, time (they create nothing in any cloud). Azure and Databricks
# code in this series is checked with terraform validate only: nothing is created in a real subscription.
# To run it yourself (Linux, macOS or WSL; some steps use grep / cat):
# 1. Install Terraform 1.17 from developer.hashicorp.com/terraform/install and put it on your PATH.
# 2. python Terraform_Interview_QA_02_Code.py in an empty folder (terraform init downloads the providers).
# ========================================================================
import os, subprocess
def write(path, text):
if text is None:
if os.path.exists(path): os.remove(path)
return
os.makedirs(os.path.dirname(path) or ".", exist_ok=True)
with open(path, "w", encoding="utf-8") as f:
f.write(text.strip("\n") + "\n")
def run(cmd, expect_error=False, allow_error=False, env=None):
print("\n$ " + cmd, flush=True)
rc = subprocess.run(cmd, shell=True, env=dict(os.environ, **(env or {}))).returncode
if rc != 0 and not (expect_error or allow_error):
raise SystemExit("stopped: this command was expected to succeed")
# Environment used by every step
os.environ.update({'TF_VAR_sql_admin_password': 'Lab-only-Passw0rd'})
# Step 0: the starting files
write('versions.tf', """\
terraform {
required_version = "~> 1.17"
required_providers {
local = {
source = "hashicorp/local"
version = "~> 2.9"
}
time = {
source = "hashicorp/time"
version = "~> 0.14"
}
}
}
""")
write('main.tf', """\
# Local stand-ins: the plan is real, nothing is created in Azure
resource "terraform_data" "resource_group" {
input = "rg-sales-data-dev"
}
resource "local_file" "landing" {
filename = "${path.module}/lake/landing.keep"
content = "container in ${terraform_data.resource_group.output}"
}
resource "local_file" "pipeline" {
filename = "${path.module}/pipeline/nightly.json"
content = jsonencode({ schedule = "02:00", source = "landing" })
}
""")
write('azure/backend.tf', """\
terraform {
backend "azurerm" {
use_oidc = true # ARM_CLIENT_ID, ARM_TENANT_ID
use_azuread_auth = true
storage_account_name = "stsalestfstate"
container_name = "tfstate"
key = "sales-data/dev.terraform.tfstate"
}
}
""")
write('azure/versions.tf', """\
terraform {
required_version = "~> 1.17"
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~> 5.11"
}
}
}
provider "azurerm" {
features {}
}
""")
write('bootstrap/versions.tf', """\
terraform {
required_version = "~> 1.17"
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~> 5.11"
}
}
}
provider "azurerm" {
features {}
}
""")
write('bootstrap/main.tf', """\
# Bootstrap: created once, keeps its own small state
resource "azurerm_storage_account" "tfstate" {
name = "stsalestfstate"
resource_group_name = "rg-sales-tfstate"
location = "eastus2"
account_tier = "Standard"
account_replication_type = "GRS"
shared_access_key_enabled = false
min_tls_version = "TLS1_2"
blob_properties {
versioning_enabled = true
delete_retention_policy {
days = 30
}
container_delete_retention_policy {
days = 30
}
}
}
resource "azurerm_storage_container" "tfstate" {
name = "tfstate"
storage_account_id = azurerm_storage_account.tfstate.id
}
""")
write('bootstrap/access.tf', """\
variable "pipeline_principal_id" {
type = string
description = "Object ID of the pipeline identity"
}
resource "azurerm_role_assignment" "pipeline_state" {
scope = azurerm_storage_container.tfstate.id
role_definition_name = "Storage Blob Data Contributor"
principal_id = var.pipeline_principal_id
}
""")
run('terraform fmt -check -recursive')
run('terraform init')
run('terraform -chdir=azure init -backend=false')
run('terraform -chdir=bootstrap init')
# Step 1: First apply, then what the state now holds
run("terraform apply -auto-approve > tf.log 2>&1; rc=$?; grep -E 'Apply complete' tf.log; exit $rc")
run('terraform state list')
# Step 2: One resource in the state
run('terraform state show terraform_data.resource_group')
# Step 3: The state file is JSON
run("jq '{version, terraform_version, serial, lineage}' terraform.tfstate")
# Step 4: Engineer A applies a slow resource; engineer B applies 4 seconds later
write('load.tf', """\
# Stand-in for a slow resource, like a Databricks workspace
resource "time_sleep" "workspace" {
create_duration = "12s"
}
""")
run('terraform apply -auto-approve > engineer-a.log 2>&1 &')
run('sleep 4')
run("terraform apply -auto-approve > tf.log 2>&1; rc=$?; sed -E 's/^( Created: +).*/\\1<timestamp>/' tf.log; exit $rc", expect_error=True) # EXPECTED TO FAIL: the video shows this error
# Step 5: Engineer A's apply finishes normally
run("until grep -q 'Apply complete' engineer-a.log; do sleep 1; done; grep -E 'Creation complete|Apply complete' engineer-a.log")
# Step 6: Validate the state storage (bootstrap) and the backend block
run('terraform -chdir=bootstrap validate')
run('terraform -chdir=azure validate')
# Step 7: Someone edits the pipeline config by hand, then plan
run("sed -i 's/02:00/03:00/' pipeline/nightly.json")
run("terraform plan > tf.log 2>&1; rc=$?; grep -v 'Refreshing state' tf.log; exit $rc")
# Step 8: refresh-only: what changed outside Terraform
run("terraform plan -refresh-only > tf.log 2>&1; rc=$?; grep -v 'Refreshing state' tf.log; exit $rc")
# Step 9: The hand change was wrong: apply puts the code version back
run("terraform apply -auto-approve > tf.log 2>&1; rc=$?; grep -E 'Apply complete' tf.log; exit $rc")
# Step 10: Rename the resource label in code
write('main.tf', """\
# Local stand-ins: the plan is real, nothing is created in Azure
resource "terraform_data" "resource_group" {
input = "rg-sales-data-dev"
}
resource "local_file" "landing_container" {
filename = "${path.module}/lake/landing.keep"
content = "container in ${terraform_data.resource_group.output}"
}
resource "local_file" "pipeline" {
filename = "${path.module}/pipeline/nightly.json"
content = jsonencode({ schedule = "02:00", source = "landing" })
}
""")
run("terraform plan > tf.log 2>&1; rc=$?; grep -v 'Refreshing state' tf.log; exit $rc")
# Step 11: Add a moved block, plan, apply
write('main.tf', """\
# Local stand-ins: the plan is real, nothing is created in Azure
resource "terraform_data" "resource_group" {
input = "rg-sales-data-dev"
}
resource "local_file" "landing_container" {
filename = "${path.module}/lake/landing.keep"
content = "container in ${terraform_data.resource_group.output}"
}
moved {
from = local_file.landing
to = local_file.landing_container
}
resource "local_file" "pipeline" {
filename = "${path.module}/pipeline/nightly.json"
content = jsonencode({ schedule = "02:00", source = "landing" })
}
""")
run("terraform plan > tf.log 2>&1; rc=$?; grep -v 'Refreshing state' tf.log; exit $rc")
run("terraform apply -auto-approve > tf.log 2>&1; rc=$?; grep -E 'Apply complete' tf.log; exit $rc")
run('terraform state list')
# Step 12: A sensitive password: hidden in the plan, plain text in the state and the plan file
write('sql.tf', """\
variable "sql_admin_password" {
type = string
sensitive = true
}
# Stand-in for azurerm_mssql_server and its admin password
resource "terraform_data" "sql_server" {
input = {
name = "sql-sales-dev"
admin_login = "sqladmin"
admin_password = var.sql_admin_password
}
}
""")
run("terraform plan -out=tfplan > tf.log 2>&1; rc=$?; grep -v 'Refreshing state' tf.log; exit $rc")
run("terraform apply tfplan > tf.log 2>&1; rc=$?; grep -E 'Apply complete' tf.log; exit $rc")
run('grep -n admin_password terraform.tfstate')
run('terraform show -json tfplan | grep -o \'"admin_password":"[^"]*"\' | sort -u')
# Step 13: (stands in for the previous blob version that versioning keeps)
run('cp terraform.tfstate previous-version.tfstate')
# Step 14: The state is deleted: Terraform thinks nothing exists
run('rm terraform.tfstate terraform.tfstate.backup', allow_error=True)
run('terraform state list', allow_error=True)
run("terraform plan > tf.log 2>&1; rc=$?; grep -v 'Refreshing state' tf.log; exit $rc", allow_error=True)
# Step 15: Restore the previous version with state push, then plan
run('terraform state push previous-version.tfstate')
run("terraform plan > tf.log 2>&1; rc=$?; grep -v 'Refreshing state' tf.log; exit $rc")
Part 3: Variables, Outputs and Expressions
The sample data on this page is invented for teaching. Every terminal output is the real output of the command above it, run with Terraform 1.17.0; IDs, timestamps and durations are masked. Azure and Databricks code is checked with terraform validate. Part of the Real Time Terraform Interview Questions and Answers series on TechBrothersIT. Features, limits and their status (GA or preview) are current as of October 2026; check the official Terraform documentation (developer.hashicorp.com/terraform) for anything you quote in an interview.
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.