In this post we are going to cover deployment, CI and slim CI in dbt, through 7 real dbt interview questions for data engineers, asked the way they come up on a real project. The last one is a scenario: you must schedule dbt in production with Airflow, with alerts and retries.
Deployment is how a change gets from your laptop to the production tables, and CI is the check that runs on every pull request before it gets there. Slim CI is the version that builds only what changed. Interviewers ask about this because it is where real teams lose hours: CI jobs that run the whole project, failed nightly runs, and changes that skip review.
Part 8 of the Real Time dbt Interview Questions and Answers series on TechBrothersIT.
- The short answer
- Let's create the project and some sample data
- Question 1: Every pull request runs the full project and CI takes an hour. How does slim CI with state:modified work?
- Question 2: A teammate asks what defer does and why CI does not rebuild unchanged parents. How do you explain it?
- Question 3: A production run failed halfway through. How do you rerun only what failed?
- Question 4: Your manager asks how you promote dbt changes from dev to prod safely. What does your deployment flow look like?
- Question 5: A teammate asks what manifest.json and run_results.json are for. What do you tell them?
- Question 6: You need a fast copy of production tables to test a change. When do you use dbt clone?
- Scenario 7: You must schedule dbt in production with Airflow, alerts and retries. How do you set it up?
- The full script
The short answer
Slim CI builds state:modified+ against the production manifest and uses --defer, so unchanged parents are read from prod instead of rebuilt. dbt retry reruns only failed and skipped nodes from run_results.json, dbt clone gives you zero-copy clones or pointer views of prod, and code reaches prod only through pull request, CI and merge.
Let's create the project and some sample data
Our project is a small shop: raw orders and customers, two staging views, and three mart tables, fct_orders, dim_customers and rpt_daily_revenue. The profile has three targets in the same DuckDB file: dev writes to dbt_ali, ci writes to ci_pr_42 (the schema for pull request 42), and prod writes to analytics. Here are all the starting files.
name: shop
version: '1.0'
profile: shop
models:
shop:
staging:
+materialized: view
marts:
+materialized: tableshop:
target: dev
outputs:
dev:
type: duckdb
path: warehouse.duckdb
schema: dbt_ali
threads: 4
ci:
type: duckdb
path: warehouse.duckdb
schema: ci_pr_42
threads: 4
prod:
type: duckdb
path: warehouse.duckdb
schema: analytics
threads: 4{% macro load_raw(late_duplicate=false) %}
{# stands in for the loading tool: lands raw data in the raw schema #}
{% do run_query("create schema if not exists raw") %}
{% do run_query("create or replace table raw.customers as select * from (values
(10, 'Ali Khan', 'US'), (11, 'Sara Lee', 'CA'), (12, 'John Smith', 'US'))
as t(customer_id, customer_name, country)") %}
{% set dup = ", (3, 10, date '2026-10-24', 20.00, 'placed')" if late_duplicate else "" %}
{% do run_query("create or replace table raw.orders as select * from (values
(1, 10, date '2026-10-23', 100.00, 'placed'),
(2, 11, date '2026-10-23', 50.50, 'cancelled'),
(3, 10, date '2026-10-24', 20.00, 'placed'),
(4, 12, date '2026-10-24', 75.00, 'placed')" ~ dup ~ ")
as t(order_id, customer_id, order_date, amount, status)") %}
{% endmacro %}sources:
- name: raw
schema: raw
tables:
- name: orders
- name: customersselect order_id, customer_id, order_date, amount, status
from {{ source('raw', 'orders') }}select customer_id, customer_name, country
from {{ source('raw', 'customers') }}models:
- name: stg_orders
columns:
- name: order_id
data_tests: [unique, not_null]select o.order_id, o.order_date, c.country, o.amount, o.status
from {{ ref('stg_orders') }} as o
join {{ ref('stg_customers') }} as c using (customer_id)select customer_id, customer_name, country
from {{ ref('stg_customers') }}select order_date, sum(amount) as revenue
from {{ ref('fct_orders') }}
group by order_datemodels:
- name: rpt_daily_revenue
columns:
- name: order_date
data_tests: [unique]In a real project a loading tool lands the raw data. Here the load_raw macro stands in for it, so first we run dbt run-operation load_raw.
$ dbt run-operation load_raw
Finished 'run-operation' successfully for target 'dev'Question 1: Every pull request runs the full project and CI takes an hour. How does slim CI with state:modified work?
Slim CI builds only what the pull request changed, plus everything downstream. To know what changed, dbt compares the project with the production manifest.
So production has to save its artifacts first. Let's run the prod build with dbt build --target prod --target-path prod_state, which writes them to a folder called prod_state.
$ dbt build --target prod --target-path prod_state
Succeeded model analytics.stg_customers (view)
Succeeded model analytics.stg_orders (view)
Passed test not_null_stg_orders_order_id
Passed test unique_stg_orders_order_id
Succeeded model analytics.dim_customers (table)
Succeeded model analytics.fct_orders (table)
Succeeded model analytics.rpt_daily_revenue (table)
Passed test unique_rpt_daily_revenue_order_date
Finished 'build' successfully for target 'prod'
Processed: 5 models | 3 tests
Summary: 8 total | 8 successProd builds everything: 8 total | 8 success. Now a pull request changes one model: fct_orders drops cancelled orders. Let's ask dbt what is modified with dbt ls -s state:modified+ --state prod_state.
select o.order_id, o.order_date, c.country, o.amount, o.status
from {{ ref('stg_orders') }} as o
join {{ ref('stg_customers') }} as c using (customer_id)
where o.status <> 'cancelled'$ dbt ls -s state:modified+ --state prod_state
=======================Selected nodes=======================
shop.marts.fct_orders
shop.marts.rpt_daily_revenue
shop.marts.unique_rpt_daily_revenue_order_date
Finished 'list' successfully for target 'dev'dbt lists the changed model fct_orders, the report downstream rpt_daily_revenue, and the report's test unique_rpt_daily_revenue_order_date. Nothing else. Now the CI job builds that selection with --defer, into the ci target.
$ dbt build -s state:modified+ --defer --state prod_state --target ci
Succeeded model ci_pr_42.fct_orders (table)
Succeeded model ci_pr_42.rpt_daily_revenue (table)
Passed test unique_rpt_daily_revenue_order_date
Finished 'build' successfully for target 'ci'
Processed: 2 models | 1 test
Summary: 3 total | 3 successTwo models and one test, 3 total | 3 success, in ci_pr_42, instead of the whole project. Staging and dim_customers are not rebuilt.
In a real project, CI downloads the latest prod manifest before every run. On the dbt platform, CI jobs do this for you: they defer to production, and drop the temporary schema when the pull request closes.
The + matters. state:modified+ means changed plus downstream. Without the plus, you test the changed model, but not the reports that read it.
Question 2: A teammate asks what defer does and why CI does not rebuild unchanged parents. How do you explain it?
With --defer, a ref to a model you didn't select, and that doesn't exist in your schema, resolves to the relation in the state manifest. In CI, that is production.
Let's see why we need it. We build just the changed model in the pull request schema, without defer, with dbt build -s fct_orders --target ci.
$ dbt build -s fct_orders --target ci
Failed model ci_pr_42.fct_orders (table)
=================== Errors and Warnings ====================
[error] [DbDriverFailed (dbt1308)]: Database Error in model fct_orders
(target/run/shop/models/marts/fct_orders.sql)
Catalog Error: Table with name stg_orders does not exist!
Did you mean "analytics.stg_orders"?
LINE 11: from "warehouse"."ci_pr_42"."stg_orders" as o
^
(in run/shop/models/marts/fct_orders.sql)
Finished 'build' with 1 error for target 'ci'
Processed: 1 model
Summary: 1 total | 1 errorIt fails: Table with name stg_orders does not exist! The parent is not in ci_pr_42. DuckDB even asks Did you mean "analytics.stg_orders"?
Now let's compile the same model both ways: first without defer, then with --defer --state prod_state.
$ dbt compile -s fct_orders --target ci
Compiled node 'fct_orders' is:
select o.order_id, o.order_date, c.country, o.amount, o.status
from "warehouse"."ci_pr_42"."stg_orders" as o
join "warehouse"."ci_pr_42"."stg_customers" as c using (customer_id)
where o.status <> 'cancelled'
Finished 'compile' successfully for target 'ci'
Processed: 1 model
Summary: 1 total | 1 success
$ dbt compile -s fct_orders --defer --state prod_state --target ci
Compiled node 'fct_orders' is:
select o.order_id, o.order_date, c.country, o.amount, o.status
from "warehouse"."analytics"."stg_orders" as o
join "warehouse"."analytics"."stg_customers" as c using (customer_id)
where o.status <> 'cancelled'
Finished 'compile' successfully for target 'ci'
Processed: 1 model
Summary: 1 total | 1 successWithout defer, both refs point at "warehouse"."ci_pr_42". With defer and the prod state, they point at "warehouse"."analytics". The model itself is still built in the pull request schema.
So CI reads unchanged parents straight from prod: no rebuild, and real data. Two more details:
- With
--favor-state, dbt prefers the prod relation even when your schema has one - Ephemeral models are never deferred
Defer reads prod data, so the CI role needs read access to prod. But it writes only to its own schema.
Question 3: A production run failed halfway through. How do you rerun only what failed?
With dbt retry. It reads run_results.json from the last command and reruns only the nodes that failed, or were skipped because of a failure.
Let's break the nightly build. First the source sends a duplicate order, so we run the loader with late_duplicate: true.
$ dbt run-operation load_raw --args '{late_duplicate: true}'
Finished 'run-operation' successfully for target 'dev'Now the nightly prod build, dbt build --target prod.
$ dbt build --target prod
Succeeded model analytics.stg_customers (view)
Succeeded model analytics.stg_orders (view)
Passed test not_null_stg_orders_order_id
Failed test unique_stg_orders_order_id (models/staging/_staging.yml:5:22)
Succeeded model analytics.dim_customers (table)
Skipped model analytics.fct_orders (table)
Skipped model analytics.rpt_daily_revenue (table)
Skipped test unique_rpt_daily_revenue_order_date (models/marts/_marts.yml:5:22)
====================== Test Failures =======================
Test failed (1 failed row(s)): unique_stg_orders_order_id
Finished 'build' with 1 error for target 'prod'
Processed: 5 models | 3 tests
Summary: 8 total | 4 success | 1 error | 3 skippedThe test unique_stg_orders_order_id failed with 1 failed row, so dbt skipped everything downstream: fct_orders, rpt_daily_revenue and its test. The other nodes succeeded: 8 total | 4 success | 1 error | 3 skipped.
Fix the cause first. Here the source resends the day without the duplicate. Notice the loader writes to its own --target-path loader_target, so the failed build's run_results.json stays in place.
$ dbt run-operation load_raw --target-path loader_target
Finished 'run-operation' successfully for target 'dev'Then we run dbt retry --target prod, with the same target as the failed build.
$ dbt retry --target prod
Retrying 4 nodes from previous build command
Passed test unique_stg_orders_order_id
Succeeded model analytics.fct_orders (table)
Succeeded model analytics.rpt_daily_revenue (table)
Passed test unique_rpt_daily_revenue_order_date
Finished 'retry' successfully for target 'prod'dbt says Retrying 4 nodes from previous build command: the failed test passes, then the two skipped models and the skipped test succeed. Nothing that succeeded runs again.
Retry reads the last run_results.json, so a build, compile or run-operation in between replaces it and leaves nothing to retry. Retry right after the fix, from the same folder. And for a bug in the code, the fix goes through CI first: retry is for the rerun, not for skipping review.
Question 4: Your manager asks how you promote dbt changes from dev to prod safely. What does your deployment flow look like?
Code only reaches prod through Git, in four stages: dev, pull request, merge, prod job.
First, dev: each developer builds in a personal schema, dbt_ali, with the dev target.
Second, the pull request triggers slim CI. We build the modified models and everything downstream, deferring to prod, in a schema for that pull request. Tests must pass, and a teammate reviews the code.
Third, merge to main. That is the only way in.
Fourth, the prod job runs dbt build --target prod, on its schedule or right after the merge, and saves its artifacts. They become the state for the next CI run.
Here is what the pull request stage looks like as a GitHub Actions workflow:
on: pull_request
jobs:
slim-ci:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/setup-python@v7
with: {python-version: '3.13'}
- run: pip install dbt==2.0.9
- run: ./get_prod_artifacts.sh prod_state
- run: >-
dbt build -s state:modified+ --defer
--state prod_state --target ciOn the dbt platform, these pieces are environments, CI jobs, deploy jobs and merge jobs. Anywhere else, it's the same flow, with a CI runner and your scheduler. And a few rules keep prod safe:
- Prod credentials only in the prod job
- No dbt runs against prod from a laptop
- Rollback = git revert + prod run, plus a full refresh if an incremental model wrote bad rows
Say "dev, pull request with slim CI, merge, prod job". Four stages, and Git is the only way in.
Question 5: A teammate asks what manifest.json and run_results.json are for. What do you tell them?
dbt writes JSON artifacts to the target folder. manifest.json is the full graph of the project, and run_results.json is the result of the last build or run.
| What is in it | Who reads it | |
|---|---|---|
| manifest.json | every node: config, dependencies, compiled code | state:, --defer, docs, lineage tools |
| run_results.json | last build or run: status, timing, message | dbt retry, result:, alerts |
The manifest has every node, with its config, its dependencies and its compiled code. State selection and defer compare against it, and docs and lineage tools read it. Run results has one entry per executed node: its status, timing and message. dbt retry and the result: selectors read it.
Let's see both in action. We ran these two commands right after the failed prod build in Question 3, before the fix and the retry: state:modified against the prod manifest, then result:fail against the target folder of the failed run.
$ dbt ls -s state:modified --state prod_state
=======================Selected nodes=======================
shop.marts.fct_orders
Finished 'list' successfully for target 'dev'
$ dbt ls -s result:fail --state target
=======================Selected nodes=======================
shop.staging.unique_stg_orders_order_id
Finished 'list' successfully for target 'dev'The manifest tells us what changed: shop.marts.fct_orders, the model the pull request changed. Run results tells us what failed: shop.staging.unique_stg_orders_order_id, the unique test.
dbt v2 can also write Parquet artifacts, and still writes the JSON for compatibility.
Upload both after every prod run. The manifest becomes the CI state, and run results feed alerts and run history.
Question 6: You need a fast copy of production tables to test a change. When do you use dbt clone?
dbt clone copies the selected models from a state manifest into your target schema, without running their SQL.
On Snowflake, Databricks and BigQuery, it is a zero-copy clone: no data is copied, so it is fast. On other platforms, dbt creates pointer views to the prod relations. And it leaves relations that already exist alone, unless you pass --full-refresh.
My main use, on a platform with zero-copy clones, is CI for incremental models. First clone the prod version of the modified incremental models, then build. Now the change runs as a real incremental run, on top of prod data, not as a first full build:
# CI: clone the modified incremental models from prod first
dbt clone --state prod_state --target ci \
-s state:modified+,config.materialized:incremental,state:old
# then build them incrementally, on top of the prod data
dbt build -s state:modified+ --defer --state prod_state --target ciOther uses:
- A developer sandbox with prod tables
- Blue/green deployments, where you build and test in one schema, then swap
So what is the difference from defer? Defer reads prod in place, while clone creates objects in your own schema, which you can build on, change and drop.
| Defer | Clone | |
|---|---|---|
| What | reads prod in place | creates objects in your schema |
| Use for | unchanged parents in CI | incremental CI, sandboxes, blue/green |
If they ask "defer or clone?", answer "defer for unchanged parents, clone when the CI run needs prod tables in its own schema".
Scenario 7: You must schedule dbt in production with Airflow, alerts and retries. How do you set it up?
Keep it to one small DAG: source freshness, then dbt build --target prod, with dbt retry on the retries, an alert after the last try, and the artifacts uploaded after every run.
First, who runs dbt? On the dbt platform, Airflow triggers a platform job with the dbt Cloud run job operator. Here, Airflow runs dbt itself, with the dbt version pinned in the worker image. In Airflow 3, the DAG comes from the task SDK, and the bash operator from the standard provider.
from datetime import timedelta
from airflow.providers.standard.operators.bash import BashOperator
from airflow.sdk import DAG
from alerts import alert_on_call # posts to Slack or on-call
DBT = "cd /opt/dbt/shop && "
BUILD = "{{ 'dbt retry' if ti.try_number > 1 else 'dbt build' }}"
with DAG("dbt_prod", schedule="0 5 * * *", catchup=False,
default_args={"retries": 2,
"retry_delay": timedelta(minutes=5),
"on_failure_callback": alert_on_call}):
fresh = BashOperator(task_id="source_freshness",
bash_command=DBT + "dbt source freshness --target prod")
build = BashOperator(task_id="dbt_build",
bash_command=DBT + BUILD + " --target prod")
fresh >> buildLet's walk through it:
- Pin the dbt version in the image
- Freshness first, then build
- Retry = dbt retry: from the second try, the template runs
dbt retry, so only failed and skipped nodes rerun. It needs the first try's target folder: same worker, or a shared volume - Alert after the last try: the failure callback posts to Slack or the on-call tool
- Upload manifest + run_results: they are your slim CI state and your run history
One task keeps it simple. For a task per model, Astronomer Cosmos renders a dbt project as Airflow tasks; check its support for your dbt version.
If the first try failed before any node ran, dbt retry has nothing to rerun and can still pass. Check run_results.json, and rerun the full build.
Key points to remember
- Slim CI: state:modified+ against the prod manifest
- --defer: unchanged parents are read from prod
- dbt retry: only failed and skipped nodes, from run_results.json
- Code reaches prod only through PR, CI and merge
- dbt clone: zero-copy clones or pointer views of prod
The full script
This Python script writes the whole project and runs every dbt command in order; run pip install dbt first.
Show the full script that builds this project and runs every step
# ========================================================================
# dbt Interview Q&A #8 - Deployment, CI and slim CI
# TechBrothersIT. Sample data is invented for teaching.
# Every step below was run with dbt v2 (dbt OSS 2.0.8) on DuckDB; the outputs in the video and the
# blog post are the real outputs. To run it yourself, in an empty folder:
# pip install dbt (dbt v2; adapters are bundled, and DuckDB needs no account or server)
# python dbt_Interview_QA_08_Code.py
# ========================================================================
import os, subprocess
def write(path, text):
os.makedirs(os.path.dirname(path) or ".", exist_ok=True)
with open(path, "w", encoding="utf-8") as f:
f.write(text.strip("\n") + "\n")
def remove(path):
if os.path.exists(path):
os.remove(path)
def dbt(cmd, expect_error=False):
print("\n$ " + cmd, flush=True)
rc = subprocess.run(cmd, shell=True).returncode
if rc != 0 and not expect_error:
raise SystemExit("stopped: this command was expected to succeed")
# Step 0: the project and its sample data
write('dbt_project.yml', """\
name: shop
version: '1.0'
profile: shop
models:
shop:
staging:
+materialized: view
marts:
+materialized: table
""")
write('profiles.yml', """\
shop:
target: dev
outputs:
dev:
type: duckdb
path: warehouse.duckdb
schema: dbt_ali
threads: 4
ci:
type: duckdb
path: warehouse.duckdb
schema: ci_pr_42
threads: 4
prod:
type: duckdb
path: warehouse.duckdb
schema: analytics
threads: 4
""")
write('macros/load_raw.sql', """\
{% macro load_raw(late_duplicate=false) %}
{# stands in for the loading tool: lands raw data in the raw schema #}
{% do run_query("create schema if not exists raw") %}
{% do run_query("create or replace table raw.customers as select * from (values
(10, 'Ali Khan', 'US'), (11, 'Sara Lee', 'CA'), (12, 'John Smith', 'US'))
as t(customer_id, customer_name, country)") %}
{% set dup = ", (3, 10, date '2026-10-24', 20.00, 'placed')" if late_duplicate else "" %}
{% do run_query("create or replace table raw.orders as select * from (values
(1, 10, date '2026-10-23', 100.00, 'placed'),
(2, 11, date '2026-10-23', 50.50, 'cancelled'),
(3, 10, date '2026-10-24', 20.00, 'placed'),
(4, 12, date '2026-10-24', 75.00, 'placed')" ~ dup ~ ")
as t(order_id, customer_id, order_date, amount, status)") %}
{% endmacro %}
""")
write('models/staging/_sources.yml', """\
sources:
- name: raw
schema: raw
tables:
- name: orders
- name: customers
""")
write('models/staging/stg_orders.sql', """\
select order_id, customer_id, order_date, amount, status
from {{ source('raw', 'orders') }}
""")
write('models/staging/stg_customers.sql', """\
select customer_id, customer_name, country
from {{ source('raw', 'customers') }}
""")
write('models/staging/_staging.yml', """\
models:
- name: stg_orders
columns:
- name: order_id
data_tests: [unique, not_null]
""")
write('models/marts/fct_orders.sql', """\
select o.order_id, o.order_date, c.country, o.amount, o.status
from {{ ref('stg_orders') }} as o
join {{ ref('stg_customers') }} as c using (customer_id)
""")
write('models/marts/dim_customers.sql', """\
select customer_id, customer_name, country
from {{ ref('stg_customers') }}
""")
write('models/marts/rpt_daily_revenue.sql', """\
select order_date, sum(amount) as revenue
from {{ ref('fct_orders') }}
group by order_date
""")
write('models/marts/_marts.yml', """\
models:
- name: rpt_daily_revenue
columns:
- name: order_date
data_tests: [unique]
""")
# Step 1: The loading tool lands raw orders and customers (a macro stands in for it)
dbt('dbt run-operation load_raw')
# Step 2: Production build: its artifacts go to the prod_state folder
dbt('dbt build --target prod --target-path prod_state')
# Step 3: A pull request changes fct_orders: what is modified?
write('models/marts/fct_orders.sql', """\
select o.order_id, o.order_date, c.country, o.amount, o.status
from {{ ref('stg_orders') }} as o
join {{ ref('stg_customers') }} as c using (customer_id)
where o.status <> 'cancelled'
""")
dbt('dbt ls -s state:modified+ --state prod_state')
# Step 4: Slim CI: build the modified models and their children, deferring to prod
dbt('dbt build -s state:modified+ --defer --state prod_state --target ci')
# Step 5: Without defer, the unchanged parent is missing from the CI schema
dbt('dbt build -s fct_orders --target ci', expect_error=True) # EXPECTED TO FAIL: the video shows this error
# Step 6: Compile with and without defer
dbt('dbt compile -s fct_orders --target ci')
dbt('dbt compile -s fct_orders --defer --state prod_state --target ci')
# Step 7: The source sends a duplicate order
dbt("dbt run-operation load_raw --args '{late_duplicate: true}'")
# Step 8: The nightly prod build fails halfway
dbt('dbt build --target prod', expect_error=True) # EXPECTED TO FAIL: the video shows this error
# Step 9: What the artifacts know: state from the manifest, results from run_results
dbt('dbt ls -s state:modified --state prod_state')
dbt('dbt ls -s result:fail --state target')
# Step 10: The source resends the day without the duplicate (the loader writes to its own target path, so run_results.json of the failed build stays)
dbt('dbt run-operation load_raw --target-path loader_target')
# Step 11: dbt retry: only the failed and skipped nodes
dbt('dbt retry --target prod')
Part 9: Governance, Performance and dbt v2
The sample data on this page is invented for teaching. Every terminal output is the real output of the dbt command above it, run with dbt v2 (dbt OSS 2.0.8) on DuckDB; run times and node counters are left out, and node lines are listed in DAG order. Part of the Real Time dbt Interview Questions and Answers series on TechBrothersIT. Features, limits and their status (GA or preview) are current as of October 2026; check the official dbt documentation (docs.getdbt.com) for anything you quote in an interview.



No comments:
Post a Comment
Note: Only a member of this blog may post a comment.