In this post we are going to cover real-world manager scenarios in Azure Data Factory: the questions a hiring manager asks to see how you handle an incident, a cost spike, a slow copy, a Fabric decision and a factory nobody documented. These are 7 real Azure Data Factory interview questions, asked the way they come up on a real project, and the last one is a scenario where the hiring manager asks you to design a daily lakehouse load from ten source systems.
This is the last part of the series, and it is different from the others. There is no single feature to explain. The manager wants to hear your order of steps: what you do first, what you check, and what you tell the business. Interviewers ask these in the final round because they show whether you have actually run ADF in production.
Part 10 of the Real Time Azure Data Factory Interview Questions and Answers series on TechBrothersIT.
- The short answer
- Question 1: The nightly pipeline failed at 2 AM and the CFO's report is due at 8. Walk me through your response.
- Question 2: The ADF bill doubled this month. How do you find out why and reduce it?
- Question 3: A copy from an on-premises Oracle database got ten times slower after a network change. How do you troubleshoot it?
- Question 4: Leadership asks whether we should move our ADF pipelines to Fabric Data Factory. How do you assess it, and what doesn't move?
- Question 5: You inherit a factory with three hundred pipelines and no documentation. What do you do in your first two weeks?
- Question 6: A source API returns data in pages and sometimes rate-limits you. How do you load it reliably in ADF?
- Scenario 7: In the final round, the hiring manager asks you to design a daily lakehouse load with ADF from ten source systems. What is your design?
The short answer
In an incident, communicate first, then triage, rerun and validate. Know the cost meters, troubleshoot slow copies stage by stage, and assess Fabric with the preview tooling, knowing what doesn't move. Design loads that are metadata-driven and safe to rerun.
Keep that in mind. Every question below is one piece of it.
Question 1: The nightly pipeline failed at 2 AM and the CFO's report is due at 8. Walk me through your response.
Communicate first, then fix. Silence is worse than bad news. The pipeline failed at 2 AM, the report is due at 8 AM, so the clock matters, and the business needs to hear from you before they find out on their own.
Here is my response, in order:
- Acknowledge. Within minutes, tell the finance contact the report may be late, and when the next update comes.
- Triage in Monitor. Find the failed activity and read the error code and message. A configuration problem, like an expired secret, a changed column or an offline self-hosted runtime, is mine to fix. A transient one, like a timeout, may only need a retry.
- Fix, then rerun from failed activity. That way the finished steps aren't repeated. If the full fix takes longer, agree a workaround with the business, like loading only the tables the report needs.
- Validate. Before I say done: row counts against the source, the downstream model refreshed, and the report totals checked. Then I confirm with finance.
- Post-mortem. A short one: the root cause, and one change, like a retry or a data check, so it doesn't happen again.
That you told finance before you started debugging, and that you validated before you said done.
Question 2: The ADF bill doubled this month. How do you find out why and reduce it?
First, know the meters. You can't find the driver if you don't know what is being charged:
| Meter | Charged by | Usual cause |
|---|---|---|
| ORCHESTRATION | per 1,000 activity, trigger and debug runs | ForEach of tiny activities, frequent trigger |
| DATA MOVEMENT | DIU-hours | too many DIUs on small files |
| PIPELINE, EXTERNAL | hours of execution | long waits, many lookups |
| DATA FLOW | vCore-hours | debug session or TTL left running |
Note that orchestration counts every activity run, trigger run and debug run, not just pipeline runs. And every duration rounds up to the next minute.
Then find the driver:
- Cost Management - in Azure Cost Management, break the cost down by meter. Per pipeline billing, in the factory settings, splits cost by pipeline, but only from the day you turn it on.
- Consumption report - for past runs, it shows each run's billable units.
- Budget alert - set one, so next time you hear about it early.
The usual causes are a ForEach running thousands of tiny activities, a trigger changed to every few minutes, a debug session or time to live left running, or too many DIUs on small files.
To cut it: batch the tiny activities, schedule only as often as the business needs, use Auto DIU, and shorten the time to live.
Never quote prices in an interview or a cost review from memory. Check the pricing page.
Question 3: A copy from an on-premises Oracle database got ten times slower after a network change. How do you troubleshoot it?
Start with evidence, not guesses. The path is Oracle on-premises, then the self-hosted runtime on your machine, then the network change (a proxy, a firewall, a route), then the Azure sink in ADLS or SQL. One of those got slower.
Let's find out which, step by step.
First, in Monitor, compare the copy details of a slow run with a good one. The duration is split into stages: queue, time to first byte, reading from source, and writing to sink. The stage that grew is your answer:
| Stage that grew | Look at |
|---|---|
| QUEUE | runtime busy: CPU, memory, concurrent jobs |
| TIME TO FIRST BYTE | Oracle slow to answer: query, load |
| READING FROM SOURCE | runtime to Oracle path |
| WRITING TO SINK | runtime to Azure: proxy, TLS inspection |
So a long queue means the self-hosted runtime is busy. If reading grew, the path from the runtime to Oracle changed. If writing grew, the path out to Azure changed, often a new proxy or TLS inspection.
Then test from the runtime machine itself, not your laptop: latency and throughput to the Oracle host and to the storage endpoint. And ask the network team exactly what changed.
The fix depends on the cause:
- Move the runtime closer to the source.
- Read Oracle in parallel with partitions.
- Use staged copy with compression on a slow link.
- Scale out, up to 4 nodes.
"What exactly changed?" saves hours. Bring the stage that grew to the network team, so the conversation starts from evidence.
Question 4: Leadership asks whether we should move our ADF pipelines to Fabric Data Factory. How do you assess it, and what doesn't move?
Start with the facts. Azure Data Factory is fully supported: not deprecated, no forced upgrade, no deadline. Microsoft's docs call Fabric Data Factory the next generation, new innovation lands there, and new users are pointed to Fabric.
Then assess it with the built-in tooling, which is in preview. It works in three steps:
- View in Fabric - shows the factory in a Fabric workspace. Nothing is upgraded.
- Assess - a read-only readiness report. Each item comes back as Ready, Needs review, Coming soon or Not compatible.
- Upgrade - you upgrade selected pipelines.
Now the part leadership needs to hear: what doesn't move as is.
| In ADF | In Fabric |
|---|---|
| SELF-HOSTED IR | on-premises data gateway |
| MANAGED VNET IR | VNet data gateway |
| TUMBLING WINDOW TRIGGER | redesign: interval schedules |
| CUSTOM EVENT TRIGGER | not upgraded |
| SCHEDULE, STORAGE TRIGGERS | upgraded, but disabled |
| LINKED SERVICES, DATASETS | connections; dataset settings in activities |
| GLOBAL PARAMETERS | variable libraries |
| CDC RESOURCE, DATA FLOWS | Copy job; data flow conversion (preview) |
The biggest items are the runtimes, which become gateways, and the triggers: tumbling window triggers need a redesign as interval schedules, custom event triggers don't upgrade, and schedule and storage triggers arrive disabled.
My recommendation: assess, pilot one workload, compare a Fabric capacity against today's meters, and upgrade incrementally.
A partial upgrade is a supported outcome. You don't have to move everything to move something.
Question 5: You inherit a factory with three hundred pipelines and no documentation. What do you do in your first two weeks?
Week one is inventory, and I change nothing. Week two is risk and documentation: what breaks soon, and who owns what.
Week one, step by step:
- ARM template export - if there is no Git, I export the ARM template as a baseline.
- Triggers, linked services, IRs - I list the started triggers, the linked services and the integration runtimes.
- Logs to Log Analytics - on day one, because Monitor keeps only 45 days.
- Runs and failures - counted per pipeline, so I know what actually runs.
This query gives me the runs and failures per pipeline, worst first:
ADFPipelineRun
| where Status in ("Succeeded", "Failed")
| summarize Runs = count(),
Failed = countif(Status == "Failed") by PipelineName
| order by Failed descA pipeline with no trigger and no runs is a candidate to retire, after checking with the owner.
Week two, I look for what breaks soon:
- A self-hosted runtime version close to expiry.
- Old connector versions.
- Passwords typed into linked services instead of Key Vault.
- The legacy trusted service mode, which retires on August 1, 2027.
Then I document the critical chains, what each one loads and who owns it, and add failure alerts.
A short list of risks and quick wins at the end of week two, not a long document nobody reads.
Question 6: A source API returns data in pages and sometimes rate-limits you. How do you load it reliably in ADF?
Use the copy activity with the REST connector, and let it page. The flow is simple: the source API, then the copy activity with pagination rules, then raw JSON in the data lake, one folder per load date.
First, paging. Pagination rules build the next request from the response: a next link in the body, a header, or an offset parameter. It stops when the next link comes back null. Add an end condition, or a maximum request number, so a buggy API that keeps returning the same next link can't loop forever.
"source": {
"type": "RestSource",
"paginationRules": {
"AbsoluteUrl": "$.nextLink",
"EndCondition:$.data": "Empty"
},
"requestInterval": "00:00:02",
"httpRequestTimeout": "00:01:40"
}Here AbsoluteUrl takes the next page from $.nextLink, and the end condition stops when $.data comes back empty.
Then rate limits. We slow down on purpose:
- Request interval - the wait between page requests. Raise it to match the API's limit; here it is 2 seconds.
- Low ForEach batch count - few calls at once.
- Activity retry - with an interval, for throttling errors.
Finally, keep the token in Key Vault, and land raw JSON in a folder per load date, so a rerun overwrites instead of duplicating.
If the paging is too odd for the rules, use an Until loop with a Web activity, and remember its 4 MB response limit.
Scenario 7: In the final round, the hiring manager asks you to design a daily lakehouse load with ADF from ten source systems. What is your design?
I'd make it metadata-driven. A new table should be a row, not a pipeline.
Let's build it from the top:
- Control table. One row per source object: system, table, load type, watermark column and target path.
- Parent pipeline. It reads the control table with a Lookup, and a ForEach calls one generic child pipeline per object.
- Trigger. One schedule trigger starts it, or a tumbling window if you need backfill.
- Child pipeline. A generic copy. It lands raw Parquet in bronze, by source and load date.
- Incremental loads. Watermarks or change tracking, and the watermark moves only after success.
- Connectivity. The ten sources are SQL, Oracle, APIs and files. On-premises systems use a self-hosted runtime with 2 or more nodes, and cloud sources use managed private endpoints.
- Bronze to silver to gold. These steps run in Databricks or data flows, called from the pipeline, and gold serves reporting.
Then everything around it:
- Secrets in Key Vault, with managed identity.
- An audit table with row counts per run.
- Failure alerts and Log Analytics.
- CI/CD from Git.
And every step is safe to rerun.
For a brand new build, I'd also evaluate Fabric Data Factory, since Microsoft points new users there. Showing that trade-off wins the room.
Key points to remember
- Incident: communicate first, triage, rerun, validate
- Cost: orchestration, DIU-hours, activity hours, vCore-hours
- Slow copy: compare stages, test from the runtime machine
- Fabric: supported ADF, preview upgrade, know what doesn't move
- Design: metadata-driven, incremental, safe to rerun
All 10 parts are listed on the Real Time Azure Data Factory Interview Questions and Answers page on this blog.
Part of the Real Time Azure Data Factory Interview Questions and Answers series on TechBrothersIT. Features, limits and their status (GA or preview) are current as of October 2026; check the Microsoft Learn page for anything you quote in an interview.
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.