In this post we are going to cover integration runtimes and security in Azure Data Factory: how ADF reaches your data, and how you keep that access safe. These are 7 real Azure Data Factory interview questions, asked the way they come up on a real project, and the last one is a scenario where an auditor asks how your data factory is secured end to end.
The integration runtime is the compute that actually moves the data. Which one you pick decides what ADF can reach: public Azure endpoints, servers in your own data center, or databases inside a private network. Interviewers ask about it because most production incidents and most security reviews land right here: an offline self-hosted runtime, a password in a linked service, or a firewall setting that is about to retire.
Part 2 of the Real Time Azure Data Factory Interview Questions and Answers series on TechBrothersIT.
- The short answer
- Question 1: Your pipeline needs to read from Azure SQL, an on-premises SQL Server and a database inside a private network. What are the integration runtime types, and when do you use each one?
- Question 2: The self-hosted integration runtime shows offline this morning and every on-premises copy is failing. How do you troubleshoot it, step by step?
- Question 3: The security team says no public endpoints: the storage account and SQL database are private only. How does ADF reach them with a managed virtual network and managed private endpoints?
- Question 4: A developer stored the SQL password directly in the linked service. How should ADF authenticate instead, with managed identity and Key Vault?
- Question 5: Your company runs SSIS packages on premises and wants to move them to Azure without rewriting. What is the Azure-SSIS integration runtime, and what are the trade-offs?
- Question 6: The storage firewall allows trusted Microsoft services, and a security review flags the legacy mode. What is changing, and what do you do?
- Scenario 7: An auditor asks how your data factory is secured end to end. What do you show?
The short answer
There are three integration runtimes: Azure (public endpoints, or inside a managed VNet with managed private endpoints), self-hosted (on-premises and private networks, outbound only) and Azure-SSIS. Authenticate with the factory's managed identity and keep every other secret in Key Vault. The legacy trusted service mode retires on August 1, 2027.
Keep that in mind. Every question below is one piece of it.
Question 1: Your pipeline needs to read from Azure SQL, an on-premises SQL Server and a database inside a private network. What are the integration runtime types, and when do you use each one?
There are three types: Azure, self-hosted and Azure-SSIS. Let's match each source to its runtime.
- Azure SQL Database, public endpoint. The Azure integration runtime is fully managed and serverless, and reaches stores over public endpoints. So Azure SQL uses the default Azure runtime, auto-resolve.
- SQL Server on-premises. The self-hosted runtime is software on a Windows machine in your own network. It only calls out on port 443, so nothing inbound is opened.
- A database inside a private network. Two options: install a self-hosted runtime on a virtual machine in that network or a peered one, or use an Azure runtime in a managed virtual network, with a managed private endpoint.
The third type, Azure-SSIS, runs SSIS packages. We get to it in Question 5.
Auto-resolve has rules too. A copy tries to run in the sink's region, while Lookup and Get Metadata use the factory's region.
If the source and the sink use different runtimes, self-hosted wins, then managed virtual network, then Azure. For data residency, create an Azure runtime in a fixed region instead of auto-resolve.
Question 2: The self-hosted integration runtime shows offline this morning and every on-premises copy is failing. How do you troubleshoot it, step by step?
I go from the machine outward, in five steps.
- Step 1, the machine and the service. Is the server up, and is the runtime service running? Check the status in the configuration manager on the node.
- Step 2, the version. Each version expires after 1 year, and an expired runtime shows offline. The portal warns 90 days before. Turn on auto-update, or install the latest version.
- Step 3, the network. Each node needs outbound port 443 to the factory, to service bus for interactive authoring, and to Key Vault. A new proxy or TLS inspection often breaks this.
- Step 4, Running (Limited). If the status is limited, not offline, either service bus is blocked, or the nodes can't talk to each other on port 8060. Check the event viewer logs, and test that port.
- Step 5, credentials. They are encrypted with Windows DPAPI and synced between nodes, and a FIPS-enabled server breaks that. Keep them in Key Vault instead.
Here is the firewall list I give the network team:
# outbound 443 from every node
{factory}.{region}.datafactory.azure.net
*.servicebus.windows.net # interactive authoring
download.microsoft.com # auto-update
{your-vault}.vault.azure.net # Key Vault
# between nodes only
port 8060 # Running (Limited)Run up to 4 nodes for high availability, and put an alert on the runtime status, so you hear about it before the business does.
Question 3: The security team says no public endpoints: the storage account and SQL database are private only. How does ADF reach them with a managed virtual network and managed private endpoints?
You use an Azure integration runtime inside a managed virtual network, plus one managed private endpoint per resource. Let's set it up in order.
First, create the Azure integration runtime inside a managed virtual network. Microsoft manages that network, and it sits in the factory's region. You can't convert an existing public runtime; you create a new one.
Then, in the Manage hub, create a managed private endpoint for each resource: one for the storage account and one for the SQL server. Each one starts as pending.
Next, the resource owner approves it. Only approved endpoints carry traffic. Point the linked services at the new runtime, and the storage account and the SQL database keep public access off.
Now the two gotchas I always mention:
- Cold start. The compute starts cold, so copies can queue for a few minutes. Time to live keeps it warm, but it bills reserved compute while it waits.
- Key Vault. Its private endpoint is created from a data store linked service.
The ADF managed network leaves outbound ports open. To control where data can go, add the Azure Policy outbound allow list, which is in preview. Full outbound blocking is a Synapse workspace feature.
Question 4: A developer stored the SQL password directly in the linked service. How should ADF authenticate instead, with managed identity and Key Vault?
The best answer is no password at all. Use the factory's managed identity wherever the store supports it, like Azure SQL and ADLS Gen2. In SQL, create a user for the factory from Entra ID. On storage, grant a role like Storage Blob Data Reader.
Where a password can't be avoided, like SQL authentication on-premises, put it in Key Vault. Let's walk through it.
- Create a Key Vault linked service that uses the factory's managed identity.
- Grant that identity Get and List on secrets, or the Key Vault Secrets User role.
- In the SQL linked service, reference the secret by name instead of typing the password.
{ "name": "ls_sqlserver_sales",
"properties": {
"type": "SqlServer",
"typeProperties": {
"server": "sql01.contoso.local",
"database": "Sales",
"authenticationType": "SQL",
"userName": "adf_reader",
"password": {
"type": "AzureKeyVaultSecret",
"store": { "referenceName": "ls_keyvault",
"type": "LinkedServiceReference" },
"secretName": "sql-sales-password" } },
"connectVia": { "referenceName": "shir-onprem",
"type": "IntegrationRuntimeReference" } } }Leave out the secret version, and ADF always uses the latest one. So when the password rotates, nothing changes in ADF.
Two more details. A system-assigned identity lives and dies with the factory; a user-assigned one can be shared across factories. And the trusted services firewall exception only works with system-assigned.
Anyone with Data Factory Contributor can build a pipeline that uses the factory's identity, and all of its access. Keep that role tight, and use secure input and output where secrets pass.
Question 5: Your company runs SSIS packages on premises and wants to move them to Azure without rewriting. What is the Azure-SSIS integration runtime, and what are the trade-offs?
The Azure-SSIS runtime is a managed cluster of Azure virtual machines that runs SSIS packages. It is lift and shift.
Here is how the move works:
- Deploy the packages to an SSIS catalog in Azure SQL Database or Managed Instance, or to a package store.
- Keep using SSDT and SSMS as before.
- Run the packages from pipelines with the Execute SSIS Package activity, so they are scheduled and monitored with everything else.
To reach on-premises data, the runtime joins an Azure Resource Manager virtual network. Classic virtual networks are being deprecated, so don't build on them.
Now the trade-offs. The good side: fast migration and no rewrite. The cost side: you pay per node-hour while it runs, by VM size, edition and licensing. The default limit is 64 cores per subscription.
It is generally available, with no retirement notice. Fabric also offers an Invoke SSIS Package activity, in preview.
Start the runtime before the batch and stop it after. Lift and shift first, then rewrite the heavy packages over time.
Question 6: The storage firewall allows trusted Microsoft services, and a security review flags the legacy mode. What is changing, and what do you do?
Many factories reach a firewalled storage account or Key Vault through the trusted Microsoft services exception, with the factory's managed identity. Factory settings now have two modes for it: legacy and modern.
Legacy mode retires on August 1, 2027. It affects these paths when they reach storage or Key Vault that way:
- The self-hosted runtime.
- The Azure-SSIS runtime.
- Web, REST and Azure Function calls.
So here is the plan, step by step:
- Inventory. Find every linked service and activity that depends on the exception, and which runtime each one uses.
- Modern mode, dev first. Switch the dev factory and test.
- Alternatives. Where a path still can't get through, use managed private endpoints, a private or service endpoint in your own network, or allow-list the stable IPs of your self-hosted or SSIS nodes.
- Test, then switch production well before the date.
Then tell the reviewer: we know the deadline, and here is the plan.
The trusted bypass only ever worked with the system-assigned identity.
Scenario 7: An auditor asks how your data factory is secured end to end. What do you show?
I walk the auditor through five layers, and I show the settings and the logs, not slides.
- Identity. No passwords in linked services. Managed identity where possible, every other secret in Key Vault, and one vault per environment.
- Network. Stores have public access off and are reached through approved managed private endpoints, and the self-hosted runtime only calls out. The factory itself can have private endpoints too: datafactory for the self-hosted command channel, and portal for ADF Studio.
- Access. Data Factory Contributor only for builders, Reader for people who monitor. Git on dev only, and production changes only through CI/CD.
- Data protection. Data at rest is encrypted by default. Customer-managed keys can only be turned on for an empty factory, and can't be turned off. And secure input and output where secrets pass.
- Evidence. Diagnostic logs go to Log Analytics, into the ADFPipelineRun, ADFActivityRun and ADFTriggerRun tables, with alerts on failures and a plan for the trusted service retirement.
Run history stays only 45 days in the monitor, so anything older has to come from the logs you kept.
Key points to remember
- Azure IR for public endpoints, self-hosted for on-premises, Azure-SSIS for packages
- SHIR offline: service, version expiry, outbound 443, port 8060
- Managed private endpoints: pending until the owner approves
- Managed identity first; every other secret in Key Vault
- Legacy trusted service mode retires August 1, 2027
Part 3: Copy Activity and Performance
Part of the Real Time Azure Data Factory Interview Questions and Answers series on TechBrothersIT. Features, limits and their status (GA or preview) are current as of October 2026; check the Microsoft Learn page for anything you quote in an interview.
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.