In this post we are going to cover data quality and health management in Microsoft Purview: how Purview measures bad data in your tables, and how it scores the governance program itself. These are 7 real Purview interview questions, asked the way they come up on a real project, and the last one is a scenario where leadership asks how healthy your governance is overall.
Data quality in Purview means rules that check your data, a scan that runs them, and scores that tell the business how good the data is. Health management goes one level up: it checks things like owners, descriptions and classification coverage. Interviewers ask about both because every governance project gets the same question sooner or later: "can we trust this data, and can you prove it?"
Part 6 of the Real Time Microsoft Purview Interview Questions and Answers series on TechBrothersIT.
- The short answer
- Question 1: Finance says the customer table has bad emails and missing countries. How does Purview data quality measure that?
- Question 2: You need a rule that order dates are never in the future and amounts are never negative. What rule types are there, and when do you write a custom rule?
- Question 3: The data quality scan fails to connect to a Fabric lakehouse. How does data quality connect and authenticate to sources?
- Question 4: The business asks for one quality number per data product. How are data quality scores calculated and rolled up?
- Question 5: Support wants to see the actual bad rows, not just a score. Where do failed records go, and what are the limits?
- Question 6: Your manager wants an alert when quality drops below 90 percent. How do thresholds, schedules and notifications work?
- Scenario 7: Leadership asks how healthy our governance is overall: owners, descriptions, quality, classification. What is health management, and how do you report it?
The short answer
Data quality runs on assets in data products: connect, profile, add built-in or custom rules, scan, and read scores that roll up from column to asset, data product and domain. Failed rows go to a Fabric lakehouse or ADLS Gen2, and alerts fire on score drops. Health management scores the governance itself with controls, actions and reports.
Keep that in mind. Every question below is one piece of it.
Question 1: Finance says the customer table has bad emails and missing countries. How does Purview data quality measure that?
Purview measures it with rules and a quality scan. Data quality lives in Unified Catalog, under Health management, and it runs on assets that are scanned into the Data Map and added to a data product in a governance domain. For an asset outside a data product, there is also a standalone asset scan.
Let's walk through the flow in order:
- Connection. Create a data quality connection to the source.
- Profile. Profile the table. It shows the shape first: nulls, distinct values and distributions per column. Here it would show that country is often empty.
- Rules. Add built-in or custom rules.
- Quality scan. Run the scan. It counts the rows that pass.
- Score. Read the score, per rule and per asset.
For our customer table, the rules look like this:
| Column | Rule | Dimension |
|---|---|---|
| String format match (regex) | Conformity | |
| country | Empty or blank fields | Completeness |
The scan runs on a Microsoft managed Spark engine, so you don't manage any compute.
Profiling tells you which columns and rules are worth the effort, before you write a single rule.
Question 2: You need a rule that order dates are never in the future and amounts are never negative. What rule types are there, and when do you write a custom rule?
Use a built-in rule when one fits. When the check is business logic, write a custom rule.
First, the built-in rules:
- Freshness
- Unique values
- String format match, with regex, LIKE or a list
- Data type match
- Duplicate rows
- Empty or blank fields
- Table lookup, where the reference table must be in the same governance domain
None of them says a date is not in the future, or an amount is not negative. So we write a custom rule, and there are two kinds. A custom expression rule uses the Azure Data Factory data flow expression language: a row expression that passing rows must meet, plus an optional filter and a null expression. A custom SQL rule uses Spark SQL, and only the row expression is required.
// Custom (expression) rules: ADF data flow language
order_date <= currentDate()
amount >= 0
-- Custom (SQL) rule: Spark SQL row expression
order_date <= current_date() AND amount >= 0| Expression rule | SQL rule | |
|---|---|---|
| LANGUAGE | ADF data flow | Spark SQL |
| WINDOW FUNCTIONS | no | yes, same dataset |
| JOINS, DML | no | no |
SQL rules can use subqueries and window functions on the same dataset, but they can't join other tables, and inserts or drops are blocked. Also, suggest rules lets AI propose a first set for you.
Freshness isn't supported for Snowflake, Databricks, BigQuery, Synapse or Azure SQL, and it only scores 100 or 0.
Question 3: The data quality scan fails to connect to a Fabric lakehouse. How does data quality connect and authenticate to sources?
For Microsoft sources, data quality connects with the Purview managed identity only. So the fix is usually a missing role, the wrong format or the wrong region.
Microsoft sources here means Fabric, ADLS Gen2, Azure SQL, SQL managed instance and Synapse. Each one needs a role for that identity:
| Source | Authentication | Grant |
|---|---|---|
| Fabric lakehouse | Purview managed identity | Contributor on the workspace |
| ADLS Gen2 | Purview managed identity | Storage Blob Data Reader |
| Azure SQL | Purview managed identity | db_datareader |
| Databricks, Snowflake, BigQuery | own credentials | stored in Key Vault |
Here is the checklist I go through, in order:
- Role: a Data Quality Steward creates the connection under Manage, Connections.
- Identity: the Purview managed identity has Contributor on the Fabric workspace.
- Format: the Fabric tables are in Delta or Iceberg format.
- Region: the Purview account and the data source are in the same Azure region.
- Private access: if the source only allows private access, data quality goes through its own managed virtual network.
Databricks, Snowflake and BigQuery can't use the managed identity. They use their own credentials, stored in Azure Key Vault.
The region rule catches a lot of people. And with the managed virtual network, test connection isn't available, so you test with a real scan.
Question 4: The business asks for one quality number per data product. How are data quality scores calculated and rolled up?
Scores roll up in four levels: column, then asset, then data product, then governance domain. Each dimension, like completeness or conformity, is scored separately.
Let's see how the data product score is calculated. A dimension score is the records that pass that dimension's rules across all the product's assets, divided by all the records measured, times 100. Then the overall score is the average of the dimensions that were measured.
| Dimension | Passing of measured | Score |
|---|---|---|
| Completeness | 9,500 of 10,000 records | 95 |
| Conformity | 9,000 of 10,000 records | 90 |
| Timeliness | no rules | excluded |
| Overall | average of 95 and 90 | 92.5 |
So completeness is 95, conformity is 90, and the product scores 92.5. Timeliness has no rules, so it is left out.
Now be careful here: a blank dimension is excluded, but a zero counts. A freshness rule only scores 100 or 0, so one stale table can pull the whole average down.
Give the business the data product score, but show the dimensions next to it. One number alone hides where the problem is.
Question 5: Support wants to see the actual bad rows, not just a score. Where do failed records go, and what are the limits?
Purview can publish the failed rows, called error records, in Delta format to a Fabric lakehouse or an ADLS Gen2 container. For on-premises sources, in preview, they go to an Oracle or SQL Server error database.
Let's set it up step by step. First, in the governance domain, go to Manage, Settings and pick the storage. Then give the Purview managed identity Contributor on the Fabric workspace, or Storage Blob Data Contributor on the container. Finally, turn on publishing of failed rows, per scan or for the domain.
Each failed row comes with metadata columns like the run ID and the rule ID, so support can filter by rule:
| customer_id | _purviewDQRuleId | |
|---|---|---|
| C1042 | sara.khan.tbshop.com | 3f9c2a1e |
| C1077 | omar@@tbshop.com | 3f9c2a1e |
| C1113 | lina@ | 3f9c2a1e |
Now the limits:
- At most 100,000 failed records per rule per run.
- The storage must be in the same region. If it isn't, publishing is silently skipped, while the scan still succeeds.
- Publishing of failed rows must be turned on.
They may be personal data. Secure that storage like the source, and give score-only users the Data Quality Metadata Reader role, which doesn't show error records.
Question 6: Your manager wants an alert when quality drops below 90 percent. How do thresholds, schedules and notifications work?
It is three pieces: a schedule that runs the scan, a threshold that marks a score as failing, and an alert that emails people.
First the schedule. In the governance domain, under Manage, Scheduled scans, you schedule products and assets, or the whole domain, with up to 30 assets per schedule. Assets also get a weekly schedule by default. For big tables, use an incremental scan. It filters on a date time column, like the last day or week, and the cumulative score averages the incremental and full scores. It isn't supported for on-premises sources.
Then the threshold and the alert:
| Setting | Example |
|---|---|
| Threshold | rule or asset fails below 90% |
| Alert | score less than 90% |
| Alert | score decreased by more than 5% |
| Failed scan | email the recipients |
You configure thresholds on rules and assets, so a score under 90 shows as failing for that table. The alert is set by a Data Quality Steward at domain level, with a trigger like score less than 90 percent, or score decreased by more than 5 percent, plus an email when a scan fails.
Schedule the scan after the nightly load. And each run uses processing units, DGPUs, so match the frequency to how often the data changes.
Also, failed rules create health actions, which brings us to the last question.
Scenario 7: Leadership asks how healthy our governance is overall: owners, descriptions, quality, classification. What is health management, and how do you report it?
Health management is the part of Unified Catalog that scores the governance itself. Let's build the answer in four steps.
- Controls. There are 12 built-in controls in six groups, like trusted data and discoverability. You can't create custom controls, but you set target scores and status bands: Healthy, Fair, Not healthy and Critical.
- Map each ask. Each of leadership's four asks maps to a control or a set of rules.
- Actions. When a rule fails, health management creates an action, in preview: a work item with an owner and a severity. If someone marks it resolved but the issue is still there, it comes back on the next refresh.
- Reports. The data governance report, in preview, shows the controls, and the data quality health report shows dimensions with a 13 month trend. Both are billed; the classic reports are free and weekly.
Here is how the mapping looks:
| Leadership asks | Health management answer |
|---|---|
| Owners | Data product ownership control |
| Descriptions | Metadata quality rules (data product usability) |
| Quality | Data quality enablement control, plus scores |
| Classification | Classification and labeling control |
So my slide is four numbers with a trend, plus open actions by owner.
These controls measure metadata and quality, not who actually has access in the source.
Key points to remember
- DQ runs on assets in data products: profile, rules, scan
- Custom rules: ADF expressions or Spark SQL, no joins
- Managed identity for Microsoft sources, same region
- Scores roll up column, asset, product, domain
- Health management: controls, actions, reports
Part 7: Access, Roles and Self-Service Requests
Part of the Real Time Microsoft Purview Interview Questions and Answers series on TechBrothersIT. Product names, features and their status (GA or preview) are current as of October 2026. Microsoft changes Purview every month, so check the Microsoft Learn page for anything you quote in an interview.
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.