In this post we are going to cover topics, subscriptions and filters in Azure Service Bus. These are 7 real Service Bus interview questions for data engineers, asked the way they come up on a real project, and the last one is a scenario: an e-commerce platform must route orders to warehouses by region and priority without changing the publisher, and you design it.
A topic is where a publisher sends each message once. Every subscription on the topic gets its own copy, and rules on the subscription decide which messages it takes. Interviewers ask about this because it's how one event feeds many teams, and because the classic bugs live here: a subscription that gets everything, a filter that silently misses, and messages that never arrive.
Part 3 of the Real Time Azure Service Bus Interview Questions and Answers series on TechBrothersIT.
- The short answer
- Let's set up the project
- Question 1: Three teams need every order event. How do topics and subscriptions give each team its own copy?
- Question 2: A subscriber only needs refund events. How do SQL filters and correlation filters work?
- Question 3: Someone asks why a subscription with no rules gets everything. What is the default rule?
- Question 4: A team wants to add a property to messages as they arrive in their subscription. How do rule actions work?
- Question 5: Messages published before a subscription existed never arrive. Why, and what do you do?
- Question 6: A teammate asks about forwarding messages from a subscription to a queue. How does auto-forwarding work?
- Scenario 7: An e-commerce platform must route orders to warehouses by region and priority without changing the publisher. How do you design it?
- The full script
The short answer
A topic gives every subscription its own copy of each message. Rules decide what a subscription takes: a SQL or correlation filter, the default true rule when you add none, and optional actions that set properties. A subscription only gets messages published after it exists, and auto-forwarding moves its messages into a queue.
Let's set up the project
We run Microsoft's official Service Bus emulator locally, with SQL Server next to it, and small Python scripts that use the azure-servicebus client, so every output is real. Features the emulator doesn't support are shown as code. The emulator reads its topics, subscriptions and rules from Config.json. send.py publishes six order events to the order-events topic, o-1 to o-6, each with a subject like order.placed or order.refunded and the application properties region and amount. read.py reads every subscription we name and prints what each one got.
{
"UserConfig": {
"Namespaces": [
{
"Name": "sbemulatorns",
"Queues": [
{
"Name": "warehouse-inbox",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"RequiresDuplicateDetection": false,
"RequiresSession": false
}
},
{
"Name": "wh-east",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"RequiresDuplicateDetection": false,
"RequiresSession": false
}
},
{
"Name": "wh-west",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"RequiresDuplicateDetection": false,
"RequiresSession": false
}
},
{
"Name": "wh-express",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"RequiresDuplicateDetection": false,
"RequiresSession": false
}
},
{
"Name": "wh-unrouted",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"RequiresDuplicateDetection": false,
"RequiresSession": false
}
}
],
"Topics": [
{
"Name": "order-events",
"Properties": {
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"RequiresDuplicateDetection": false
},
"Subscriptions": [
{
"Name": "billing",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"RequiresSession": false
}
},
{
"Name": "shipping",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"RequiresSession": false
}
},
{
"Name": "email",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"RequiresSession": false
}
},
{
"Name": "big-refunds",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"RequiresSession": false
},
"Rules": [
{
"Name": "big-only",
"Properties": {
"FilterType": "Sql",
"SqlFilter": {
"SqlExpression": "sys.Label = 'order.refunded' AND amount >= 100"
}
}
}
]
},
{
"Name": "eu-refunds",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"RequiresSession": false
},
"Rules": [
{
"Name": "eu-only",
"Properties": {
"FilterType": "Correlation",
"CorrelationFilter": {
"Label": "order.refunded",
"Properties": {
"region": "eu"
}
}
}
}
]
},
{
"Name": "audit",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"RequiresSession": false
}
},
{
"Name": "two-rules",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"RequiresSession": false
},
"Rules": [
{
"Name": "match-all",
"Properties": {
"FilterType": "Sql",
"SqlFilter": {
"SqlExpression": "1=1"
}
}
},
{
"Name": "big-only",
"Properties": {
"FilterType": "Sql",
"SqlFilter": {
"SqlExpression": "sys.Label = 'order.refunded' AND amount >= 100"
}
}
}
]
},
{
"Name": "paused",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"RequiresSession": false
},
"Rules": [
{
"Name": "match-none",
"Properties": {
"FilterType": "Sql",
"SqlFilter": {
"SqlExpression": "1=0"
}
}
}
]
},
{
"Name": "refund-desk",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"RequiresSession": false
},
"Rules": [
{
"Name": "high-value",
"Properties": {
"FilterType": "Sql",
"SqlFilter": {
"SqlExpression": "sys.Label = 'order.refunded' AND amount >= 100"
},
"Action": {
"SqlExpression": "SET priority = 'high'"
}
}
},
{
"Name": "normal-value",
"Properties": {
"FilterType": "Sql",
"SqlFilter": {
"SqlExpression": "sys.Label = 'order.refunded' AND amount < 100"
},
"Action": {
"SqlExpression": "SET priority = 'normal'"
}
}
}
]
},
{
"Name": "warehouse",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "warehouse-inbox",
"RequiresSession": false
},
"Rules": [
{
"Name": "placed-only",
"Properties": {
"FilterType": "Sql",
"SqlFilter": {
"SqlExpression": "sys.Label = 'order.placed'"
}
}
}
]
}
]
},
{
"Name": "orders",
"Properties": {
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"RequiresDuplicateDetection": false
},
"Subscriptions": [
{
"Name": "east",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "wh-east",
"RequiresSession": false
},
"Rules": [
{
"Name": "east-only",
"Properties": {
"FilterType": "Correlation",
"CorrelationFilter": {
"Properties": {
"region": "east"
}
}
}
}
]
},
{
"Name": "west",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "wh-west",
"RequiresSession": false
},
"Rules": [
{
"Name": "west-only",
"Properties": {
"FilterType": "Correlation",
"CorrelationFilter": {
"Properties": {
"region": "west"
}
}
}
}
]
},
{
"Name": "express",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "wh-express",
"RequiresSession": false
},
"Rules": [
{
"Name": "express-only",
"Properties": {
"FilterType": "Correlation",
"CorrelationFilter": {
"Properties": {
"priority": "express"
}
}
}
}
]
},
{
"Name": "unrouted",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "wh-unrouted",
"RequiresSession": false
},
"Rules": [
{
"Name": "no-warehouse",
"Properties": {
"FilterType": "Sql",
"SqlFilter": {
"SqlExpression": "region IS NULL OR region NOT IN ('east', 'west')"
}
}
}
]
}
]
}
]
}
],
"Logging": {
"Type": "File"
}
}
}import os
from azure.servicebus import ServiceBusClient, ServiceBusMessage
EVENTS = [("o-1", "order.placed", "eu", 120),
("o-2", "order.placed", "us", 40),
("o-3", "order.refunded", "eu", 150),
("o-4", "order.refunded", "us", 30),
("o-5", "order.refunded", "us", 200),
("o-6", "order.shipped", "eu", 120)]
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
with client, client.get_topic_sender("order-events") as sender:
sender.send_messages([ServiceBusMessage(
f"{mid} {subject}", message_id=mid, subject=subject,
application_properties={"region": region, "amount": amount})
for mid, subject, region, amount in EVENTS])
print("published", len(EVENTS), "events to topic order-events")import os, sys
from azure.servicebus import ServiceBusClient
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
with client:
for sub in sys.argv[1:]: # subscription names
got = []
with client.get_subscription_receiver(
"order-events", sub, max_wait_time=2) as receiver:
while batch := receiver.receive_messages(
max_message_count=20, max_wait_time=2):
for msg in batch:
got.append(msg.message_id)
receiver.complete_message(msg) # this copy only
print(f"{sub:<15}got", sorted(got))Question 1: Three teams need every order event. How do topics and subscriptions give each team its own copy?
A topic is where the publisher sends each event once. Every subscription on that topic gets its own copy, and Microsoft describes a subscription as a virtual queue. So billing, shipping and email each get a subscription.
- Order service - publishes once
- order-events - topic
- billing, shipping, email - subscriptions, each with its own copy
Let's publish the six events in one send with python send.py. The order service doesn't know who is listening. Then each team reads its own subscription with python read.py billing shipping email.
$ python send.py
published 6 events to topic order-events
$ python read.py billing shipping email
billing got ['o-1', 'o-2', 'o-3', 'o-4', 'o-5', 'o-6']
shipping got ['o-1', 'o-2', 'o-3', 'o-4', 'o-5', 'o-6']
email got ['o-1', 'o-2', 'o-3', 'o-4', 'o-5', 'o-6']published 6 events to topic order-events, and billing, shipping and email each got ['o-1', 'o-2', 'o-3', 'o-4', 'o-5', 'o-6']. Each one completes only its own copy, so completing in billing takes nothing away from shipping.
From the receiver side, a subscription works exactly like a queue: peek-lock, delivery count and its own dead-letter queue.
Give every consuming application its own subscription. Two services sharing one subscription compete for its messages, and each sees only part of the stream.
Question 2: A subscriber only needs refund events. How do SQL filters and correlation filters work?
A filter is a rule on the subscription, checked as each message arrives. A SQL filter is a condition on user and system properties, and system ones start with sys.. A correlation filter is a set of exact matches that must all match.
| SQL filter | Correlation filter | |
|---|---|---|
| Tests | a condition on user and system (sys.) properties | exact matches on system fields and user properties |
| Operators | =, <>, >=, LIKE, IN, EXISTS, IS NULL, AND, OR, NOT | equality only; several fields = AND |
| Throughput | lower overall throughput | much more efficient: prefer it |
On Azure, we create rules with the administration client, then delete the default rule, as question 3 shows.
import os
from azure.servicebus.management import (
ServiceBusAdministrationClient, SqlRuleFilter,
CorrelationRuleFilter)
admin = ServiceBusAdministrationClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"]) # an Azure namespace
admin.create_rule("order-events", "big-refunds", "big-only",
filter=SqlRuleFilter(
"sys.Label = 'order.refunded' AND amount >= 100"))
admin.create_rule("order-events", "eu-refunds", "eu-only",
filter=CorrelationRuleFilter(label="order.refunded",
properties={"region": "eu"}))
# both still have the default true rule: delete it (question 3)The emulator keeps these two rules in its config file. big-refunds is SQL: subject order.refunded, and amount at least 100. eu-refunds is a correlation filter: subject order.refunded, and region eu. Let's read both from the same six events with python read.py big-refunds eu-refunds.
$ python read.py big-refunds eu-refunds
big-refunds got ['o-3', 'o-5']
eu-refunds got ['o-3']big-refunds got ['o-3', 'o-5'], the two large refunds. eu-refunds got only ['o-3'].
Filters can't read the message body, so route on properties. Microsoft says correlation filters are much more efficient than SQL filters: prefer them when equality is enough.
String values are case-sensitive, so a publisher sending EU in capitals silently misses the filter. Agree names and values with the publishing team.
Question 3: Someone asks why a subscription with no rules gets everything. What is the default rule?
Every new subscription starts with a default rule. Its filter is the true filter, which matches every message, and it has no action. So a subscription with no rules of our own receives everything.
Let's take three subscriptions on the same topic.
| Subscription | Rules in the emulator config |
|---|---|
| audit | none: only the default true rule |
| two-rules | match-all (1=1) and big-only |
| paused | match-none: 1=0, a false filter |
two-rules has a match-all rule, 1=1, which is what the default rule does, plus the big refunds filter. We read all three with python read.py audit two-rules paused.
$ python read.py audit two-rules paused
audit got ['o-1', 'o-2', 'o-3', 'o-4', 'o-5', 'o-6']
two-rules got ['o-1', 'o-2', 'o-3', 'o-4', 'o-5', 'o-6']
paused got []audit gets all six events. two-rules also gets all six, because the rules on one subscription are combined with OR: if any rule matches, the message comes in, once. paused gets [].
That's the classic bug on Azure. A team creates a subscription, adds a refunds rule, and forgets that the default rule is still there, so they still get everything. The fix: list the rules and delete every one except ours.
rules = admin.list_rules("order-events", "big-refunds")
for rule in list(rules): # includes the default rule
if rule.name != "big-only":
admin.delete_rule("order-events", "big-refunds", rule.name)It stops new copies without deleting the subscription, which is handy while you retire a consumer.
Question 4: A team wants to add a property to messages as they arrive in their subscription. How do rule actions work?
A rule can carry an action. It runs after the filter matches, before the message lands in the subscription, and it sets or removes properties. Actions use a SQL-like syntax, and they change only that subscription's copy.
The refund-desk subscription has two rules.
| Rule | SQL filter | Action |
|---|---|---|
| high-value | sys.Label = 'order.refunded' AND amount >= 100 | SET priority = 'high' |
| normal-value | sys.Label = 'order.refunded' AND amount < 100 | SET priority = 'normal' |
Let's read the three refunds from question 1 with python tag.py. It prints each message's application properties.
import os
from azure.servicebus import ServiceBusClient
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
with client, client.get_subscription_receiver(
"order-events", "refund-desk", max_wait_time=2) as receiver:
msgs = []
while batch := receiver.receive_messages(max_message_count=20,
max_wait_time=2):
msgs += batch
for msg in sorted(msgs, key=lambda m: m.message_id):
raw = msg.application_properties # bytes keys
props = {k.decode(): v.decode() if isinstance(v, bytes)
else v for k, v in raw.items()}
print(msg.message_id, dict(sorted(props.items())))
receiver.complete_message(msg)$ python tag.py
o-3 {'RuleName': 'high-value', 'amount': 150, 'priority': 'high', 'region': 'eu'}
o-4 {'RuleName': 'normal-value', 'amount': 30, 'priority': 'normal', 'region': 'us'}
o-5 {'RuleName': 'high-value', 'amount': 200, 'priority': 'high', 'region': 'us'}o-3 and o-5 arrive with 'priority': 'high', and o-4 with 'priority': 'normal'. Region and amount are kept. Service Bus also adds RuleName, the matching rule: high-value or normal-value. And two matching rules with actions give two copies.
Microsoft recommends you don't change system properties in an action, even though you can. Actions are set with the CLI, PowerShell or code, not in the portal.
Actions are great for tags and routing hints. Keep business logic in the consumer: an action hidden in a rule is easy to forget.
Question 5: Messages published before a subscription existed never arrive. Why, and what do you do?
Because a topic copies a message only to the subscriptions it has when the message is published. Microsoft's docs say each published message is made available to each subscription registered with the topic, and messages are evaluated against filters as they are sent.
Let's walk through a timeline:
- 09:00 publish o-1 to o-3 - billing gets all three
- 09:05 create analytics - a new subscription
- 09:10 publish o-4 - copied to both
- billing - o-1, o-2, o-3, o-4
- analytics - o-4 only: never gets o-1 to o-3
On Azure, we create a subscription with the CLI or a template, like this. The emulator builds subscriptions only from its config file, so this one is shown as code, not run.
az servicebus topic subscription create \
--resource-group rg-shop --namespace-name sb-shop \
--topic-name order-events --name analyticsThe fix:
- Create subscriptions first - same deployment, before the publisher goes live
- Backfill from the system of record - a one-off load for the new consumer
- Need replay? - that is an Event Hubs feature, not Service Bus
The consumer's infrastructure ships after the producer. Deploy the subscription first, then turn on publishing.
Question 6: A teammate asks about forwarding messages from a subscription to a queue. How does auto-forwarding work?
We set ForwardTo on a queue or a subscription. Service Bus then removes each message from that source and puts it in the destination, a queue or topic in the same namespace. Consumers read the destination.
Here the warehouse subscription has a filter for placed orders, and forwards to the warehouse-inbox queue. forward.py reads that queue, then opens a receiver on the warehouse subscription itself. We run python forward.py.
import os
from azure.servicebus import ServiceBusClient
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
with client:
with client.get_queue_receiver("warehouse-inbox",
max_wait_time=2) as receiver:
msgs = receiver.receive_messages(max_message_count=20,
max_wait_time=2)
for msg in msgs:
receiver.complete_message(msg)
print("warehouse-inbox queue got",
sorted(m.message_id for m in msgs))
try: # the forwarding source
with client.get_subscription_receiver(
"order-events", "warehouse", max_wait_time=2) as r:
r.receive_messages(max_wait_time=2)
except Exception as e:
print(type(e).__name__)
print(str(e).split(" TrackingId")[0])$ python forward.py
warehouse-inbox queue got ['o-1', 'o-2']
ServiceBusError
Cannot create a message receiver on an entity with auto-forwarding enabled.The queue returns ['o-1', 'o-2'], the two placed orders from question 1. The receiver on the source fails with ServiceBusError: Cannot create a message receiver on an entity with auto-forwarding enabled. Always receive from the target.
From Microsoft's docs:
- Destination exists first
- Max 4 hops - then dead-lettered
- Session-enabled source - not allowed
- Not on Basic - one billed operation per forward
I forward subscriptions into one queue per consuming team. The team owns a plain queue, and the routing stays in the topic.
Scenario 7: An e-commerce platform must route orders to warehouses by region and priority without changing the publisher. How do you design it?
One topic, orders. The publisher adds two properties, region and priority, and nothing else. Each routing decision is a subscription with a filter, forwarding to a queue that a warehouse team reads.
- east - correlation: region = 'east', forwards to wh-east
- west - correlation: region = 'west', forwards to wh-west
- express - correlation: priority = 'express', forwards to wh-express
- unrouted - SQL: region missing or unknown, forwards to wh-unrouted
East, west and express are correlation filters: the efficient kind. So an express order lands twice, in its region's queue and in the express queue. Unrouted needs SQL, for a missing or unknown region.
Let's publish five orders with python publish.py, then read every warehouse queue with python warehouses.py.
import os
from azure.servicebus import ServiceBusClient, ServiceBusMessage
ORDERS = [("o-11", "east", "standard"), ("o-12", "west", "express"),
("o-13", "east", "express"), ("o-14", "west", "standard"),
("o-15", "north", "standard")]
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
with client, client.get_topic_sender("orders") as sender:
for order_id, region, priority in ORDERS:
sender.send_messages(ServiceBusMessage(
order_id, message_id=order_id,
application_properties={"region": region, # routing
"priority": priority})) # only
print("published", len(ORDERS), "orders; no warehouse named")import os
from azure.servicebus import ServiceBusClient
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
with client:
for queue in ["wh-east", "wh-west", "wh-express", "wh-unrouted"]:
got = []
with client.get_queue_receiver(queue, max_wait_time=3) as r:
while batch := r.receive_messages(max_message_count=20,
max_wait_time=3):
for msg in batch:
got.append(msg.message_id)
r.complete_message(msg)
print(f"{queue:<12}", sorted(got))$ python publish.py
published 5 orders; no warehouse named
$ python warehouses.py
wh-east ['o-11', 'o-13']
wh-west ['o-12', 'o-14']
wh-express ['o-12', 'o-13']
wh-unrouted ['o-15']wh-east gets ['o-11', 'o-13'], wh-west gets ['o-12', 'o-14'], and wh-express gets the two express orders, ['o-12', 'o-13']. Order o-15, from the north, lands in wh-unrouted, where someone will see it.
A new warehouse in the north is one more subscription and queue, plus a change to the unrouted filter. The publisher never changes. That's the point of routing with a topic.
Alert when the unrouted queue isn't empty: it means a new region or a bad publisher. And keep every subscription and rule in source control.
Key points to remember
- A topic gives every subscription its own copy: one subscription per app
- SQL filters test conditions; correlation filters match exactly and are faster
- No rules of your own = the default true rule; rules are OR-ed
- Actions SET or REMOVE properties on that subscription's copy
- Create subscriptions before publishing; receive from the forward target
The full script
This is a Python script that writes every file and runs every command above in order against the Service Bus emulator: start the emulator with the Config.json above, run pip install azure-servicebus first, and set SERVICEBUS_CONNECTION_STRING.
Show the full script that writes the files and runs every step
# ========================================================================
# Service Bus Interview Q&A #3 - Topics, subscriptions and filters
# TechBrothersIT. Names and sample data are invented for teaching.
# Every step below was run on the official Azure Service Bus emulator (Docker image, version 2.0.1) with
# SQL Server 2022 and azure-servicebus 7.15.0; the outputs in the video and the blog post are the real outputs.
# To run it yourself:
# 1. Start the emulator with Microsoft's installer (github.com/Azure/azure-service-bus-emulator-installer),
# using the Config.json this script writes first (Step 0) as the emulator's config file; it holds every
# queue, topic, subscription, rule and setting (lock duration, max delivery count, duplicate detection...)
# the steps depend on.
# 2. pip install azure-servicebus
# 3. Set SERVICEBUS_CONNECTION_STRING to the emulator connection string (Endpoint=sb://localhost;
# SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=SAS_KEY_VALUE;UseDevelopmentEmulator=true;)
# then run python ServiceBus_Interview_QA_03_Code.py.
# Run it in an empty folder and on a freshly started emulator: it creates its files (and any SQLite
# databases) there, and a second run on old files or old messages gives different output.
# Steps that restart the emulator say so in a comment: stop and start it by hand at that point.
# ========================================================================
import os, subprocess, time
def write(path, text):
if text is None:
if os.path.exists(path): os.remove(path)
return
os.makedirs(os.path.dirname(path) or ".", exist_ok=True)
with open(path, "w", encoding="utf-8") as f:
f.write(text.strip("\n") + "\n")
def run(cmd, expect_error=False, allow_error=False, env=None):
print("\n$ " + cmd, flush=True)
rc = subprocess.run(cmd, shell=True, env=dict(os.environ, **(env or {}))).returncode
if rc != 0 and not (expect_error or allow_error):
raise SystemExit("stopped: this command was expected to succeed")
# Step 0: the emulator config (mount it as ServiceBus_Emulator/ConfigFiles/Config.json, then start the emulator)
write('Config.json', """\
{
"UserConfig": {
"Namespaces": [
{
"Name": "sbemulatorns",
"Queues": [
{
"Name": "warehouse-inbox",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"RequiresDuplicateDetection": false,
"RequiresSession": false
}
},
{
"Name": "wh-east",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"RequiresDuplicateDetection": false,
"RequiresSession": false
}
},
{
"Name": "wh-west",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"RequiresDuplicateDetection": false,
"RequiresSession": false
}
},
{
"Name": "wh-express",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"RequiresDuplicateDetection": false,
"RequiresSession": false
}
},
{
"Name": "wh-unrouted",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"RequiresDuplicateDetection": false,
"RequiresSession": false
}
}
],
"Topics": [
{
"Name": "order-events",
"Properties": {
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"RequiresDuplicateDetection": false
},
"Subscriptions": [
{
"Name": "billing",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"RequiresSession": false
}
},
{
"Name": "shipping",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"RequiresSession": false
}
},
{
"Name": "email",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"RequiresSession": false
}
},
{
"Name": "big-refunds",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"RequiresSession": false
},
"Rules": [
{
"Name": "big-only",
"Properties": {
"FilterType": "Sql",
"SqlFilter": {
"SqlExpression": "sys.Label = 'order.refunded' AND amount >= 100"
}
}
}
]
},
{
"Name": "eu-refunds",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"RequiresSession": false
},
"Rules": [
{
"Name": "eu-only",
"Properties": {
"FilterType": "Correlation",
"CorrelationFilter": {
"Label": "order.refunded",
"Properties": {
"region": "eu"
}
}
}
}
]
},
{
"Name": "audit",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"RequiresSession": false
}
},
{
"Name": "two-rules",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"RequiresSession": false
},
"Rules": [
{
"Name": "match-all",
"Properties": {
"FilterType": "Sql",
"SqlFilter": {
"SqlExpression": "1=1"
}
}
},
{
"Name": "big-only",
"Properties": {
"FilterType": "Sql",
"SqlFilter": {
"SqlExpression": "sys.Label = 'order.refunded' AND amount >= 100"
}
}
}
]
},
{
"Name": "paused",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"RequiresSession": false
},
"Rules": [
{
"Name": "match-none",
"Properties": {
"FilterType": "Sql",
"SqlFilter": {
"SqlExpression": "1=0"
}
}
}
]
},
{
"Name": "refund-desk",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"RequiresSession": false
},
"Rules": [
{
"Name": "high-value",
"Properties": {
"FilterType": "Sql",
"SqlFilter": {
"SqlExpression": "sys.Label = 'order.refunded' AND amount >= 100"
},
"Action": {
"SqlExpression": "SET priority = 'high'"
}
}
},
{
"Name": "normal-value",
"Properties": {
"FilterType": "Sql",
"SqlFilter": {
"SqlExpression": "sys.Label = 'order.refunded' AND amount < 100"
},
"Action": {
"SqlExpression": "SET priority = 'normal'"
}
}
}
]
},
{
"Name": "warehouse",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "warehouse-inbox",
"RequiresSession": false
},
"Rules": [
{
"Name": "placed-only",
"Properties": {
"FilterType": "Sql",
"SqlFilter": {
"SqlExpression": "sys.Label = 'order.placed'"
}
}
}
]
}
]
},
{
"Name": "orders",
"Properties": {
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"RequiresDuplicateDetection": false
},
"Subscriptions": [
{
"Name": "east",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "wh-east",
"RequiresSession": false
},
"Rules": [
{
"Name": "east-only",
"Properties": {
"FilterType": "Correlation",
"CorrelationFilter": {
"Properties": {
"region": "east"
}
}
}
}
]
},
{
"Name": "west",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "wh-west",
"RequiresSession": false
},
"Rules": [
{
"Name": "west-only",
"Properties": {
"FilterType": "Correlation",
"CorrelationFilter": {
"Properties": {
"region": "west"
}
}
}
}
]
},
{
"Name": "express",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "wh-express",
"RequiresSession": false
},
"Rules": [
{
"Name": "express-only",
"Properties": {
"FilterType": "Correlation",
"CorrelationFilter": {
"Properties": {
"priority": "express"
}
}
}
}
]
},
{
"Name": "unrouted",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "wh-unrouted",
"RequiresSession": false
},
"Rules": [
{
"Name": "no-warehouse",
"Properties": {
"FilterType": "Sql",
"SqlFilter": {
"SqlExpression": "region IS NULL OR region NOT IN ('east', 'west')"
}
}
}
]
}
]
}
]
}
],
"Logging": {
"Type": "File"
}
}
}
""")
# Step 0: the starting files
write('send.py', """\
import os
from azure.servicebus import ServiceBusClient, ServiceBusMessage
EVENTS = [("o-1", "order.placed", "eu", 120),
("o-2", "order.placed", "us", 40),
("o-3", "order.refunded", "eu", 150),
("o-4", "order.refunded", "us", 30),
("o-5", "order.refunded", "us", 200),
("o-6", "order.shipped", "eu", 120)]
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
with client, client.get_topic_sender("order-events") as sender:
sender.send_messages([ServiceBusMessage(
f"{mid} {subject}", message_id=mid, subject=subject,
application_properties={"region": region, "amount": amount})
for mid, subject, region, amount in EVENTS])
print("published", len(EVENTS), "events to topic order-events")
""")
write('read.py', """\
import os, sys
from azure.servicebus import ServiceBusClient
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
with client:
for sub in sys.argv[1:]: # subscription names
got = []
with client.get_subscription_receiver(
"order-events", sub, max_wait_time=2) as receiver:
while batch := receiver.receive_messages(
max_message_count=20, max_wait_time=2):
for msg in batch:
got.append(msg.message_id)
receiver.complete_message(msg) # this copy only
print(f"{sub:<15}got", sorted(got))
""")
# Step 1: One publish to the topic, three subscriptions each read their own copy
run('python send.py')
run('python read.py billing shipping email')
# Step 2: A SQL filter and a correlation filter on the same six events
run('python read.py big-refunds eu-refunds')
# Step 3: No rules (default rule), a leftover match-all rule, and a false filter
run('python read.py audit two-rules paused')
# Step 4: Two rules with SQL actions tag each refund with a priority
write('tag.py', """\
import os
from azure.servicebus import ServiceBusClient
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
with client, client.get_subscription_receiver(
"order-events", "refund-desk", max_wait_time=2) as receiver:
msgs = []
while batch := receiver.receive_messages(max_message_count=20,
max_wait_time=2):
msgs += batch
for msg in sorted(msgs, key=lambda m: m.message_id):
raw = msg.application_properties # bytes keys
props = {k.decode(): v.decode() if isinstance(v, bytes)
else v for k, v in raw.items()}
print(msg.message_id, dict(sorted(props.items())))
receiver.complete_message(msg)
""")
run('python tag.py')
# Step 5: A subscription forwards placed orders to a queue; a receiver on the source fails
write('forward.py', """\
import os
from azure.servicebus import ServiceBusClient
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
with client:
with client.get_queue_receiver("warehouse-inbox",
max_wait_time=2) as receiver:
msgs = receiver.receive_messages(max_message_count=20,
max_wait_time=2)
for msg in msgs:
receiver.complete_message(msg)
print("warehouse-inbox queue got",
sorted(m.message_id for m in msgs))
try: # the forwarding source
with client.get_subscription_receiver(
"order-events", "warehouse", max_wait_time=2) as r:
r.receive_messages(max_wait_time=2)
except Exception as e:
print(type(e).__name__)
print(str(e).split(" TrackingId")[0])
""")
run('python forward.py')
# Step 6: Routing orders to warehouse queues by region and priority
write('publish.py', """\
import os
from azure.servicebus import ServiceBusClient, ServiceBusMessage
ORDERS = [("o-11", "east", "standard"), ("o-12", "west", "express"),
("o-13", "east", "express"), ("o-14", "west", "standard"),
("o-15", "north", "standard")]
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
with client, client.get_topic_sender("orders") as sender:
for order_id, region, priority in ORDERS:
sender.send_messages(ServiceBusMessage(
order_id, message_id=order_id,
application_properties={"region": region, # routing
"priority": priority})) # only
print("published", len(ORDERS), "orders; no warehouse named")
""")
write('warehouses.py', """\
import os
from azure.servicebus import ServiceBusClient
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
with client:
for queue in ["wh-east", "wh-west", "wh-express", "wh-unrouted"]:
got = []
with client.get_queue_receiver(queue, max_wait_time=3) as r:
while batch := r.receive_messages(max_message_count=20,
max_wait_time=3):
for msg in batch:
got.append(msg.message_id)
r.complete_message(msg)
print(f"{queue:<12}", sorted(got))
""")
run('python publish.py')
run('python warehouses.py')
Part 4: Peek-Lock, Settlement and Redelivery
The sample data on this page is invented for teaching. Every terminal output is the real output of the command above it, run on the official Azure Service Bus emulator (2.0.1) with SQL Server and azure-servicebus 7.15.0. Features the emulator does not support are shown as code. Part of the Real Time Azure Service Bus Interview Questions and Answers series on TechBrothersIT. Features, limits and their status (GA or preview) are current as of October 2026; check the official Azure Service Bus documentation (learn.microsoft.com/azure/service-bus-messaging) for anything you quote in an interview.
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.