In this post we are going to cover real-world manager scenarios for Azure Service Bus. These are 7 real Service Bus interview questions for data engineers, asked the way they come up on a real project, and the last one is a scenario: in the final round, the hiring manager asks you to design the messaging backbone for an online retailer on Azure.
These are the questions a manager asks when something is already wrong: a queue that keeps growing, duplicate emails, lost locks, a cost spike, a migration, a partner that needs access. Interviewers ask them because they show whether you can investigate before you change things, and whether you can pull everything from this series into one design.
Part 10 of the Real Time Azure Service Bus Interview Questions and Answers series on TechBrothersIT.
- The short answer
- Let's set up the project
- Question 1: The queue's active message count grows every evening. How do you investigate and fix it?
- Question 2: Customers got the same email twice. Where do duplicates come from, and how do you stop them?
- Question 3: Consumers keep losing their locks and messages are processed twice. How do you fix it?
- Question 4: Service Bus costs jumped this month. Where do you look first?
- Question 5: A team wants to move from RabbitMQ to Service Bus. How do you plan the migration?
- Question 6: Messages from a partner must reach your queue safely. How do you set it up?
- Scenario 7: In the final round, the hiring manager asks you to design the messaging backbone for an online retailer on Azure. What is your design?
- The full script
The short answer
Count the backlog and its delivery counts before you scale; stop duplicates at both ends with a stable message ID plus duplicate detection and an idempotent consumer; renew locks with AutoLockRenewer; read cost by meter; bridge RabbitMQ with Shovel; give partners a send-only door; and build the backbone on topics, outbox and idempotency.
Let's set up the project
We run Microsoft's official Service Bus emulator locally with SQL Server, and small Python scripts that use the azure-servicebus client, so every output is real. Features the emulator doesn't support are shown as code. Config.json creates seven queues: orders, emails and emails-dedup with a 5 second lock, reports and reports-renew with a 5 second lock too, partner-orders with duplicate detection, and orders-in. It also creates the order-events topic with duplicate detection and three subscriptions: billing and shipping, each with a correlation filter on the subject order.paid, and email, which gets everything.
{
"UserConfig": {
"Namespaces": [
{
"Name": "sbemulatorns",
"Queues": [
{
"Name": "orders",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"RequiresDuplicateDetection": false,
"RequiresSession": false
}
},
{
"Name": "emails",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"LockDuration": "PT5S",
"MaxDeliveryCount": 3,
"RequiresDuplicateDetection": false,
"RequiresSession": false
}
},
{
"Name": "emails-dedup",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"LockDuration": "PT5S",
"MaxDeliveryCount": 3,
"RequiresDuplicateDetection": true,
"RequiresSession": false
}
},
{
"Name": "reports",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"LockDuration": "PT5S",
"MaxDeliveryCount": 5,
"RequiresDuplicateDetection": false,
"RequiresSession": false
}
},
{
"Name": "reports-renew",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"LockDuration": "PT5S",
"MaxDeliveryCount": 3,
"RequiresDuplicateDetection": false,
"RequiresSession": false
}
},
{
"Name": "partner-orders",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"RequiresDuplicateDetection": true,
"RequiresSession": false
}
},
{
"Name": "orders-in",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"RequiresDuplicateDetection": false,
"RequiresSession": false
}
}
],
"Topics": [
{
"Name": "order-events",
"Properties": {
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"RequiresDuplicateDetection": true
},
"Subscriptions": [
{
"Name": "billing",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"RequiresSession": false
},
"Rules": [
{
"Name": "paid-only",
"Properties": {
"FilterType": "Correlation",
"CorrelationFilter": {
"Label": "order.paid"
}
}
}
]
},
{
"Name": "shipping",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"RequiresSession": false
},
"Rules": [
{
"Name": "paid-only",
"Properties": {
"FilterType": "Correlation",
"CorrelationFilter": {
"Label": "order.paid"
}
}
}
]
},
{
"Name": "email",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"RequiresSession": false
}
}
]
}
]
}
],
"Logging": {
"Type": "File"
}
}
}Question 1: The queue's active message count grows every evening. How do you investigate and fix it?
First find out which kind of growth it is: arrivals outpacing consumers, or the same messages coming back. In Azure, these are the metrics to compare:
- IncomingMessages vs OutgoingMessages - arrivals outpacing consumers?
- ActiveMessages - the backlog trend, per entity
- CompleteMessage vs AbandonMessage - the same messages coming back?
- ThrottledRequests - Standard credits; Premium: CPU and memory
The emulator has no runtime counts, so we count with peek. Thirty orders arrive each evening, and one consumer gets through ten. We run two evenings: python evening.py 1001 30, python workers.py 1 10 and python backlog.py, then the same with 1031.
import os, sys
from azure.servicebus import ServiceBusClient, ServiceBusMessage
first, count = int(sys.argv[1]), int(sys.argv[2])
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
with client, client.get_queue_sender("orders") as sender:
sender.send_messages([ServiceBusMessage(f"order {n}",
message_id=f"order-{n}")
for n in range(first, first + count)])
print(f"evening: {count} new orders")import collections, os
from azure.servicebus import ServiceBusClient
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
seen, seq = [], 1
with client, client.get_queue_receiver("orders") as receiver:
while batch := receiver.peek_messages(250, sequence_number=seq):
seen += batch # peek: no lock, no side effect
seq = batch[-1].sequence_number + 1
counts = collections.Counter(m.delivery_count for m in seen)
print("backlog:", len(seen), "| delivery counts:", dict(counts))import os, sys, threading, time
from azure.servicebus import ServiceBusClient
consumers, capacity = int(sys.argv[1]), int(sys.argv[2])
done = []
def consumer(): # one client per thread
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
count = 0
with client, client.get_queue_receiver("orders") as receiver:
while count < capacity: # what it gets through in an evening
msgs = receiver.receive_messages(max_wait_time=5)
if not msgs:
break
time.sleep(0.1) # the work for one order
receiver.complete_message(msgs[0])
count += 1
done.append(count)
pool = [threading.Thread(target=consumer) for _ in range(consumers)]
for t in pool: t.start()
for t in pool: t.join()
print(consumers, "consumer(s), processed each:", sorted(done))$ python evening.py 1001 30
evening: 30 new orders
$ python workers.py 1 10
1 consumer(s), processed each: [10]
$ python backlog.py
backlog: 20 | delivery counts: {0: 20}
$ python evening.py 1031 30
evening: 30 new orders
$ python workers.py 1 10
1 consumer(s), processed each: [10]
$ python backlog.py
backlog: 40 | delivery counts: {0: 40}The backlog is 20, then 40, and every delivery count is zero: {0: 40}. Nothing is retried. That is a capacity problem. If delivery counts were above zero, it would be a different problem: lost locks, crashes or abandons. More consumers won't fix that, and question three does.
For capacity, add competing consumers. The third evening runs four consumers on the same queue: python evening.py 1061 30, python workers.py 4 10 and python backlog.py.
$ python evening.py 1061 30
evening: 30 new orders
$ python workers.py 4 10
4 consumer(s), processed each: [10, 10, 10, 10]
$ python backlog.py
backlog: 30 | delivery counts: {0: 30}Each consumer processes 10, [10, 10, 10, 10], and the backlog drops to 30. Also make the work itself faster, like batching database writes.
On Standard, throttled requests mean the credit limit. On Premium, check CPU and memory and add messaging units. And keep prefetch modest: prefetched messages are already locked.
Question 2: Customers got the same email twice. Where do duplicates come from, and how do you stop them?
Service Bus delivers at least once, so duplicates are expected. They come from two places:
- A publisher retry - the send worked, the ack was lost, so it is sent again
- A redelivery - a crash or a lost lock before complete
Let's create both, for real. The publisher sends order 1002, times out and sends it again, with no message ID of its own. The consumer queues the email for 1001, then crashes before completing. We run python publish.py emails no-id, CRASH=1 python send_emails.py emails plain, python send_emails.py emails plain and python report.py emails.
import os, sys
from azure.servicebus import ServiceBusClient, ServiceBusMessage
queue, stable_id = sys.argv[1], sys.argv[2] == "with-id"
def confirmation(order):
return ServiceBusMessage(f"confirmation for order {order}",
message_id=f"confirm-{order}" if stable_id else None)
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
with client, client.get_queue_sender(queue) as sender:
sender.send_messages(confirmation(1001))
sender.send_messages(confirmation(1002))
sender.send_messages(confirmation(1002)) # timeout, so a retry
print("sent: 1001, 1002, then 1002 again after a timeout")import os, sqlite3, sys
from azure.servicebus import ServiceBusClient
queue, mode = sys.argv[1], sys.argv[2] # plain or idempotent
db = sqlite3.connect("mail.db")
db.execute("CREATE TABLE IF NOT EXISTS outbox (queue, email)")
db.execute("CREATE TABLE IF NOT EXISTS inbox"
" (message_id TEXT PRIMARY KEY)")
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
with client, client.get_queue_receiver(queue, max_wait_time=8) as rx:
for msg in rx:
with db: # one transaction for both rows
new = mode == "plain" or db.execute(
"INSERT OR IGNORE INTO inbox VALUES (?)",
(msg.message_id,)).rowcount
if new: # the mailer sends from outbox
db.execute("INSERT INTO outbox VALUES (?, ?)",
(queue, str(msg)))
status = "email queued" if new else "skipped: already sent"
print(msg, "|", status, "| delivery", msg.delivery_count)
if os.environ.get("CRASH"):
print("crash before complete"); os._exit(1)
rx.complete_message(msg)import sqlite3, sys
db = sqlite3.connect("mail.db")
for email, n in db.execute("SELECT email, COUNT(*) FROM outbox"
" WHERE queue = ? GROUP BY email"
" ORDER BY email", (sys.argv[1],)):
print(f"{email}: {n} email{'s' if n > 1 else ''}")$ python publish.py emails no-id
sent: 1001, 1002, then 1002 again after a timeout
$ CRASH=1 python send_emails.py emails plain
confirmation for order 1001 | email queued | delivery 0
crash before complete
$ python send_emails.py emails plain
confirmation for order 1002 | email queued | delivery 0
confirmation for order 1002 | email queued | delivery 0
confirmation for order 1001 | email queued | delivery 1
$ python report.py emails
confirmation for order 1001: 2 emails
confirmation for order 1002: 2 emailsThe retry gives two copies of 1002, both email queued. 1001 comes back after the lock expires with delivery 1 and is queued again. The report shows 2 emails for each order.
Now both fixes. At the publisher, a stable message ID from the business key, confirm-1002, on a queue with duplicate detection. Python sets none for you. At the consumer, write the message ID to an inbox table in the same transaction as the email row, then complete. Same retry, same crash: python publish.py emails-dedup with-id, CRASH=1 python send_emails.py emails-dedup idempotent, python send_emails.py emails-dedup idempotent and python report.py emails-dedup.
$ python publish.py emails-dedup with-id
sent: 1001, 1002, then 1002 again after a timeout
$ CRASH=1 python send_emails.py emails-dedup idempotent
confirmation for order 1001 | email queued | delivery 0
crash before complete
$ python send_emails.py emails-dedup idempotent
confirmation for order 1002 | email queued | delivery 0
confirmation for order 1001 | skipped: already sent | delivery 1
$ python report.py emails-dedup
confirmation for order 1001: 1 email
confirmation for order 1002: 1 emailThe retry is dropped, and the redelivery of 1001 prints skipped: already sent. The report shows 1 email each.
Duplicate detection only drops repeated sends inside its window, 10 minutes by default, and it can't stop a redelivery.
Question 3: Consumers keep losing their locks and messages are processed twice. How do you fix it?
A peek-lock lasts the queue's lock duration: 1 minute by default, 5 at most. When the work takes longer, the lock expires, the message is released, and another consumer starts the same work.
Let's see it. The lock here is 5 seconds and the report takes 8. First without renewal on reports, then with Python's AutoLockRenewer on reports-renew, which renews in the background while we work, up to a maximum renewal duration, here 60 seconds. We run python lock.py reports plain and python lock.py reports-renew renew.
import os, sys, time
from azure.servicebus import (AutoLockRenewer, ServiceBusClient,
ServiceBusMessage)
from azure.servicebus.exceptions import MessageLockLostError
queue, renew = sys.argv[1], sys.argv[2] == "renew"
alr = AutoLockRenewer(max_lock_renewal_duration=60) if renew else None
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
with client:
with client.get_queue_sender(queue) as sender:
sender.send_messages(ServiceBusMessage("report 501"))
with client.get_queue_receiver(queue,
auto_lock_renewer=alr) as rx:
for attempt in (1, 2):
msg = rx.receive_messages(max_wait_time=10)[0]
print("processing", msg, "| delivery", msg.delivery_count)
time.sleep(8) # the work: 8 s, lock: 5 s
try:
rx.complete_message(msg)
print("completed"); break
except MessageLockLostError as e:
print(type(e).__name__ + ":", e)
if alr: # inside the receiver block
alr.close()$ python lock.py reports plain
processing report 501 | delivery 0
MessageLockLostError: The lock on the message lock has expired.
processing report 501 | delivery 1
MessageLockLostError: The lock on the message lock has expired.
$ python lock.py reports-renew renew
processing report 501 | delivery 0
completedWithout renewal, complete fails with MessageLockLostError: The lock on the message lock has expired. The report comes back with delivery 1, is processed again, and fails again. Max delivery count will dead-letter it. With the renewer, the same eight second job prints completed on the first try, delivery 0.
Also set the lock duration above your normal processing time, and keep prefetch low, because buffered messages are already locked.
| Setting | Default |
|---|---|
| Lock duration (Azure) | 1 min, max 5 min |
| max_lock_renewal_duration | 300 s |
| Functions maxAutoLockRenewalDuration | 00:05:00 |
| prefetch_count (Python) | 0: off |
A lock is lost if the connection drops, even with renewal. So the handler must still be idempotent, like question two.
Question 4: Service Bus costs jumped this month. Where do you look first?
Start in cost analysis, by meter, because each tier bills differently:
| Tier | You pay for | Look at first |
|---|---|---|
| Basic | operations, per million; no base charge | operations per entity |
| Standard | base charge + tiered operations + brokered connections over the included amount | operations, connections |
| Premium | messaging units per hour; operations and connections not charged | MU history, geo replicas |
On Basic and Standard, an operation is any API call, management included. Let's look at what multiplies them:
- Operation = any API call - sends, receives, management
- Over 64 KB - billed in 64 KB units
- Each subscription copy counts - and each auto-forward hop is one more
- Prefetch - no change in billable operations
So a new subscription on a busy topic, or a forwarding chain, multiplies operations without anyone sending more.
On Premium, look at the messaging unit history:
- Premium: hourly, at the highest MU count - scaling down within the hour doesn't reduce it
- Geo-replication - replica MUs + data replicated per GB
My first checks are operations by entity, connections, new subscriptions or forwards, and the messaging unit history. There are no prices here: they vary by region and change.
When operations jump and traffic didn't, look for a new subscription or a new forwarding hop first.
Question 5: A team wants to move from RabbitMQ to Service Bus. How do you plan the migration?
Be clear about sources first. I know of no Microsoft RabbitMQ migration guide. Microsoft documents a bridge from RabbitMQ to Service Bus with the Shovel plugin. The concept mapping below is my own practice, not a Microsoft table.
| RabbitMQ | Service Bus (my mapping) |
|---|---|
| queue | queue |
| exchange + bindings | topic + subscriptions |
| binding / routing key | rule: correlation filter (exact), SQL filter with LIKE (simple wildcards) |
| ack / nack | complete / abandon (peek-lock) |
| dead letter exchange | built-in dead-letter queue per queue and subscription |
A binding becomes a rule: a correlation filter for exact keys, a SQL filter with LIKE for simple wildcards.
The protocol changes too. Most RabbitMQ clients speak AMQP 0.9.1, and Service Bus speaks AMQP 1.0, so client libraries change. Shovel bridges the two, sending to Service Bus with a send-only SAS policy:
# on RabbitMQ: the Shovel plugin with its management UI
rabbitmq-plugins enable rabbitmq_shovel_management
# source: the RabbitMQ queue (AMQP 0.9.1)
# destination: Service Bus (AMQP 1.0), a Send-only SAS policy
amqps://send-only:<KEY>@tbshop.servicebus.windows.net:5671/?sasl=plainThe plan, in order:
- Entities as code - topics, subscriptions, rules in Bicep
- Bridge with Shovel - both sides run
- Move consumers, then publishers
- Drain RabbitMQ - then remove the bridge
Consumers see duplicates during a cutover, so make them idempotent first. And request-reply needs more than a shovel, for example a messaging bridge like NServiceBus.
Question 6: Messages from a partner must reach your queue safely. How do you set it up?
Give the partner the smallest door: their own queue, send rights only, nothing else in the namespace, and validate their messages before anything reaches your internal queues.
Best is the partner's multitenant Entra app, consented into your tenant, with Azure Service Bus Data Sender on that queue and no shared secret. Otherwise, a SAS policy on the queue with only the Send right:
# best: the partner's multitenant Entra app, Data Sender on one queue
az ad sp create --id "$PARTNER_APP_ID" # consent in your tenant
QUEUE_ID=$(az servicebus queue show --resource-group rg-shop \
--namespace-name tbshop-bus --name partner-orders \
--query id --output tsv)
az role assignment create --assignee "$PARTNER_APP_ID" \
--role "Azure Service Bus Data Sender" --scope "$QUEUE_ID"
# otherwise: a Send-only SAS policy on that queue only
az servicebus queue authorization-rule create -g rg-shop \
--namespace-name tbshop-bus --queue-name partner-orders \
--name partner-send --rights SendEven then, keep the key. You sign a token with an expiry, and the partner's connection string holds only the token. The partner sends three messages, and the third is a retry of the first. Then our validator reads the queue. The emulator checks no signatures, so this shows the client side. We run python make_token.py, python partner_send.py and python validate.py.
import base64, hashlib, hmac, urllib.parse
def sas_token(uri, key_name, key, expiry):
sr = urllib.parse.quote_plus(uri)
sig = hmac.new(key.encode(), f"{sr}\n{expiry}".encode(),
hashlib.sha256).digest()
sig = urllib.parse.quote_plus(base64.b64encode(sig))
return (f"SharedAccessSignature sr={sr}&sig={sig}"
f"&se={expiry}&skn={key_name}")
# you sign with the queue's Send-only policy; the key stays with you
token = sas_token("sb://localhost/partner-orders", "partner-send",
"PARTNER_SEND_KEY", 1798761600) # 2027-01-01 UTC
conn = (f"Endpoint=sb://localhost;SharedAccessSignature={token};"
"EntityPath=partner-orders;UseDevelopmentEmulator=true")
open("partner.conn", "w").write(conn)
print("partner gets: endpoint, token, entity path")
print("key in it:", "SharedAccessKey=" in conn)
print("token expires (epoch seconds):", token.split("se=")[1][:10])import json
from azure.servicebus import ServiceBusClient, ServiceBusMessage
conn = open("partner.conn").read() # a token, no key
orders = [{"order_id": "P-1001", "amount": 120},
{"order_id": "P-1002"}, # no amount
{"order_id": "P-1001", "amount": 120}] # a retry
with ServiceBusClient.from_connection_string(conn) as client:
with client.get_queue_sender("partner-orders") as sender:
for order in orders:
sender.send_messages(ServiceBusMessage(
json.dumps(order), message_id=order["order_id"],
content_type="application/json"))
print("partner sent", len(orders), "messages")import json, os
from azure.servicebus import (ServiceBusClient, ServiceBusMessage,
ServiceBusSubQueue)
def valid(body):
try:
return type(json.loads(body).get("amount")) in (int, float)
except ValueError:
return False
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
with client:
rx = client.get_queue_receiver("partner-orders", max_wait_time=3)
with rx, client.get_queue_sender("orders-in") as orders:
for msg in rx:
if valid(str(msg)):
orders.send_messages(ServiceBusMessage(
str(msg), message_id=msg.message_id))
rx.complete_message(msg)
print(msg.message_id, "valid: sent to orders-in")
else:
rx.dead_letter_message(msg, reason="SchemaInvalid",
error_description="amount missing")
print(msg.message_id, "invalid: dead-lettered")
with client.get_queue_receiver("partner-orders",
sub_queue=ServiceBusSubQueue.DEAD_LETTER) as dlq:
for m in dlq.peek_messages(10, sequence_number=1):
print("DLQ:", m.message_id, m.dead_letter_reason, "-",
m.dead_letter_error_description)$ python make_token.py
partner gets: endpoint, token, entity path
key in it: False
token expires (epoch seconds): 1798761600
$ python partner_send.py
partner sent 3 messages
$ python validate.py
P-1001 valid: sent to orders-in
P-1002 invalid: dead-lettered
DLQ: P-1002 SchemaInvalid - amount missingThe partner gets endpoint, token, entity path, and key in it: False. The token expires at 1798761600. The partner sends 3 messages, but partner-orders has duplicate detection, so the retry is dropped and the validator reads two. P-1001 is valid and goes on to orders-in. P-1002 has no amount and is dead-lettered with a reason: SchemaInvalid - amount missing.
If the namespace isn't private, add the IP firewall with the partner's ranges.
Re-sign tokens with the secondary key before you regenerate the primary.
Scenario 7: In the final round, the hiring manager asks you to design the messaging backbone for an online retailer on Azure. What is your design?
I start from the flows. Commands with one owner, like take a payment, go to queues. Business events, like order paid, go to one topic, with a filtered subscription per team, so new teams never touch the publisher.
- Order service - DB + outbox table
order-events- topic, duplicate detectionbilling- filter: order.paidshipping- filter: order.paid, sessions by orderemail- everythingpaymentsqueue - a command, one owner
Let's run the core, for real. The order service publishes five events, one of them a retry of order.paid, and each subscription is peeked. We run python backbone.py.
import os
from azure.servicebus import ServiceBusClient, ServiceBusMessage
events = [("5001", "order.placed"), ("5001", "order.paid"),
("5001", "order.paid"), # a publisher retry
("5002", "order.placed"), ("5001", "order.shipped")]
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
with client:
with client.get_topic_sender("order-events") as sender:
for order, event in events: # relayed from the outbox
sender.send_messages(ServiceBusMessage(
f"{order} {event}", subject=event,
message_id=f"{order}-{event}")) # the event ID
counts = {}
for sub in ["billing", "shipping", "email"]:
rx = client.get_subscription_receiver("order-events", sub)
with rx:
got = rx.peek_messages(20, sequence_number=1)
counts[sub] = len(got)
for m in got:
print(f"{sub:9}", m)
print("published", len(events), "| per subscription:", counts)$ python backbone.py
billing 5001 order.paid
shipping 5001 order.paid
email 5001 order.placed
email 5001 order.paid
email 5002 order.placed
email 5001 order.shipped
published 5 | per subscription: {'billing': 1, 'shipping': 1, 'email': 4}Billing and shipping each get one 5001 order.paid, and email gets everything: {'billing': 1, 'shipping': 1, 'email': 4}. Duplicate detection dropped the retry.
Reliability comes from the patterns in this series:
- Outbox - events written in the same DB transaction
- Event ID = message ID - duplicate detection on the topic
- Idempotent consumers - inbox table per service
- Dead-letter queues - plus alerts and a resubmit runbook
- Sessions by order ID - where order matters
Then the platform:
- Premium - for predictable performance, private endpoints and Geo-replication, or Standard for a smaller shop
- Entra ID + managed identities - local auth disabled, all in Bicep
Operations close it out: alerts on active and dead-lettered messages, throttling and CPU, a dead-letter resubmit runbook, and failover drills.
A backbone that survives outages, retries and duplicates.
Key points to remember
- Count the backlog and its delivery counts before you add consumers
- Duplicates: stable message ID + duplicate detection, and an inbox in the consumer
- Long work: AutoLockRenewer, and a lock duration above normal processing time
- Cost by meter: operations on Basic / Standard, messaging units on Premium
- RabbitMQ: Shovel bridge; partners: a send-only door and a validator
The full script
This is a Python script that writes every file and runs every command above in order against the Service Bus emulator: start the emulator with SQL Server, run pip install azure-servicebus first, set SERVICEBUS_CONNECTION_STRING, and run it in an empty folder.
Show the full script that writes the files and runs every step
# ========================================================================
# Service Bus Interview Q&A #10 - Real-world manager scenarios
# TechBrothersIT. Names and sample data are invented for teaching.
# Every step below was run on the official Azure Service Bus emulator (Docker image, version 2.0.1) with
# SQL Server 2022 and azure-servicebus 7.15.0; the outputs in the video and the blog post are the real outputs.
# To run it yourself:
# 1. Start the emulator with Microsoft's installer (github.com/Azure/azure-service-bus-emulator-installer),
# using the Config.json this script writes first (Step 0) as the emulator's config file; it holds every
# queue, topic, subscription, rule and setting (lock duration, max delivery count, duplicate detection...)
# the steps depend on.
# 2. pip install azure-servicebus
# 3. Set SERVICEBUS_CONNECTION_STRING to the emulator connection string (Endpoint=sb://localhost;
# SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=SAS_KEY_VALUE;UseDevelopmentEmulator=true;)
# then run python ServiceBus_Interview_QA_10_Code.py.
# Run it in an empty folder and on a freshly started emulator: it creates its files (and any SQLite
# databases) there, and a second run on old files or old messages gives different output.
# Steps that restart the emulator say so in a comment: stop and start it by hand at that point.
# ========================================================================
import os, subprocess, time
def write(path, text):
if text is None:
if os.path.exists(path): os.remove(path)
return
os.makedirs(os.path.dirname(path) or ".", exist_ok=True)
with open(path, "w", encoding="utf-8") as f:
f.write(text.strip("\n") + "\n")
def run(cmd, expect_error=False, allow_error=False, env=None):
print("\n$ " + cmd, flush=True)
rc = subprocess.run(cmd, shell=True, env=dict(os.environ, **(env or {}))).returncode
if rc != 0 and not (expect_error or allow_error):
raise SystemExit("stopped: this command was expected to succeed")
# Step 0: the emulator config (mount it as ServiceBus_Emulator/ConfigFiles/Config.json, then start the emulator)
write('Config.json', """\
{
"UserConfig": {
"Namespaces": [
{
"Name": "sbemulatorns",
"Queues": [
{
"Name": "orders",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"RequiresDuplicateDetection": false,
"RequiresSession": false
}
},
{
"Name": "emails",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"LockDuration": "PT5S",
"MaxDeliveryCount": 3,
"RequiresDuplicateDetection": false,
"RequiresSession": false
}
},
{
"Name": "emails-dedup",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"LockDuration": "PT5S",
"MaxDeliveryCount": 3,
"RequiresDuplicateDetection": true,
"RequiresSession": false
}
},
{
"Name": "reports",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"LockDuration": "PT5S",
"MaxDeliveryCount": 5,
"RequiresDuplicateDetection": false,
"RequiresSession": false
}
},
{
"Name": "reports-renew",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"LockDuration": "PT5S",
"MaxDeliveryCount": 3,
"RequiresDuplicateDetection": false,
"RequiresSession": false
}
},
{
"Name": "partner-orders",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"RequiresDuplicateDetection": true,
"RequiresSession": false
}
},
{
"Name": "orders-in",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"RequiresDuplicateDetection": false,
"RequiresSession": false
}
}
],
"Topics": [
{
"Name": "order-events",
"Properties": {
"DefaultMessageTimeToLive": "PT1H",
"DuplicateDetectionHistoryTimeWindow": "PT20S",
"RequiresDuplicateDetection": true
},
"Subscriptions": [
{
"Name": "billing",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"RequiresSession": false
},
"Rules": [
{
"Name": "paid-only",
"Properties": {
"FilterType": "Correlation",
"CorrelationFilter": {
"Label": "order.paid"
}
}
}
]
},
{
"Name": "shipping",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"RequiresSession": false
},
"Rules": [
{
"Name": "paid-only",
"Properties": {
"FilterType": "Correlation",
"CorrelationFilter": {
"Label": "order.paid"
}
}
}
]
},
{
"Name": "email",
"Properties": {
"DeadLetteringOnMessageExpiration": false,
"DefaultMessageTimeToLive": "PT1H",
"LockDuration": "PT1M",
"MaxDeliveryCount": 3,
"ForwardDeadLetteredMessagesTo": "",
"ForwardTo": "",
"RequiresSession": false
}
}
]
}
]
}
],
"Logging": {
"Type": "File"
}
}
}
""")
# Step 1: Two evenings: 30 orders arrive, one consumer gets through 10; the backlog counted with peek
write('evening.py', """\
import os, sys
from azure.servicebus import ServiceBusClient, ServiceBusMessage
first, count = int(sys.argv[1]), int(sys.argv[2])
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
with client, client.get_queue_sender("orders") as sender:
sender.send_messages([ServiceBusMessage(f"order {n}",
message_id=f"order-{n}")
for n in range(first, first + count)])
print(f"evening: {count} new orders")
""")
write('backlog.py', """\
import collections, os
from azure.servicebus import ServiceBusClient
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
seen, seq = [], 1
with client, client.get_queue_receiver("orders") as receiver:
while batch := receiver.peek_messages(250, sequence_number=seq):
seen += batch # peek: no lock, no side effect
seq = batch[-1].sequence_number + 1
counts = collections.Counter(m.delivery_count for m in seen)
print("backlog:", len(seen), "| delivery counts:", dict(counts))
""")
write('workers.py', """\
import os, sys, threading, time
from azure.servicebus import ServiceBusClient
consumers, capacity = int(sys.argv[1]), int(sys.argv[2])
done = []
def consumer(): # one client per thread
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
count = 0
with client, client.get_queue_receiver("orders") as receiver:
while count < capacity: # what it gets through in an evening
msgs = receiver.receive_messages(max_wait_time=5)
if not msgs:
break
time.sleep(0.1) # the work for one order
receiver.complete_message(msgs[0])
count += 1
done.append(count)
pool = [threading.Thread(target=consumer) for _ in range(consumers)]
for t in pool: t.start()
for t in pool: t.join()
print(consumers, "consumer(s), processed each:", sorted(done))
""")
run('python evening.py 1001 30')
run('python workers.py 1 10')
run('python backlog.py')
run('python evening.py 1031 30')
run('python workers.py 1 10')
run('python backlog.py')
# Step 2: The third evening with four competing consumers on the same queue
run('python evening.py 1061 30')
run('python workers.py 4 10')
run('python backlog.py')
# Step 3: Duplicate emails: a publisher retry without a message ID, and a crash before complete
write('publish.py', """\
import os, sys
from azure.servicebus import ServiceBusClient, ServiceBusMessage
queue, stable_id = sys.argv[1], sys.argv[2] == "with-id"
def confirmation(order):
return ServiceBusMessage(f"confirmation for order {order}",
message_id=f"confirm-{order}" if stable_id else None)
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
with client, client.get_queue_sender(queue) as sender:
sender.send_messages(confirmation(1001))
sender.send_messages(confirmation(1002))
sender.send_messages(confirmation(1002)) # timeout, so a retry
print("sent: 1001, 1002, then 1002 again after a timeout")
""")
write('send_emails.py', """\
import os, sqlite3, sys
from azure.servicebus import ServiceBusClient
queue, mode = sys.argv[1], sys.argv[2] # plain or idempotent
db = sqlite3.connect("mail.db")
db.execute("CREATE TABLE IF NOT EXISTS outbox (queue, email)")
db.execute("CREATE TABLE IF NOT EXISTS inbox"
" (message_id TEXT PRIMARY KEY)")
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
with client, client.get_queue_receiver(queue, max_wait_time=8) as rx:
for msg in rx:
with db: # one transaction for both rows
new = mode == "plain" or db.execute(
"INSERT OR IGNORE INTO inbox VALUES (?)",
(msg.message_id,)).rowcount
if new: # the mailer sends from outbox
db.execute("INSERT INTO outbox VALUES (?, ?)",
(queue, str(msg)))
status = "email queued" if new else "skipped: already sent"
print(msg, "|", status, "| delivery", msg.delivery_count)
if os.environ.get("CRASH"):
print("crash before complete"); os._exit(1)
rx.complete_message(msg)
""")
write('report.py', """\
import sqlite3, sys
db = sqlite3.connect("mail.db")
for email, n in db.execute("SELECT email, COUNT(*) FROM outbox"
" WHERE queue = ? GROUP BY email"
" ORDER BY email", (sys.argv[1],)):
print(f"{email}: {n} email{'s' if n > 1 else ''}")
""")
run('python publish.py emails no-id', allow_error=True)
run('CRASH=1 python send_emails.py emails plain', allow_error=True)
run('python send_emails.py emails plain', allow_error=True)
run('python report.py emails', allow_error=True)
# Step 4: The fix: a stable message ID with duplicate detection, and an inbox table in the consumer
run('python publish.py emails-dedup with-id', allow_error=True)
run('CRASH=1 python send_emails.py emails-dedup idempotent', allow_error=True)
run('python send_emails.py emails-dedup idempotent', allow_error=True)
run('python report.py emails-dedup', allow_error=True)
# Step 5: Work longer than the lock (5 s lock, 8 s of work), then the same job with AutoLockRenewer
write('lock.py', """\
import os, sys, time
from azure.servicebus import (AutoLockRenewer, ServiceBusClient,
ServiceBusMessage)
from azure.servicebus.exceptions import MessageLockLostError
queue, renew = sys.argv[1], sys.argv[2] == "renew"
alr = AutoLockRenewer(max_lock_renewal_duration=60) if renew else None
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
with client:
with client.get_queue_sender(queue) as sender:
sender.send_messages(ServiceBusMessage("report 501"))
with client.get_queue_receiver(queue,
auto_lock_renewer=alr) as rx:
for attempt in (1, 2):
msg = rx.receive_messages(max_wait_time=10)[0]
print("processing", msg, "| delivery", msg.delivery_count)
time.sleep(8) # the work: 8 s, lock: 5 s
try:
rx.complete_message(msg)
print("completed"); break
except MessageLockLostError as e:
print(type(e).__name__ + ":", e)
if alr: # inside the receiver block
alr.close()
""")
run('python lock.py reports plain')
run('python lock.py reports-renew renew')
# Step 6: The partner sends with a token-only connection string; a validator forwards or dead-letters
write('make_token.py', 'import base64, hashlib, hmac, urllib.parse\n\ndef sas_token(uri, key_name, key, expiry):\n sr = urllib.parse.quote_plus(uri)\n sig = hmac.new(key.encode(), f"{sr}\\n{expiry}".encode(),\n hashlib.sha256).digest()\n sig = urllib.parse.quote_plus(base64.b64encode(sig))\n return (f"SharedAccessSignature sr={sr}&sig={sig}"\n f"&se={expiry}&skn={key_name}")\n\n# you sign with the queue\'s Send-only policy; the key stays with you\ntoken = sas_token("sb://localhost/partner-orders", "partner-send",\n "PARTNER_SEND_KEY", 1798761600) # 2027-01-01 UTC\nconn = (f"Endpoint=sb://localhost;SharedAccessSignature={token};"\n "EntityPath=partner-orders;UseDevelopmentEmulator=true")\nopen("partner.conn", "w").write(conn)\nprint("partner gets: endpoint, token, entity path")\nprint("key in it:", "SharedAccessKey=" in conn)\nprint("token expires (epoch seconds):", token.split("se=")[1][:10])\n')
write('partner_send.py', """\
import json
from azure.servicebus import ServiceBusClient, ServiceBusMessage
conn = open("partner.conn").read() # a token, no key
orders = [{"order_id": "P-1001", "amount": 120},
{"order_id": "P-1002"}, # no amount
{"order_id": "P-1001", "amount": 120}] # a retry
with ServiceBusClient.from_connection_string(conn) as client:
with client.get_queue_sender("partner-orders") as sender:
for order in orders:
sender.send_messages(ServiceBusMessage(
json.dumps(order), message_id=order["order_id"],
content_type="application/json"))
print("partner sent", len(orders), "messages")
""")
write('validate.py', """\
import json, os
from azure.servicebus import (ServiceBusClient, ServiceBusMessage,
ServiceBusSubQueue)
def valid(body):
try:
return type(json.loads(body).get("amount")) in (int, float)
except ValueError:
return False
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
with client:
rx = client.get_queue_receiver("partner-orders", max_wait_time=3)
with rx, client.get_queue_sender("orders-in") as orders:
for msg in rx:
if valid(str(msg)):
orders.send_messages(ServiceBusMessage(
str(msg), message_id=msg.message_id))
rx.complete_message(msg)
print(msg.message_id, "valid: sent to orders-in")
else:
rx.dead_letter_message(msg, reason="SchemaInvalid",
error_description="amount missing")
print(msg.message_id, "invalid: dead-lettered")
with client.get_queue_receiver("partner-orders",
sub_queue=ServiceBusSubQueue.DEAD_LETTER) as dlq:
for m in dlq.peek_messages(10, sequence_number=1):
print("DLQ:", m.message_id, m.dead_letter_reason, "-",
m.dead_letter_error_description)
""")
run('python make_token.py')
run('python partner_send.py')
run('python validate.py')
# Step 7: The backbone core: one topic, filtered subscriptions, duplicate detection on the topic
write('backbone.py', """\
import os
from azure.servicebus import ServiceBusClient, ServiceBusMessage
events = [("5001", "order.placed"), ("5001", "order.paid"),
("5001", "order.paid"), # a publisher retry
("5002", "order.placed"), ("5001", "order.shipped")]
client = ServiceBusClient.from_connection_string(
os.environ["SERVICEBUS_CONNECTION_STRING"])
with client:
with client.get_topic_sender("order-events") as sender:
for order, event in events: # relayed from the outbox
sender.send_messages(ServiceBusMessage(
f"{order} {event}", subject=event,
message_id=f"{order}-{event}")) # the event ID
counts = {}
for sub in ["billing", "shipping", "email"]:
rx = client.get_subscription_receiver("order-events", sub)
with rx:
got = rx.peek_messages(20, sequence_number=1)
counts[sub] = len(got)
for m in got:
print(f"{sub:9}", m)
print("published", len(events), "| per subscription:", counts)
""")
run('python backbone.py')
All 10 parts are listed on the Real Time Azure Service Bus Interview Questions and Answers page on this blog.
The sample data on this page is invented for teaching. Every terminal output is the real output of the command above it, run on the official Azure Service Bus emulator (2.0.1) with SQL Server and azure-servicebus 7.15.0. Features the emulator does not support are shown as code. Part of the Real Time Azure Service Bus Interview Questions and Answers series on TechBrothersIT. Features, limits and their status (GA or preview) are current as of October 2026; check the official Azure Service Bus documentation (learn.microsoft.com/azure/service-bus-messaging) for anything you quote in an interview.
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.