In this post we are going to cover Azure Event Hubs fundamentals and architecture. These are 7 real Event Hubs interview questions for data engineers, asked the way they come up on a real project, and the last one is a scenario: the business wants website clickstream events available to analytics and fraud detection within seconds, and you design it.
Event Hubs is the front door for streaming data in Azure. Apps send events in, and many systems read them out, each at its own pace. Interviewers start here because every later question depends on it: namespaces, partitions, consumer groups, retention and tiers. If you can explain why reading an event never deletes it, you show that you understand how a streaming log works.
Part 1 of the Real Time Azure Event Hubs Interview Questions and Answers series on TechBrothersIT.
- The short answer
- Let's set up the project
- Question 1: Your manager asks what Azure Event Hubs is and why the team needs it for streaming data. What do you say?
- Question 2: A new teammate asks how namespaces, event hubs, partitions and consumer groups fit together. How do you explain it?
- Question 3: Two teams both need every clickstream event. How does Event Hubs let each team read the same stream?
- Question 4: Someone asks when to use Event Hubs, Service Bus or Event Grid. How do you choose?
- Question 5: A teammate asks whether Event Hubs is a queue that deletes events once they are read. How do you answer?
- Question 6: Your manager asks which tier to pick: Basic, Standard, Premium or Dedicated. How do you decide?
- Scenario 7: The business wants website clickstream events available to analytics and fraud detection within seconds. How would you design it?
- The full script
The short answer
Azure Event Hubs is a fully managed, real-time data streaming platform. A namespace holds event hubs; an event hub is an append-only log split into ordered partitions; every consumer group reads the whole stream with its own position. Reading never deletes events: only retention removes them.
Let's set up the project
We run Microsoft's official Event Hubs emulator locally, with Azurite for the checkpoint store, and small Python scripts that use the azure-eventhub client, so every output is real. Features the emulator doesn't support are shown as code. The emulator reads its entities from Config.json: one event hub, clicks, with 4 partitions and the consumer groups analytics and fraud. Before the first step we run python setup_storage.py once. It creates the blob container fraud-checkpoints in Azurite.
{
"UserConfig": {
"NamespaceConfig": [
{
"Type": "EventHub",
"Name": "emulatorNs1",
"Entities": [
{
"Name": "clicks",
"PartitionCount": 4,
"ConsumerGroups": [
{
"Name": "analytics"
},
{
"Name": "fraud"
}
]
}
]
}
],
"LoggingConfig": {
"Type": "File"
}
}
}import os
from azure.storage.blob import BlobServiceClient
blob = BlobServiceClient.from_connection_string(
os.environ["STORAGE_CONNECTION_STRING"])
blob.create_container("fraud-checkpoints") # the checkpoint storeimport json, os
from azure.eventhub import EventData, EventHubProducerClient
producer = EventHubProducerClient.from_connection_string(
os.environ["EVENTHUB_CONNECTION_STRING"], eventhub_name="clicks")
clicks = {"user-1": ["/home", "/shoes", "/cart", "/checkout"],
"user-2": ["/home", "/search"],
"user-3": ["/home", "/shoes"]}
with producer:
for user, pages in clicks.items():
batch = producer.create_batch(partition_key=user)
for page in pages:
click = {"user": user, "page": page}
batch.add(EventData(json.dumps(click)))
producer.send_batch(batch)
print("sent", len(batch), "clicks for", user)import os
from azure.eventhub import EventHubProducerClient
client = EventHubProducerClient.from_connection_string(
os.environ["EVENTHUB_CONNECTION_STRING"], eventhub_name="clicks")
with client:
hub = client.get_eventhub_properties()
print("event hub:", hub["eventhub_name"])
print("partition ids:", hub["partition_ids"])
for pid in hub["partition_ids"]:
p = client.get_partition_properties(pid)
first = p["beginning_sequence_number"]
last = p["last_enqueued_sequence_number"]
print(f"partition {pid}: first {first}, last {last}, "
f"empty {p['is_empty']}")import logging, os, sys, threading
from azure.eventhub import EventHubConsumerClient
logging.getLogger("azure.eventhub").setLevel(logging.ERROR)
group = sys.argv[1]
consumer = EventHubConsumerClient.from_connection_string(
os.environ["EVENTHUB_CONNECTION_STRING"],
consumer_group=group, eventhub_name="clicks")
seen = []
def on_event(partition_context, event):
click = event.body_as_json()
seen.append((partition_context.partition_id,
event.sequence_number, click["user"], click["page"]))
threading.Timer(6, consumer.close).start() # stop after 6 s
with consumer:
consumer.receive(on_event=on_event, starting_position="-1")
print(f"consumer group {group}: {len(seen)} events")
for pid, seq, user, page in sorted(seen):
print(f" partition {pid} seq {seq} {user} {page}")import logging, os, threading
from azure.eventhub import EventHubConsumerClient
from azure.eventhub.extensions.checkpointstoreblob import (
BlobCheckpointStore)
logging.getLogger("azure.eventhub").setLevel(logging.ERROR)
store = BlobCheckpointStore.from_connection_string(
os.environ["STORAGE_CONNECTION_STRING"], "fraud-checkpoints")
consumer = EventHubConsumerClient.from_connection_string(
os.environ["EVENTHUB_CONNECTION_STRING"], consumer_group="fraud",
eventhub_name="clicks", checkpoint_store=store)
scored = []
def on_batch(partition_context, events):
if events:
scored.extend(events) # score the clicks
partition_context.update_checkpoint() # after the batch
threading.Timer(8, consumer.close).start()
with consumer:
consumer.receive_batch(on_event_batch=on_batch, max_wait_time=1,
starting_position="-1")
print("fraud scored", len(scored), "clicks")
checkpoints = store.list_checkpoints("localhost", "clicks", "fraud")
for cp in sorted(checkpoints, key=lambda c: c["partition_id"]):
print(f" checkpoint partition {cp['partition_id']}: "
f"sequence {cp['sequence_number']}")Question 1: Your manager asks what Azure Event Hubs is and why the team needs it for streaming data. What do you say?
Azure Event Hubs is a fully managed, real-time data streaming platform. It can ingest millions of events per second with low latency, and it has built-in Apache Kafka compatibility, with no cluster to manage.
Why we need it: it sits between the apps that produce events and the systems that consume them. The website just sends clicks. Analytics, fraud detection and the data lake each read at their own pace, and nobody waits for anybody.
Let's send some events. send.py sends eight clickstream events for three users, one batch per user, with the user as the partition key. We run python send.py.
$ python send.py
sent 4 clicks for user-1
sent 2 clicks for user-2
sent 2 clicks for user-3Three sends: sent 4 clicks for user-1, sent 2 clicks for user-2 and sent 2 clicks for user-3. The events are now stored in the clicks event hub.
How do clients talk to it?
- AMQP 1.0 - send and receive
- Kafka 1.0 and later - send and receive
- HTTPS - send only
And consumers pull: Event Hubs uses a pull model.
It's a streaming engine, not a database. Events stay for the retention period, so for long-term history, turn on Capture.
Question 2: A new teammate asks how namespaces, event hubs, partitions and consumer groups fit together. How do you explain it?
Top down. A namespace is the management container. It holds the network rules and the capacity, throughput units or processing units, shared by every event hub inside it.
Then the event hub. It's an append-only log, the equivalent of a Kafka topic. It's split into partitions, and each partition is an ordered sequence of events, numbered by sequence number.
Let's look at the clicks event hub's properties after our send, with python props.py.
$ python props.py
event hub: clicks
partition ids: ['0', '1', '2', '3']
partition 0: first -1, last -1, empty True
partition 1: first 0, last 3, empty False
partition 2: first 0, last 3, empty False
partition 3: first -1, last -1, empty TrueWe get partition ids: ['0', '1', '2', '3'], so four partitions. On this emulator, the keys put our clicks on partitions 1 and 2: first 0, last 3 on each. Partitions 0 and 3 are still empty True.
Last, the consumer group. It's an independent view of the whole event hub, with its own position in every partition. Every event hub has the $Default group. Standard allows up to 20 consumer groups per event hub, 32 partitions per event hub, and 10 event hubs per namespace.
On the emulator, the entities come from its config file. In Azure, we create all of this with the CLI, Bicep or Terraform.
az eventhubs namespace create -g rg-shop -n shop-events \
--sku Standard --capacity 2
az eventhubs eventhub create -g rg-shop \
--namespace-name shop-events -n clicks --partition-count 4
az eventhubs eventhub consumer-group create -g rg-shop \
--namespace-name shop-events --eventhub-name clicks \
-n analyticsIn Standard, the partition count is fixed at creation, so choose it before the first deploy.
Question 3: Two teams both need every clickstream event. How does Event Hubs let each team read the same stream?
Give each team its own consumer group. Each group tracks its own position, so one team's reading never moves the other team's.
Let's try it. read.py takes the consumer group as an argument and reads from the beginning. First we run python read.py analytics.
$ python read.py analytics
consumer group analytics: 8 events
partition 1 seq 0 user-1 /home
partition 1 seq 1 user-1 /shoes
partition 1 seq 2 user-1 /cart
partition 1 seq 3 user-1 /checkout
partition 2 seq 0 user-2 /home
partition 2 seq 1 user-2 /search
partition 2 seq 2 user-3 /home
partition 2 seq 3 user-3 /shoesconsumer group analytics: 8 events. Sorted by partition, each partition is in sequence order: partition 1 has user-1 from seq 0 to seq 3, and partition 2 has user-2 and user-3. Now the fraud team reads with python read.py fraud.
$ python read.py fraud
consumer group fraud: 8 events
partition 1 seq 0 user-1 /home
partition 1 seq 1 user-1 /shoes
partition 1 seq 2 user-1 /cart
partition 1 seq 3 user-1 /checkout
partition 2 seq 0 user-2 /home
partition 2 seq 1 user-2 /search
partition 2 seq 2 user-3 /home
partition 2 seq 3 user-3 /shoesconsumer group fraud: 8 events, from the same partitions, with the same sequence numbers. Nothing was copied: both groups read one stored stream.
Here are the limits to know, per event hub.
| Consumer groups per event hub | Basic | Standard | Premium | Dedicated |
|---|---|---|---|---|
| maximum | 1 | 20 | 100 | 1,000 |
Up to five readers can share a partition within one group, but plan for one active reader per partition.
Two apps sharing $Default with one checkpoint store. Their processors split the partitions between them, and each app misses events. One consumer group per application.
Question 4: Someone asks when to use Event Hubs, Service Bus or Event Grid. How do you choose?
Start with what you're moving. Let's compare the three.
| Service | Purpose | Data | Typical use |
|---|---|---|---|
| Event Hubs | Big data streaming and ingestion | Event streams (time-ordered series) | Telemetry, data streaming, real-time analytics |
| Service Bus | Enterprise transactional messaging | Messages (high-value payloads) | Orders, financial transactions, workflows |
| Event Grid | Reactive event routing | Events (discrete) | React to status changes, serverless |
Event Hubs is for big data streaming and ingestion: event streams, a time-ordered series, like telemetry and clickstreams for real-time analytics.
Service Bus is enterprise transactional messaging: high-value messages, where a contract exists between the publisher and the consumer, like order processing, financial transactions and workflows. With sessions, it can also keep order.
Event Grid is reactive event routing: discrete events, lightweight notifications that a state changed, where the publisher has no expectation about how they're handled. It fits serverless apps that react to status changes.
All three deliver at least once, and they often work together. Microsoft's own example is an e-commerce site:
- Service Bus - process orders
- Event Hubs - capture site telemetry
- Event Grid - react to events like an item being shipped
A stream you analyze: Event Hubs. A business message that must be processed: Service Bus. A notification that something changed: Event Grid.
Question 5: A teammate asks whether Event Hubs is a queue that deletes events once they are read. How do you answer?
No. Event Hubs is a log, not a queue. Reading an event doesn't remove it. Only the retention policy removes events, by time, or by key with log compaction, and you can't delete them explicitly.
Let's prove it. The analytics group already read all eight clicks. We read with analytics again, then check the partition properties: python read.py analytics and python props.py.
$ python read.py analytics
consumer group analytics: 8 events
partition 1 seq 0 user-1 /home
partition 1 seq 1 user-1 /shoes
partition 1 seq 2 user-1 /cart
partition 1 seq 3 user-1 /checkout
partition 2 seq 0 user-2 /home
partition 2 seq 1 user-2 /search
partition 2 seq 2 user-3 /home
partition 2 seq 3 user-3 /shoes
$ python props.py
event hub: clicks
partition ids: ['0', '1', '2', '3']
partition 0: first -1, last -1, empty True
partition 1: first 0, last 3, empty False
partition 2: first 0, last 3, empty False
partition 3: first -1, last -1, empty TrueStill consumer group analytics: 8 events. And the partition properties haven't changed: partitions 1 and 2 still show first 0, last 3.
That works because the position belongs to the consumer, not the service. The consumer chooses where to start:
"-1"- the beginning"@latest"- the end, the default- an offset - a string
- a sequence number - an int
- an enqueued time - a datetime
And saving that position, checkpointing, is the consumer's job.
Retention is set per event hub, in hours. Here is the maximum per tier.
| Max retention | Basic | Standard | Premium | Dedicated |
|---|---|---|---|---|
| per event hub | 1 day | 7 days | 90 days | 90 days |
If you need a work queue, where each message is processed and removed, use Service Bus. And for history beyond retention, use Capture.
Question 6: Your manager asks which tier to pick: Basic, Standard, Premium or Dedicated. How do you decide?
Start with Standard, and move up when you need isolation or scale. Let's go through the four tiers.
| Tier | Capacity | Consumer groups | Max retention | Highlights |
|---|---|---|---|---|
| Basic | throughput units (max 40) | 1 | 1 day | no Kafka, no Capture |
| Standard | throughput units (max 40), auto-inflate | 20 | 7 days | Kafka, Capture (extra), Geo-DR |
| Premium | processing units (1 to 16) | 100 | 90 days | isolation, CMK, add partitions, geo-replication |
| Dedicated | capacity units, single tenant | 1,000 | 90 days | highest quotas, all Premium features |
Basic is for simple, low-volume streaming: one consumer group, one day of retention, and no Kafka or Capture. Standard fits most production workloads: twenty consumer groups, seven days, Kafka, Capture and geo-disaster recovery.
Both are sized in throughput units, up to 40 per namespace, shared by every event hub in it. One throughput unit gives:
- In - 1 MB/s or 1,000 events/s, whichever comes first
- Out - 2 MB/s or 4,096 events/s
Standard can auto-inflate them.
Premium gives predictable performance through resource isolation, sized in processing units. It adds 90 days of retention, customer-managed keys, dynamic partition scale-out and geo-replication.
Dedicated is a single-tenant cluster, sized in capacity units, for the largest and most demanding workloads, with the highest quotas and every Premium feature.
Start with Standard. Move to Premium for isolation, longer retention, customer-managed keys or more than 40 throughput units. And avoid Basic for a shared stream: with one consumer group, a second team can't get its own.
Scenario 7: The business wants website clickstream events available to analytics and fraud detection within seconds. How would you design it?
One event hub, one consumer group per consumer, and checkpoints so each consumer resumes where it stopped.
First, the producer. The website sends clicks to one event hub, with the user or session ID as the partition key, so each user's clicks stay in order. Producers batch, and partitions are sized for peak, because Standard can't change them later.
Then the consumers, each with its own consumer group:
- analytics - Stream Analytics or Databricks, into the lake and the dashboards
- fraud - an event processor that checkpoints to blob storage
And Capture keeps the raw events in ADLS Gen2.
Let's build the fraud consumer. processor.py reads batches, scores them, and checkpoints after each batch. We run python processor.py, then python send.py to send eight more clicks, then python processor.py again.
$ python processor.py
fraud scored 8 clicks
checkpoint partition 1: sequence 3
checkpoint partition 2: sequence 3
$ python send.py
sent 4 clicks for user-1
sent 2 clicks for user-2
sent 2 clicks for user-3
$ python processor.py
fraud scored 8 clicks
checkpoint partition 1: sequence 7
checkpoint partition 2: sequence 7The first run prints fraud scored 8 clicks, with checkpoint partition 1: sequence 3 and checkpoint partition 2: sequence 3. Then eight more clicks arrive, and the processor starts again. It resumes from its checkpoints and scores only the new ones: fraud scored 8 clicks, and the checkpoints move to sequence 7.
Delivery is at least once, so fraud scoring must be idempotent. And monitor throughput and consumer lag, so "within seconds" stays true at peak.
Key points to remember
- Event Hubs: a fully managed, real-time data streaming platform with a Kafka endpoint
- Namespace holds capacity; an event hub is a log split into ordered partitions
- Each consumer group reads every event with its own position
- Reading never deletes: retention does; Service Bus for work queues
- Standard for most workloads; Premium or Dedicated for isolation and scale
The full script
This is a Python script that writes every file and runs every command above in order against the Event Hubs emulator: start the emulator with the Config.json above, run pip install azure-eventhub azure-eventhub-checkpointstoreblob azure-storage-blob confluent-kafka fastavro first, and set EVENTHUB_CONNECTION_STRING and STORAGE_CONNECTION_STRING.
Show the full script that writes the files and runs every step
# ========================================================================
# Event Hubs Interview Q&A #1 - Event Hubs fundamentals and architecture
# TechBrothersIT. Names and sample data are invented for teaching.
# Every step below was run on the official Azure Event Hubs emulator (Docker image, version 2.2.1) with Azurite
# and azure-eventhub 5.15.1; the outputs in the video and the blog post are the real outputs.
# To run it yourself:
# 1. Start the emulator with Microsoft's installer (github.com/Azure/azure-event-hubs-emulator-installer),
# using a Config.json with the event hubs and consumer groups listed below.
# 2. pip install azure-eventhub azure-eventhub-checkpointstoreblob azure-storage-blob confluent-kafka fastavro
# 3. Set EVENTHUB_CONNECTION_STRING to the emulator connection string (Endpoint=sb://localhost;
# SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=SAS_KEY_VALUE;UseDevelopmentEmulator=true;)
# and STORAGE_CONNECTION_STRING to the Azurite blob connection string, then run python EventHubs_Interview_QA_01_Code.py.
# Entities in Config.json: clicks (4 partitions, groups analytics, fraud)
# Steps that restart the emulator say so in a comment: stop and start it by hand at that point.
# ========================================================================
import os, subprocess, time
def write(path, text):
if text is None:
if os.path.exists(path): os.remove(path)
return
os.makedirs(os.path.dirname(path) or ".", exist_ok=True)
with open(path, "w", encoding="utf-8") as f:
f.write(text.strip("\n") + "\n")
def run(cmd, expect_error=False, allow_error=False, env=None):
print("\n$ " + cmd, flush=True)
rc = subprocess.run(cmd, shell=True, env=dict(os.environ, **(env or {}))).returncode
if rc != 0 and not (expect_error or allow_error):
raise SystemExit("stopped: this command was expected to succeed")
# Step 0: the starting files
write('setup_storage.py', """\
import os
from azure.storage.blob import BlobServiceClient
blob = BlobServiceClient.from_connection_string(
os.environ["STORAGE_CONNECTION_STRING"])
blob.create_container("fraud-checkpoints") # the checkpoint store
""")
write('send.py', """\
import json, os
from azure.eventhub import EventData, EventHubProducerClient
producer = EventHubProducerClient.from_connection_string(
os.environ["EVENTHUB_CONNECTION_STRING"], eventhub_name="clicks")
clicks = {"user-1": ["/home", "/shoes", "/cart", "/checkout"],
"user-2": ["/home", "/search"],
"user-3": ["/home", "/shoes"]}
with producer:
for user, pages in clicks.items():
batch = producer.create_batch(partition_key=user)
for page in pages:
click = {"user": user, "page": page}
batch.add(EventData(json.dumps(click)))
producer.send_batch(batch)
print("sent", len(batch), "clicks for", user)
""")
write('props.py', """\
import os
from azure.eventhub import EventHubProducerClient
client = EventHubProducerClient.from_connection_string(
os.environ["EVENTHUB_CONNECTION_STRING"], eventhub_name="clicks")
with client:
hub = client.get_eventhub_properties()
print("event hub:", hub["eventhub_name"])
print("partition ids:", hub["partition_ids"])
for pid in hub["partition_ids"]:
p = client.get_partition_properties(pid)
first = p["beginning_sequence_number"]
last = p["last_enqueued_sequence_number"]
print(f"partition {pid}: first {first}, last {last}, "
f"empty {p['is_empty']}")
""")
write('read.py', """\
import logging, os, sys, threading
from azure.eventhub import EventHubConsumerClient
logging.getLogger("azure.eventhub").setLevel(logging.ERROR)
group = sys.argv[1]
consumer = EventHubConsumerClient.from_connection_string(
os.environ["EVENTHUB_CONNECTION_STRING"],
consumer_group=group, eventhub_name="clicks")
seen = []
def on_event(partition_context, event):
click = event.body_as_json()
seen.append((partition_context.partition_id,
event.sequence_number, click["user"], click["page"]))
threading.Timer(6, consumer.close).start() # stop after 6 s
with consumer:
consumer.receive(on_event=on_event, starting_position="-1")
print(f"consumer group {group}: {len(seen)} events")
for pid, seq, user, page in sorted(seen):
print(f" partition {pid} seq {seq} {user} {page}")
""")
write('processor.py', """\
import logging, os, threading
from azure.eventhub import EventHubConsumerClient
from azure.eventhub.extensions.checkpointstoreblob import (
BlobCheckpointStore)
logging.getLogger("azure.eventhub").setLevel(logging.ERROR)
store = BlobCheckpointStore.from_connection_string(
os.environ["STORAGE_CONNECTION_STRING"], "fraud-checkpoints")
consumer = EventHubConsumerClient.from_connection_string(
os.environ["EVENTHUB_CONNECTION_STRING"], consumer_group="fraud",
eventhub_name="clicks", checkpoint_store=store)
scored = []
def on_batch(partition_context, events):
if events:
scored.extend(events) # score the clicks
partition_context.update_checkpoint() # after the batch
threading.Timer(8, consumer.close).start()
with consumer:
consumer.receive_batch(on_event_batch=on_batch, max_wait_time=1,
starting_position="-1")
print("fraud scored", len(scored), "clicks")
checkpoints = store.list_checkpoints("localhost", "clicks", "fraud")
for cp in sorted(checkpoints, key=lambda c: c["partition_id"]):
print(f" checkpoint partition {cp['partition_id']}: "
f"sequence {cp['sequence_number']}")
""")
run('python setup_storage.py')
# Step 1: A producer sends 8 clickstream events for 3 users, one batch per user, partition key = user
run('python send.py')
# Step 2: Event hub and partition properties after the send
run('python props.py')
# Step 3: Consumer group analytics reads the whole stream from the beginning
run('python read.py analytics')
# Step 4: Consumer group fraud reads the same 8 events independently
run('python read.py fraud')
# Step 5: Read again with analytics: still 8 events; the partitions still start at sequence 0
run('python read.py analytics')
run('python props.py')
# Step 6: Fraud event processor with a blob checkpoint store: run, 8 more clicks arrive, run again
run('python processor.py')
run('python send.py')
run('python processor.py')
Part 2: Partitions and Throughput
The sample data on this page is invented for teaching. Every terminal output is the real output of the command above it, run on the official Azure Event Hubs emulator (2.2.1) with Azurite and azure-eventhub 5.15.1. Features the emulator does not support are shown as code. Part of the Real Time Azure Event Hubs Interview Questions and Answers series on TechBrothersIT. Features, limits and their status (GA or preview) are current as of October 2026; check the official Azure Event Hubs documentation (learn.microsoft.com/azure/event-hubs) for anything you quote in an interview.
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.