Event Streaming with Kafka in Practice: Move Data While It's Still True
Signature Course No. 431 — A Signature AI Socratic Coaching Course.
No experience needed — your coach starts where you are.
A New Kind of Learning
Socratic coaching, delivered by an agentic AI coach.
Not a course you watch. An AI coach you work with.
The Lighthouse Coach begins by discovering how you think about the subject, teaches what is missing, and gives you realistic practice before asking you to prove that you can use it. You may test out of concepts you already understand, while capstones turn the learning into a practical plan, decision, conversation, or other real-world result.
The curriculum is deliberately authored; the AI delivers it through coaching. It is not an AI randomly generating lessons.
A coach that teaches by asking — not by lecturing
The Lighthouse Coach is an AI Socratic coach built on Cognosco Learning's Event Streaming with Kafka in Practice: Move Data While It's Still True course, in the Personal Effectiveness & Well-Being category. It does not play videos at you. It holds a conversation: it asks you a question, listens to your answer, names the misconception it hears, affirms what you got right, and asks the next question — the way a great coach works in the room.
A stream is a promise about the present. Build it so the promise is the one you actually keep. That is this course's premise. Most company data still moves the way it moved twenty years ago: in batches, after the fact — the order placed at nine, the warehouse learning at midnight, the fraud check running on yesterday's file. Event streaming moves data while it is still true: systems publish what just happened, the moment it happens, and every system that cares reads the event as it flows past. Kafka is the platform this course teaches for that work — a distributed log in which events are written to topics, kept for a governed period, and read by as many independent consumers as the business needs.
It starts with the stream. You will build the estate correctly, from the seat of the engineer who owns it: what a partition actually guarantees and what it does not; how a key decides which events stay together; how a producer publishes under a standard that survives retries and leader changes; how consumer groups divide work, and why partitions, not consumers, set the ceiling; what an offset commit promises and where yours must sit; how to name a path's delivery semantics without flattering it; how idempotence turns replays from corruption into routine; how schema discipline keeps events readable as they evolve; and how retention and compaction decide whether your log is a record you can replay or a pipe you can only watch go by.
It works in two movements. First, Module 1 — Build the Stream: Topics, Keys, and Guarantees — work through Concepts 1–9, finishing with The Stream That Tells the Truth; then, Module 2 — Run the Stream: Pressure, Time, and Recovery — work through Concepts 10–18, finishing with Peak Day, Broken Night.
Both capstones produce real artifacts — a Streaming Design and Operating Plan for Talvane Mercantile, a fictional retailer selling home goods online and through a small chain of stores, with its own delivery fleet: topics designed around the ordering their consumers actually rely on, a producer standard for order and payment events, fulfillment consumption with its commits and honestly named semantics, event contracts enforced before publication, and retention sized so replay is a plan, not a hope; and an Operations and Recovery Plan for Talvane's peak day — lag answered without silent dropping, late data ruled on in writing, poison events quarantined with their evidence, access held at least privilege under pressure, and a producer defect recovered by controlled replay and reconciled proof.
The honesty standard is part of the course. Streaming systems are surrounded by inflated language — exactly-once, real-time, never lost, never duplicated — spoken as if a platform setting could repeal arithmetic. This course refuses that language wherever the facts do not carry it. It teaches Kafka at the level of its durable operating model — the architecture and the judgments that survive the next release — not console click-paths or any single version's feature list, and it flags every version-specific detail to verify against current documentation before you rely on it. Nothing in the course is endorsed by, or written on behalf of, the Apache Software Foundation or any managed-service provider.
The course is self-paced and always on — roughly 30 minutes per concept, about 9 hours in full, taken in as many sittings as you like. You can type your answers, or talk out loud: the Voice Coach speaks its questions and listens to your spoken replies. Learners who already know the material can test out concept by concept.
Event Streaming with Kafka in Practice: Move Data While It's Still True is part of The Lighthouse Signature Series — the online counterpart to the one-day, in-person learning Cognosco Learning delivers.
Course Facts
One full concept, coached end to end — free: verify your email and we’ll send you a link to start.
Already enrolled? Enter the course →
Diagnose → Learn & Practice → Prove → Capstone
Every learner moves through the same discipline a Cognosco instructor would insist on in the room.
Diagnose
The Coach starts where the course starts: with topics and partitions. Pick a topic you own or consume and state, in one sentence, the order its consumers actually depend on — and name the entity whose events must stay sequential; say whether you know that topic's partition count and can say why it is that number and not another, or admit it was the default and say what that has cost; and say where in your current work someone has assumed a stronger ordering guarantee than the design provides, and what the visible symptom was. Every concept opens with three questions that locate where you actually stand, and your answers shape how deeply the Coach works the concept with you. Everyone begins with Module 1 — no skipping the foundations you only think you have.
Learn & Practice
Work each concept in conversation with the Coach — spoken or written — then practice it against realistic situations until it holds up.
Prove
Every concept ends in proof: answer the check cold and demonstrate the skill. Know it already? Test out and take credit immediately.
Capstone
Finish with two capstones — The Stream That Tells the Truth for Module 1 and Peak Day, Broken Night for Module 2 — that pull the whole course together, and earn the certificate.
What You'll Learn
Build the stream, then run it under pressure. Every skill in this course is the same discipline at a different depth: scope order to the partition and the key, publish nothing you cannot stand behind, commit only what you have actually done, name the guarantee your whole path actually provides, and keep the log worth returning to.
Module 1 builds the stream. Topics and Partitions; Keys and Partitioning; Producers; Consumers and Consumer Groups; Offsets and Commits; Delivery Semantics; Idempotence and Deduplication; Schemas and Event Contracts; and Retention, Compaction, and Replay — the ordering a partition really guarantees, the keys that keep each entity's story together, the producer standard, the commit discipline, honestly named semantics, replay-safe effects, governed schemas, and a log retained well enough to return to.
Module 2 runs the stream. Lag and Backpressure; Stream Processing Judgment; Windows, Event Time, and Late Data; Rebalancing and Membership; Poison Events and Dead-Letter Handling; Security and Access for Streams; Observability and the Cluster as a Service; Capacity, Partitioning Growth, and Cost; and Incident Response and Controlled Replay — lag read as a distance and a trend, freshness spent only where it pays, time governed as data, failures quarantined with their evidence intact, the cluster run as a service other teams can build on, and recovery by preservation, controlled replay, and proof.
And it holds its boundary throughout. This course owns event streaming in depth only. It does not teach data-engineering basics — pipelines and batch fundamentals are Signature Course No. 151's ground, assumed here in full — and it does not teach console click-paths or any single version's feature list, because those churn, and a course that froze them into print would mislead you on the day it mattered.
What you will learn
- Design topics and partitions so the ordering your consumers rely on is ordering the platform actually guarantees.
- Choose event keys that keep each entity's history together, and state plainly what a key choice costs and buys.
- Publish events under a producer standard — acknowledgment, bounded retry, idempotent production — that neither loses events silently nor duplicates them on retry.
- Organize consumers into groups that share work in proportion to partitions, and read each group's independence correctly.
- Place offset commits so a crash costs you a stated, bounded amount of reprocessing — never silent loss.
- Name the delivery semantics a whole path provides, end to end, and design honestly for the duplicates or losses that remain.
- Make consumers idempotent, so a replay, a retry, or a rebalance is an inconvenience, not a corruption.
- Govern event schemas as contracts: evolve them compatibly, check them before publishing, and keep old readers working.
- Set retention and compaction as policy, and replay a topic safely when a consumer, a bug, or an incident demands it.
- Read lag as a distance and a trend, and answer a falling-behind consumer with backpressure, scale, or shedding — never silent dropping.
- Decide what earns processing in motion and what belongs at rest, and defend the decision on the value of freshness.
- Handle windows, event time, and late data with an explicit rule every consumer of the result can read.
- Survive rebalancing and membership change without losing work or double-counting it.
- Quarantine poison events without blocking a partition or discarding evidence, and return them to the flow repaired.
- Control who may publish and who may read, topic by topic, at least privilege, with an audit trail.
- Operate the cluster as a service: explicit promises to internal teams, signals that page on consumer harm, and a capacity and cost discipline that keeps growth a decision instead of a surprise.
- Run an incident on the estate: stop the harm first, preserve the log, recover by controlled replay, and verify before you declare it over.
Everything in the course
- All 18 concepts of Event Streaming with Kafka in Practice: Move Data While It's Still True, in 2 modules
- A starting-point check first — the Coach finds where you actually stand before teaching
- Learn, Practice, Prove for every concept — no passive video watching
- A test-out on every concept: prove you know it and move on
- The Voice Coach: talk with the Coach out loud — it speaks, listens, and replies
- A personal Learner Record of your work across the course
- Two capstone exercises: The Stream That Tells the Truth and Peak Day, Broken Night
- A printable session summary whenever you pause
- The Lighthouse Coach completion certificate (Signature Series)
- Your progress is saved as you go — stop anytime, resume where you left off
Built for the people who run the stream other teams build on
Data-platform engineers who run, or are about to run, a Kafka estate that other teams depend on, and who need the full operating discipline, not just a working cluster; and data engineers moving from batch to streaming — fluent in pipelines, new to the guarantees, failure modes, and judgments that continuous data demands. This course assumes your ground; it does not re-teach it.
Backend and product engineers who publish or consume events in a product they own, and who keep discovering that their local choices — a key, a commit point, a retry — have estate-wide consequences.
Stream-processing developers building transformations, joins, and windowed results on live data, who must defend their design when data arrives late, twice, or not at all; and platform and site-reliability engineers who carry the pager for the cluster and need the consumer's-eye view: what lag means, what a rebalance costs, and what to protect first in an incident. This course assumes working data-engineering fluency at the level of No. 151, plus general engineering fluency; it assumes no prior Kafka operation — the platform's operating model is taught here, in full, from Concept 1.
Prefer to learn live and in person? The same material anchors Cognosco's one-day programs — browse the in-person catalog.
Start with Topics and Partitions
$195 — introductory price. One payment. No subscription, no password to manage.
Here is exactly what happens when you enroll:
- Pay securely with Stripe — you are taken straight into the course
- Create your learner profile and begin the Module 1 starting-point check
- Save the access key you are given — it restores your enrollment on any device
- Learn at your pace; your progress is stored in your browser on your device
For the full voice experience, use Chrome and allow the microphone when asked. Typing your answers works everywhere, all the time. Questions before you enroll? Email hello@cognoscolearning.com.
Before you enroll
What is a Socratic coaching session?
A conversation, not a video. Your coach asks a question; you answer — out loud by voice, or by typing — and the coach responds to what you said: naming the misconception it hears, affirming what you got right, and asking the next question. You advance only when you have mastered each concept, so a finished course means the learning held.
How long does it take — and how long do I have?
About 30 minutes per concept. Learn in sittings of any length; your progress is saved as you go. You have a full year from purchase to finish, and you can revisit and retake within that year.
What does the certificate verify?
That you completed every concept and both capstones of this course. Anyone can confirm a Cognosco certificate online — enter its certificate ID on our verification page.
What if it isn't for me — what are the refunds?
If you have not started, cancel for a full refund within seven days of purchase. Once you have started, there is no cash refund — stop before finishing and we credit the full amount you paid toward any other course in the library (credit valid one year from your original purchase). Once you complete the course, no refund or credit applies.
More Signature Courses are coming
Event Streaming with Kafka in Practice: Move Data While It's Still True is Signature Course No. 431. Leave your email and we'll send you news of future Signature Courses and Lighthouse updates as they are announced — nothing else.