Change Data Capture with Camel and Debezium: When the Database Is Your Event Source

Most integration work starts at a message. Sometimes it starts at a table. A team needs order changes out of a packaged ERP that has no event stream, no outbound hooks worth using, and a vendor support contract that forbids triggers. The two usual answers are a polling query with a last_modified watermark, or change data capture — reading the database's own transaction log and turning committed row changes into messages.

Camel supports both. The camel-debezium-* components embed Debezium's engine directly in a Camel route, so a MySQL binlog, a Postgres logical replication slot, or a SQL Server change table becomes from(...). That is attractive, and it is also the part people adopt without reading the operational fine print. Here is what we tell clients.

What the component actually does

from("debezium-postgres:orders"
        + "?databaseHostname=db-host"
        + "&databaseUser=cdc"
        + "&databasePassword={{db.password}}"
        + "&databaseDbname=erp"
        + "&databaseServerName=erp-prod"
        + "&schemaIncludeList=sales"
        + "&tableIncludeList=sales.orders"
        + "&offsetStorageFileName=/data/offsets.dat")
    .routeId("orders-cdc")
    .to("bean:orderEventMapper")
    .to("kafka:order-events");

The consumer runs Debezium's embedded engine inside your Camel application. Each captured change arrives as an exchange whose body is the row state and whose headers carry the operation (c, u, d, r for snapshot reads), the source table, and the log coordinates. There is no Kafka Connect cluster in this picture. For a shop that already runs Camel and does not want a second distributed platform to operate, that is the whole appeal.

It is also the whole risk. Kafka Connect gives you offset storage, restarts, and rebalancing as someone else's problem. Embedded Debezium makes them yours.

The four things that decide whether this works

1. Offsets are state, and state needs a real home. The engine records how far it has read in the log. Lose that file and you re-snapshot the table or skip changes; put it on a container's ephemeral filesystem and you will lose it. Use a persistent volume, or configure the Kafka-backed offset store if you have a broker anyway. Treat the offset store as production data: backed up, monitored, and never shared between two instances.

2. It is single-writer by design. One connector reads one log position. You cannot run three replicas of the pod for availability and expect them to cooperate — you get three snapshots and triple events. Deploy one instance and plan for failover by restart (a Kubernetes Deployment with one replica and a quick restart is usually acceptable, because the log persists while you are down). If your SLA cannot tolerate the restart gap, that is an argument for Kafka Connect, not for cleverness.

3. The database team has to be in the room. CDC needs a replication slot or binlog retention, and both have teeth. A Postgres logical slot that stops being consumed pins WAL forever and eventually fills the disk — your stalled integration becomes their production incident. MySQL binlog retention sets a hard limit on how long you can be down before resuming is impossible. Get retention, slot monitoring, and an alert on replication lag agreed before the first route is written, not after the first page.

4. Snapshots are not free. The initial snapshot reads the whole table. On a large table under load that is a long, heavy operation, and your downstream has to absorb it. Decide deliberately between a full snapshot, snapshot.mode=never with a documented backfill, and incremental snapshots — and size the downstream for the snapshot rate, not the steady-state rate.

The part that is integration design, not plumbing

Getting change events is the easy half. The hard half is that a row change is not a business event. The log gives you "row 88213 in sales.orders changed status from 2 to 4." Nobody downstream wants that. They want "order shipped," with the fields that matter and a schema you can version.

So the route's real job is translation, and it has the familiar problems:

• Schema coupling. Publishing raw CDC payloads exports the ERP's physical schema to every consumer. A vendor patch that renames a column becomes everyone's outage. Map to an explicit outbound contract in the route, and keep that mapping under test.
• Joins the log cannot do. An order header change usually needs its lines, or a customer name from another table. CDC gives you one table at a time. The options are an enrichment call back to the database (pollEnrich against a read replica), a Camel aggregator keyed on the business id with a completion timeout, or publishing a thin event and letting consumers fetch. All three are defensible; picking silently is not.
• Ordering and duplicates. Debezium gives per-table ordering and at-least-once delivery. After a restart you can see replayed events. If the downstream effect is not naturally idempotent, put an idempotent consumer keyed on the business identifier plus the log coordinate in front of it.
• Deletes and tombstones. A delete arrives with a null-ish after-state. Decide what that means to a consumer — a tombstone, a soft-delete event, or nothing — rather than letting it fall through a content-based router into the dead letter channel.

When to use something simpler

CDC earns its complexity when you need low latency, you cannot modify the source application, and you cannot afford to miss intermediate states. Plenty of integrations need none of that.

• If you own the writing application, use the outbox pattern. A row written in the same transaction, read by a Camel route, gives you real business events with no log access, no slot management, and no schema coupling. It is almost always the better answer when it is available.
• If the table has a reliable, indexed, monotonic last_modified and minute-level latency is fine, poll. A timer plus sql consumer with a watermark is a hundred lines of boring, restartable, debuggable code. It misses intermediate states and it needs the column to be trustworthy — but when those hold, it beats a replication slot every time.
• If you already run Kafka Connect, run Debezium there. Use Camel for what Camel is good at: consuming those topics and doing the routing, enrichment, and protocol bridging. Embedding the engine is for estates that genuinely do not want a Connect cluster.

What we check in a review

When we find camel-debezium in an estate, four questions settle whether it is healthy: Where do offsets live, and has a restart been tested? Is exactly one instance running, and what happens when it dies? Who is watching replication lag and slot size, and do they know this route exists? And is the published event a contract, or is it the vendor's table?

CDC is a good tool. It is also a piece of database infrastructure living inside your integration application, and it should be reviewed as both.