Connector Certification

Every connector in krishiv-connectors carries a maturity label and a delivery guarantee. The certification suite is the single source of truth for what is real and what is aspirational.

Maturity labels

Label	Meaning
Certified	Passes the full certification.rs suite. Has a confirmed production deployment. Documented in this guide.
Preview	Code works, certification suite has run, but no production deployment yet. The API may change.
Experimental	Early-stage. May be incomplete. The API will change.

Delivery guarantee matrix

The effective delivery guarantee is the weakest across the source, sink, and durability profile. Combinations outside this table are unsupported (the runtime fails the query at submit time with a clear error).

Source	Sink	Profile	Guarantee	Certified
Parquet (batch)	Parquet (batch)	any	Best effort	Yes
Parquet (batch)	Parquet (two-phase)	distributed-durable	Effectively once	Yes
Kafka	Parquet	single-node-durable+	At-least-once	Yes
Kafka (transactional)	Parquet (two-phase)	distributed-durable	Exactly once	Yes
Kafka (transactional)	Kafka (transactional)	distributed-durable	Exactly once	Yes
Kafka (transactional)	Iceberg (two-phase)	distributed-durable	Exactly once	Yes
Iceberg (batch)	Iceberg (batch)	any	Best effort	Yes
Delta (batch, local)	Delta (batch)	any	Best effort	Yes (local fs only)
Delta (batch, local)	Delta (two-phase)	distributed-durable	Effectively once	Yes (local fs only)
Hudi (batch, local)	Hudi (two-phase)	distributed-durable	Effectively once	Yes (local fs only)
In-memory stream	In-memory sink	any	Best effort	Yes
Vector sinks	—	any	Best effort	No (Experimental)

The certification suite

Lives at crates/krishiv-connectors/certification.rs (gated #[cfg(test)]). Runs the canonical failure-and-recovery matrix for every Certified connector:

1. Source reconnect after connection loss.
2. Source offset commit before the batch is acked.
3. Sink write-failure rollback and retry.
4. Two-phase commit prepare → coordinator crash → restart → resolve.
5. Schema-evolution compatibility (added, dropped, widened columns).
6. Late-event handling for sources with event time.

Run with:

cargo test -p krishiv-connectors --features='lakehouse kafka iceberg' --test certification

Adding a new connector

1. Implement SourceDriver and SinkDriver in crates/krishiv-connectors/src/<kind>.rs.
2. Register the kind in ConnectorKind and add a capabilities profile in registry/capabilities.rs.
3. Add a certification test to certification.rs covering the failures you intend to handle.
4. Bump the maturity from Preview to Certified only after the suite passes in CI for 4 weeks and you have a production deployment.

See also

• Connectors Overview
• Two-phase commit