Apache Kafka provides schema management capabilities through the Schema Registry, including various levels of compatibility. In this article, we will discuss three types of transitive compatibility: BACKWARD_TRANSITIVE, FORWARD_TRANSITIVE, and FULL_TRANSITIVE. Additionally, we will analyze their pros, cons, and provide code examples. BACKWARD_TRANSITIVE Description Backward transitive compatibility means that a new version of the schema must be compatible with all previous versions. This is useful in systems with a long data lifecycle. Code Example curl -X PUT \ -H "Content-Type: application/vnd.schemaregistry.v1+json" \ --data '{"compatibility": "BACKWARD_TRANSITIVE"}' \ http://localhost:8081/config/<subject_name> Java Code Example import io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient; import io.confluent.kafka.schemaregistry.client.SchemaRegistryClient; import io.confluent.kafka.schemaregistry.rest.exceptions.RestClientException; import java.io.IOException; public class BackwardTransitiveExample { public static void main(String[] args) throws IOException, RestClientException { String schemaRegistryUrl = "http://localhost:8081"; SchemaRegistryClient client = new CachedSchemaRegistryClient(schemaRegistryUrl, 10); String subject = "<subject_name>"; String compatibility = "BACKWARD_TRANSITIVE"; // Set compatibility type client.updateCompatibility(subject, compatibility); // Check current compatibility type String currentCompatibility = client.getCompatibility(subject); System.out.println("Current compatibility level: " + currentCompatibility); } } Pros ✅ Ensures compatibility with historical data. ✅ Useful for long-term data storage. Cons ❌ May restrict adding new fields or changing the structure. FORWARD_TRANSITIVE Description Forward transitive compatibility requires that a new version of the schema be compatible with all future versions. This is useful when consumers are updated asynchronously with producers. Code Example curl -X PUT \ -H "Content-Type: application/vnd.schemaregistry.v1+json" \ --data '{"compatibility": "FORWARD_TRANSITIVE"}' \ http://localhost:8081/config/<subject_name> Java Code Example import io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient; import io.confluent.kafka.schemaregistry.client.SchemaRegistryClient; import io.confluent.kafka.schemaregistry.rest.exceptions.RestClientException; import java.io.IOException; public class ForwardTransitiveExample { public static void main(String[] args) throws IOException, RestClientException { String schemaRegistryUrl = "http://localhost:8081"; SchemaRegistryClient client = new CachedSchemaRegistryClient(schemaRegistryUrl, 10); String subject = "<subject_name>"; String compatibility = "FORWARD_TRANSITIVE"; // Set compatibility type client.updateCompatibility(subject, compatibility); // Check current compatibility type String currentCompatibility = client.getCompatibility(subject); System.out.println("Current compatibility level: " + currentCompatibility); } } Pros ✅ Ensures stability for old consumers with new data. ✅ Convenient for systems with asynchronous updates. Cons ❌ Does not guarantee compatibility with historical data. FULL_TRANSITIVE Description Full transitive compatibility combines the requirements of backward and forward compatibility. A new schema version must be compatible with both previous and future versions. Code Example curl -X PUT \ -H "Content-Type: application/vnd.schemaregistry.v1+json" \ --data '{"compatibility": "FULL_TRANSITIVE"}' \ http://localhost:8081/config/<subject_name> Java Code Example import io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient; import io.confluent.kafka.schemaregistry.client.SchemaRegistryClient; import io.confluent.kafka.schemaregistry.rest.exceptions.RestClientException; import java.io.IOException; public class FullTransitiveExample { public static void main(String[] args) throws IOException, RestClientException { String schemaRegistryUrl = "http://localhost:8081"; SchemaRegistryClient client = new CachedSchemaRegistryClient(schemaRegistryUrl, 10); String subject = "<subject_name>"; String compatibility = "FULL_TRANSITIVE"; // Set compatibility type client.updateCompatibility(subject, compatibility); // Check current compatibility type String currentCompatibility = client.getCompatibility(subject); System.out.println("Current compatibility level: " + currentCompatibility); } } Pros ✅ Maximum flexibility and resilience. ✅ Guarantees compatibility with all schema versions. Cons ❌ The strictest compatibility type. ❌ Requires careful planning for schema changes. Comparison Table Compatibility Type Compatibility with Past Versions Compatibility with Future Versions Ease of Use Applicability BACKWARD_TRANSITIVE ✅ ❌ Medium Long-term data storage FORWARD_TRANSITIVE ❌ ✅ Medium Asynchronous updates FULL_TRANSITIVE ✅ ✅ Low Maximum resilience These compatibility types allow you to configure the system to meet your stability and flexibility requirements. The correct choice depends on your system architecture and use cases. Apache Kafka provides schema management capabilities through the Schema Registry, including various levels of compatibility. In this article, we will discuss three types of transitive compatibility: BACKWARD_TRANSITIVE, FORWARD_TRANSITIVE, and FULL_TRANSITIVE. Additionally, we will analyze their pros, cons, and provide code examples. BACKWARD_TRANSITIVE Description Backward transitive compatibility means that a new version of the schema must be compatible with all previous versions. This is useful in systems with a long data lifecycle. all Code Example curl -X PUT \ -H "Content-Type: application/vnd.schemaregistry.v1+json" \ --data '{"compatibility": "BACKWARD_TRANSITIVE"}' \ http://localhost:8081/config/<subject_name> curl -X PUT \ -H "Content-Type: application/vnd.schemaregistry.v1+json" \ --data '{"compatibility": "BACKWARD_TRANSITIVE"}' \ http://localhost:8081/config/<subject_name> Java Code Example import io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient; import io.confluent.kafka.schemaregistry.client.SchemaRegistryClient; import io.confluent.kafka.schemaregistry.rest.exceptions.RestClientException; import java.io.IOException; public class BackwardTransitiveExample { public static void main(String[] args) throws IOException, RestClientException { String schemaRegistryUrl = "http://localhost:8081"; SchemaRegistryClient client = new CachedSchemaRegistryClient(schemaRegistryUrl, 10); String subject = "<subject_name>"; String compatibility = "BACKWARD_TRANSITIVE"; // Set compatibility type client.updateCompatibility(subject, compatibility); // Check current compatibility type String currentCompatibility = client.getCompatibility(subject); System.out.println("Current compatibility level: " + currentCompatibility); } } import io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient; import io.confluent.kafka.schemaregistry.client.SchemaRegistryClient; import io.confluent.kafka.schemaregistry.rest.exceptions.RestClientException; import java.io.IOException; public class BackwardTransitiveExample { public static void main(String[] args) throws IOException, RestClientException { String schemaRegistryUrl = "http://localhost:8081"; SchemaRegistryClient client = new CachedSchemaRegistryClient(schemaRegistryUrl, 10); String subject = "<subject_name>"; String compatibility = "BACKWARD_TRANSITIVE"; // Set compatibility type client.updateCompatibility(subject, compatibility); // Check current compatibility type String currentCompatibility = client.getCompatibility(subject); System.out.println("Current compatibility level: " + currentCompatibility); } } Pros ✅ Ensures compatibility with historical data. ✅ Useful for long-term data storage. ✅ Ensures compatibility with historical data. ✅ Useful for long-term data storage. Cons ❌ May restrict adding new fields or changing the structure. ❌ May restrict adding new fields or changing the structure. FORWARD_TRANSITIVE Description Forward transitive compatibility requires that a new version of the schema be compatible with all future versions. This is useful when consumers are updated asynchronously with producers. all Code Example curl -X PUT \ -H "Content-Type: application/vnd.schemaregistry.v1+json" \ --data '{"compatibility": "FORWARD_TRANSITIVE"}' \ http://localhost:8081/config/<subject_name> curl -X PUT \ -H "Content-Type: application/vnd.schemaregistry.v1+json" \ --data '{"compatibility": "FORWARD_TRANSITIVE"}' \ http://localhost:8081/config/<subject_name> Java Code Example import io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient; import io.confluent.kafka.schemaregistry.client.SchemaRegistryClient; import io.confluent.kafka.schemaregistry.rest.exceptions.RestClientException; import java.io.IOException; public class ForwardTransitiveExample { public static void main(String[] args) throws IOException, RestClientException { String schemaRegistryUrl = "http://localhost:8081"; SchemaRegistryClient client = new CachedSchemaRegistryClient(schemaRegistryUrl, 10); String subject = "<subject_name>"; String compatibility = "FORWARD_TRANSITIVE"; // Set compatibility type client.updateCompatibility(subject, compatibility); // Check current compatibility type String currentCompatibility = client.getCompatibility(subject); System.out.println("Current compatibility level: " + currentCompatibility); } } import io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient; import io.confluent.kafka.schemaregistry.client.SchemaRegistryClient; import io.confluent.kafka.schemaregistry.rest.exceptions.RestClientException; import java.io.IOException; public class ForwardTransitiveExample { public static void main(String[] args) throws IOException, RestClientException { String schemaRegistryUrl = "http://localhost:8081"; SchemaRegistryClient client = new CachedSchemaRegistryClient(schemaRegistryUrl, 10); String subject = "<subject_name>"; String compatibility = "FORWARD_TRANSITIVE"; // Set compatibility type client.updateCompatibility(subject, compatibility); // Check current compatibility type String currentCompatibility = client.getCompatibility(subject); System.out.println("Current compatibility level: " + currentCompatibility); } } Pros ✅ Ensures stability for old consumers with new data. ✅ Ensures stability for old consumers with new data. ✅ Convenient for systems with asynchronous updates. ✅ Convenient for systems with asynchronous updates. Cons ❌ Does not guarantee compatibility with historical data. ❌ Does not guarantee compatibility with historical data. FULL_TRANSITIVE Description Full transitive compatibility combines the requirements of backward and forward compatibility. A new schema version must be compatible with both previous and future versions. Code Example curl -X PUT \ -H "Content-Type: application/vnd.schemaregistry.v1+json" \ --data '{"compatibility": "FULL_TRANSITIVE"}' \ http://localhost:8081/config/<subject_name> curl -X PUT \ -H "Content-Type: application/vnd.schemaregistry.v1+json" \ --data '{"compatibility": "FULL_TRANSITIVE"}' \ http://localhost:8081/config/<subject_name> Java Code Example import io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient; import io.confluent.kafka.schemaregistry.client.SchemaRegistryClient; import io.confluent.kafka.schemaregistry.rest.exceptions.RestClientException; import java.io.IOException; public class FullTransitiveExample { public static void main(String[] args) throws IOException, RestClientException { String schemaRegistryUrl = "http://localhost:8081"; SchemaRegistryClient client = new CachedSchemaRegistryClient(schemaRegistryUrl, 10); String subject = "<subject_name>"; String compatibility = "FULL_TRANSITIVE"; // Set compatibility type client.updateCompatibility(subject, compatibility); // Check current compatibility type String currentCompatibility = client.getCompatibility(subject); System.out.println("Current compatibility level: " + currentCompatibility); } } import io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient; import io.confluent.kafka.schemaregistry.client.SchemaRegistryClient; import io.confluent.kafka.schemaregistry.rest.exceptions.RestClientException; import java.io.IOException; public class FullTransitiveExample { public static void main(String[] args) throws IOException, RestClientException { String schemaRegistryUrl = "http://localhost:8081"; SchemaRegistryClient client = new CachedSchemaRegistryClient(schemaRegistryUrl, 10); String subject = "<subject_name>"; String compatibility = "FULL_TRANSITIVE"; // Set compatibility type client.updateCompatibility(subject, compatibility); // Check current compatibility type String currentCompatibility = client.getCompatibility(subject); System.out.println("Current compatibility level: " + currentCompatibility); } } Pros ✅ Maximum flexibility and resilience. ✅ Guarantees compatibility with all schema versions. ✅ Maximum flexibility and resilience. ✅ Guarantees compatibility with all schema versions. Cons ❌ The strictest compatibility type. ❌ Requires careful planning for schema changes. ❌ The strictest compatibility type. ❌ Requires careful planning for schema changes. Comparison Table Compatibility Type Compatibility with Past Versions Compatibility with Future Versions Ease of Use Applicability BACKWARD_TRANSITIVE ✅ ❌ Medium Long-term data storage FORWARD_TRANSITIVE ❌ ✅ Medium Asynchronous updates FULL_TRANSITIVE ✅ ✅ Low Maximum resilience Compatibility Type Compatibility with Past Versions Compatibility with Future Versions Ease of Use Applicability BACKWARD_TRANSITIVE ✅ ❌ Medium Long-term data storage FORWARD_TRANSITIVE ❌ ✅ Medium Asynchronous updates FULL_TRANSITIVE ✅ ✅ Low Maximum resilience Compatibility Type Compatibility with Past Versions Compatibility with Future Versions Ease of Use Applicability Compatibility Type Compatibility Type Compatibility with Past Versions Compatibility with Past Versions Compatibility with Future Versions Compatibility with Future Versions Ease of Use Ease of Use Applicability Applicability BACKWARD_TRANSITIVE ✅ ❌ Medium Long-term data storage BACKWARD_TRANSITIVE BACKWARD_TRANSITIVE ✅ ✅ ❌ ❌ Medium Medium Long-term data storage Long-term data storage FORWARD_TRANSITIVE ❌ ✅ Medium Asynchronous updates FORWARD_TRANSITIVE FORWARD_TRANSITIVE ❌ ❌ ✅ ✅ Medium Medium Asynchronous updates Asynchronous updates FULL_TRANSITIVE ✅ ✅ Low Maximum resilience FULL_TRANSITIVE FULL_TRANSITIVE ✅ ✅ ✅ ✅ Low Low Maximum resilience Maximum resilience These compatibility types allow you to configure the system to meet your stability and flexibility requirements. The correct choice depends on your system architecture and use cases.
