Apache Kafka provides schema management capabilities through the Schema Registry, including various levels of compatibility. In this article, we will discuss three types of transitive compatibility: BACKWARD_TRANSITIVE, FORWARD_TRANSITIVE, and FULL_TRANSITIVE. Additionally, we will analyze their pros, cons, and provide code examples. BACKWARD_TRANSITIVE Description Backward transitive compatibility means that a new version of the schema must be compatible with all previous versions. This is useful in systems with a long data lifecycle. Code Example curl -X PUT \
  -H "Content-Type: application/vnd.schemaregistry.v1+json" \
  --data '{"compatibility": "BACKWARD_TRANSITIVE"}' \
  http://localhost:8081/config/<subject_name> Java Code Example import io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient;
import io.confluent.kafka.schemaregistry.client.SchemaRegistryClient;
import io.confluent.kafka.schemaregistry.rest.exceptions.RestClientException;

import java.io.IOException;

public class BackwardTransitiveExample {
    public static void main(String[] args) throws IOException, RestClientException {
        String schemaRegistryUrl = "http://localhost:8081";
        SchemaRegistryClient client = new CachedSchemaRegistryClient(schemaRegistryUrl, 10);

        String subject = "<subject_name>";
        String compatibility = "BACKWARD_TRANSITIVE";

        // Set compatibility type
        client.updateCompatibility(subject, compatibility);

        // Check current compatibility type
        String currentCompatibility = client.getCompatibility(subject);
        System.out.println("Current compatibility level: " + currentCompatibility);
    }
} Pros ✅ Ensures compatibility with historical data.
✅ Useful for long-term data storage. Cons ❌ May restrict adding new fields or changing the structure. FORWARD_TRANSITIVE Description Forward transitive compatibility requires that a new version of the schema be compatible with all future versions. This is useful when consumers are updated asynchronously with producers. Code Example curl -X PUT \
  -H "Content-Type: application/vnd.schemaregistry.v1+json" \
  --data '{"compatibility": "FORWARD_TRANSITIVE"}' \
  http://localhost:8081/config/<subject_name> Java Code Example import io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient;
import io.confluent.kafka.schemaregistry.client.SchemaRegistryClient;
import io.confluent.kafka.schemaregistry.rest.exceptions.RestClientException;

import java.io.IOException;

public class ForwardTransitiveExample {
    public static void main(String[] args) throws IOException, RestClientException {
        String schemaRegistryUrl = "http://localhost:8081";
        SchemaRegistryClient client = new CachedSchemaRegistryClient(schemaRegistryUrl, 10);

        String subject = "<subject_name>";
        String compatibility = "FORWARD_TRANSITIVE";

        // Set compatibility type
        client.updateCompatibility(subject, compatibility);

        // Check current compatibility type
        String currentCompatibility = client.getCompatibility(subject);
        System.out.println("Current compatibility level: " + currentCompatibility);
    }
} Pros ✅ Ensures stability for old consumers with new data. ✅ Convenient for systems with asynchronous updates. Cons ❌ Does not guarantee compatibility with historical data. FULL_TRANSITIVE Description Full transitive compatibility combines the requirements of backward and forward compatibility. A new schema version must be compatible with both previous and future versions. Code Example curl -X PUT \
  -H "Content-Type: application/vnd.schemaregistry.v1+json" \
  --data '{"compatibility": "FULL_TRANSITIVE"}' \
  http://localhost:8081/config/<subject_name> Java Code Example import io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient;
import io.confluent.kafka.schemaregistry.client.SchemaRegistryClient;
import io.confluent.kafka.schemaregistry.rest.exceptions.RestClientException;

import java.io.IOException;

public class FullTransitiveExample {
    public static void main(String[] args) throws IOException, RestClientException {
        String schemaRegistryUrl = "http://localhost:8081";
        SchemaRegistryClient client = new CachedSchemaRegistryClient(schemaRegistryUrl, 10);

        String subject = "<subject_name>";
        String compatibility = "FULL_TRANSITIVE";

        // Set compatibility type
        client.updateCompatibility(subject, compatibility);

        // Check current compatibility type
        String currentCompatibility = client.getCompatibility(subject);
        System.out.println("Current compatibility level: " + currentCompatibility);
    }
} Pros ✅ Maximum flexibility and resilience.
✅ Guarantees compatibility with all schema versions. Cons ❌ The strictest compatibility type.
❌ Requires careful planning for schema changes. Comparison Table Compatibility Type

Compatibility with Past Versions

Compatibility with Future Versions

Ease of Use

Applicability



BACKWARD_TRANSITIVE

✅

❌

Medium

Long-term data storage



FORWARD_TRANSITIVE

❌

✅

Medium

Asynchronous updates



FULL_TRANSITIVE

✅

✅

Low

Maximum resilience These compatibility types allow you to configure the system to meet your stability and flexibility requirements. The correct choice depends on your system architecture and use cases. Apache Kafka provides schema management capabilities through the Schema Registry, including various levels of compatibility. In this article, we will discuss three types of transitive compatibility: BACKWARD_TRANSITIVE, FORWARD_TRANSITIVE, and FULL_TRANSITIVE. Additionally, we will analyze their pros, cons, and provide code examples. BACKWARD_TRANSITIVE Description Backward transitive compatibility means that a new version of the schema must be compatible with all previous versions. This is useful in systems with a long data lifecycle. all Code Example curl -X PUT \
  -H "Content-Type: application/vnd.schemaregistry.v1+json" \
  --data '{"compatibility": "BACKWARD_TRANSITIVE"}' \
  http://localhost:8081/config/<subject_name> curl -X PUT \
  -H "Content-Type: application/vnd.schemaregistry.v1+json" \
  --data '{"compatibility": "BACKWARD_TRANSITIVE"}' \
  http://localhost:8081/config/<subject_name> Java Code Example import io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient;
import io.confluent.kafka.schemaregistry.client.SchemaRegistryClient;
import io.confluent.kafka.schemaregistry.rest.exceptions.RestClientException;

import java.io.IOException;

public class BackwardTransitiveExample {
    public static void main(String[] args) throws IOException, RestClientException {
        String schemaRegistryUrl = "http://localhost:8081";
        SchemaRegistryClient client = new CachedSchemaRegistryClient(schemaRegistryUrl, 10);

        String subject = "<subject_name>";
        String compatibility = "BACKWARD_TRANSITIVE";

        // Set compatibility type
        client.updateCompatibility(subject, compatibility);

        // Check current compatibility type
        String currentCompatibility = client.getCompatibility(subject);
        System.out.println("Current compatibility level: " + currentCompatibility);
    }
} import io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient;
import io.confluent.kafka.schemaregistry.client.SchemaRegistryClient;
import io.confluent.kafka.schemaregistry.rest.exceptions.RestClientException;

import java.io.IOException;

public class BackwardTransitiveExample {
    public static void main(String[] args) throws IOException, RestClientException {
        String schemaRegistryUrl = "http://localhost:8081";
        SchemaRegistryClient client = new CachedSchemaRegistryClient(schemaRegistryUrl, 10);

        String subject = "<subject_name>";
        String compatibility = "BACKWARD_TRANSITIVE";

        // Set compatibility type
        client.updateCompatibility(subject, compatibility);

        // Check current compatibility type
        String currentCompatibility = client.getCompatibility(subject);
        System.out.println("Current compatibility level: " + currentCompatibility);
    }
} Pros ✅ Ensures compatibility with historical data. ✅ Useful for long-term data storage. ✅ Ensures compatibility with historical data. ✅ Useful for long-term data storage. Cons ❌ May restrict adding new fields or changing the structure. ❌ May restrict adding new fields or changing the structure. FORWARD_TRANSITIVE Description Forward transitive compatibility requires that a new version of the schema be compatible with all future versions. This is useful when consumers are updated asynchronously with producers. all Code Example curl -X PUT \
  -H "Content-Type: application/vnd.schemaregistry.v1+json" \
  --data '{"compatibility": "FORWARD_TRANSITIVE"}' \
  http://localhost:8081/config/<subject_name> curl -X PUT \
  -H "Content-Type: application/vnd.schemaregistry.v1+json" \
  --data '{"compatibility": "FORWARD_TRANSITIVE"}' \
  http://localhost:8081/config/<subject_name> Java Code Example import io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient;
import io.confluent.kafka.schemaregistry.client.SchemaRegistryClient;
import io.confluent.kafka.schemaregistry.rest.exceptions.RestClientException;

import java.io.IOException;

public class ForwardTransitiveExample {
    public static void main(String[] args) throws IOException, RestClientException {
        String schemaRegistryUrl = "http://localhost:8081";
        SchemaRegistryClient client = new CachedSchemaRegistryClient(schemaRegistryUrl, 10);

        String subject = "<subject_name>";
        String compatibility = "FORWARD_TRANSITIVE";

        // Set compatibility type
        client.updateCompatibility(subject, compatibility);

        // Check current compatibility type
        String currentCompatibility = client.getCompatibility(subject);
        System.out.println("Current compatibility level: " + currentCompatibility);
    }
} import io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient;
import io.confluent.kafka.schemaregistry.client.SchemaRegistryClient;
import io.confluent.kafka.schemaregistry.rest.exceptions.RestClientException;

import java.io.IOException;

public class ForwardTransitiveExample {
    public static void main(String[] args) throws IOException, RestClientException {
        String schemaRegistryUrl = "http://localhost:8081";
        SchemaRegistryClient client = new CachedSchemaRegistryClient(schemaRegistryUrl, 10);

        String subject = "<subject_name>";
        String compatibility = "FORWARD_TRANSITIVE";

        // Set compatibility type
        client.updateCompatibility(subject, compatibility);

        // Check current compatibility type
        String currentCompatibility = client.getCompatibility(subject);
        System.out.println("Current compatibility level: " + currentCompatibility);
    }
} Pros ✅ Ensures stability for old consumers with new data. ✅ Ensures stability for old consumers with new data. ✅ Convenient for systems with asynchronous updates. ✅ Convenient for systems with asynchronous updates. Cons ❌ Does not guarantee compatibility with historical data. ❌ Does not guarantee compatibility with historical data. FULL_TRANSITIVE Description Full transitive compatibility combines the requirements of backward and forward compatibility. A new schema version must be compatible with both previous and future versions. Code Example curl -X PUT \
  -H "Content-Type: application/vnd.schemaregistry.v1+json" \
  --data '{"compatibility": "FULL_TRANSITIVE"}' \
  http://localhost:8081/config/<subject_name> curl -X PUT \
  -H "Content-Type: application/vnd.schemaregistry.v1+json" \
  --data '{"compatibility": "FULL_TRANSITIVE"}' \
  http://localhost:8081/config/<subject_name> Java Code Example import io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient;
import io.confluent.kafka.schemaregistry.client.SchemaRegistryClient;
import io.confluent.kafka.schemaregistry.rest.exceptions.RestClientException;

import java.io.IOException;

public class FullTransitiveExample {
    public static void main(String[] args) throws IOException, RestClientException {
        String schemaRegistryUrl = "http://localhost:8081";
        SchemaRegistryClient client = new CachedSchemaRegistryClient(schemaRegistryUrl, 10);

        String subject = "<subject_name>";
        String compatibility = "FULL_TRANSITIVE";

        // Set compatibility type
        client.updateCompatibility(subject, compatibility);

        // Check current compatibility type
        String currentCompatibility = client.getCompatibility(subject);
        System.out.println("Current compatibility level: " + currentCompatibility);
    }
} import io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient;
import io.confluent.kafka.schemaregistry.client.SchemaRegistryClient;
import io.confluent.kafka.schemaregistry.rest.exceptions.RestClientException;

import java.io.IOException;

public class FullTransitiveExample {
    public static void main(String[] args) throws IOException, RestClientException {
        String schemaRegistryUrl = "http://localhost:8081";
        SchemaRegistryClient client = new CachedSchemaRegistryClient(schemaRegistryUrl, 10);

        String subject = "<subject_name>";
        String compatibility = "FULL_TRANSITIVE";

        // Set compatibility type
        client.updateCompatibility(subject, compatibility);

        // Check current compatibility type
        String currentCompatibility = client.getCompatibility(subject);
        System.out.println("Current compatibility level: " + currentCompatibility);
    }
} Pros ✅ Maximum flexibility and resilience. ✅ Guarantees compatibility with all schema versions. ✅ Maximum flexibility and resilience. ✅ Guarantees compatibility with all schema versions. Cons ❌ The strictest compatibility type. ❌ Requires careful planning for schema changes. ❌ The strictest compatibility type. ❌ Requires careful planning for schema changes. Comparison Table Compatibility Type

Compatibility with Past Versions

Compatibility with Future Versions

Ease of Use

Applicability



BACKWARD_TRANSITIVE

✅

❌

Medium

Long-term data storage



FORWARD_TRANSITIVE

❌

✅

Medium

Asynchronous updates



FULL_TRANSITIVE

✅

✅

Low

Maximum resilience Compatibility Type

Compatibility with Past Versions

Compatibility with Future Versions

Ease of Use

Applicability BACKWARD_TRANSITIVE

✅

❌

Medium

Long-term data storage FORWARD_TRANSITIVE

❌

✅

Medium

Asynchronous updates FULL_TRANSITIVE

✅

✅

Low

Maximum resilience Compatibility Type Compatibility with Past Versions Compatibility with Future Versions Ease of Use Applicability Compatibility Type Compatibility Type Compatibility with Past Versions Compatibility with Past Versions Compatibility with Future Versions Compatibility with Future Versions Ease of Use Ease of Use Applicability Applicability BACKWARD_TRANSITIVE ✅ ❌ Medium Long-term data storage BACKWARD_TRANSITIVE BACKWARD_TRANSITIVE ✅ ✅ ❌ ❌ Medium Medium Long-term data storage Long-term data storage FORWARD_TRANSITIVE ❌ ✅ Medium Asynchronous updates FORWARD_TRANSITIVE FORWARD_TRANSITIVE ❌ ❌ ✅ ✅ Medium Medium Asynchronous updates Asynchronous updates FULL_TRANSITIVE ✅ ✅ Low Maximum resilience FULL_TRANSITIVE FULL_TRANSITIVE ✅ ✅ ✅ ✅ Low Low Maximum resilience Maximum resilience These compatibility types allow you to configure the system to meet your stability and flexibility requirements. The correct choice depends on your system architecture and use cases.

Transitive Compatibility Types in Schema Registry for Apache Kafka

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

Building a Simple Custom Processor With Apache Ni-Fi

The Noonification: If You Cant Beat Em, Buy Em! (10/18/2023)

The Noonification: The Conversational AI Arms Race Has Begun (2/9/2023)

3 Top Resources To Learn About Apache Kafka

47 Stories To Learn About Kafka

Building a Simple Custom Processor With Apache Ni-Fi

The Noonification: If You Cant Beat Em, Buy Em! (10/18/2023)

The Noonification: The Conversational AI Arms Race Has Begun (2/9/2023)

3 Top Resources To Learn About Apache Kafka

47 Stories To Learn About Kafka

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps