Mastering Kafka Streams and ksqlDB

Appendix A. Kafka Streams Configuration

Kafka Streams is highly configurable, and the available parameters, as well as their default values, are a moving target. Therefore, the configuration properties listed in this appendix should be used as a starting point for familiarizing yourself with the various configuration parameters, but please refer to the official documentation for the latest information.

Configuration Management

In this book, we’ve been configuring Kafka Streams applications by creating a Properties instance, and setting the various configuration parameters manually. An example of this strategy is shown in the following code:

class App {

  public static void main(String[] args) {
    Topology topology = GreeterTopology.build();

    Properties config = new Properties();
    config.put(StreamsConfig.APPLICATION_ID_CONFIG, "dev-consumer");
    config.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "kafka:9092");

    KafkaStreams streams = new KafkaStreams(topology, config);

    // ....
  }
}

However, when it’s time to ship your application to production, you should consider loading your configuration from a file, instead of hardcoding the values directly in your application. Being able to make configuration changes without touching code is less error-prone, and if the configuration file can be overridden at runtime (e.g., via a system flag), then it’s extremely easy to manage multiple deployments of your application.¹

A full discussion of configuration management strategies for Java applications ...