Stream Processing with Kafka Streams

• An understanding of Apache Kafka
• Java programming experience

Kafka Streams is a client-side library for building applications and microservices whose data is passed to and from a Kafka messaging system. Traditionally, Apache Kafka has relied on Apache Spark or Apache Storm to process data between message producers and consumers. By calling the Kafka Streams API from within an application, data can be processed directly within Kafka, bypassing the need for sending the data to a separate cluster for processing.

In this instructor-led, live training, participants will learn how to integrate Kafka Streams into a set of sample Java applications that pass data to and from Apache Kafka for stream processing.

By the end of this training, participants will be able to:

• Understand Kafka Streams features and advantages over other stream processing frameworks
• Process stream data directly within a Kafka cluster
• Write a Java or Scala application or microservice that integrates with Kafka and Kafka Streams
• Write concise code that transforms input Kafka topics into output Kafka topics
• Build, package and deploy the application

Audience

• Developers

Format of the course

• Part lecture, part discussion, exercises and heavy hands-on practice

Notes

• To request a customized training for this course, please contact us to arrange

Introduction

• Kafka vs Spark, Flink, and Storm

Overview of Kafka Streams Features

• Stateful and stateless processing, event-time processing, DSL, event-time based windowing operations, etc.

Case Study: Kafka Streams API for Predictive Budgeting

Setting up the Development Environment

Creating a Streams Application

Starting the Kafka Cluster

Preparing the Topics and Input Data

Options for Processing Stream Data

• High-level Kafka Streams DSL
• Lower-level Processor

Transforming the Input Data

Inspecting the Output Data

Stopping the Kafka Cluster

Options for Deploying the Application

• Classic ops tools (Puppet, Chef and Salt)
• Docker
• WAR file

Troubleshooting

Summary and Conclusion