Course Code: sparkstreaming
Duration: 7 hours
Prerequisites:
  • Experience with Python and Apache Kafka
  • Familiarity with stream-processing platforms

Audience

  • Data engineers
  • Data scientists
  • Programmers
Overview:

Apache Spark Streaming is a scalable, open source stream processing system that allows users to process real-time data from supported sources. Spark Streaming enables fault-tolerant processing of data streams.

This instructor-led, live training (online or onsite) is aimed at data engineers, data scientists, and programmers who wish to use Spark Streaming features in processing and analyzing real-time data.

By the end of this training, participants will be able to use Spark Streaming to process live data streams for use in databases, filesystems, and live dashboards.

Format of the Course

  • Interactive lecture and discussion.
  • Lots of exercises and practice.
  • Hands-on implementation in a live-lab environment.

Course Customization Options

  • To request a customized training for this course, please contact us to arrange.
Course Outline:

Introduction

Overview of Spark Streaming Features and Architecture

  • Supported data sources
  • Core APIs

Preparing the Environment

  • Dependencies
  • Spark and streaming context
  • Connecting to Kafka

Processing Messages

  • Parsing inbound messages as JSON
  • ETL processes
  • Starting the streaming context

Performing a Windowed Stream Processing

  • Slide interval
  • Checkpoint delivery configuration
  • Launching the environment

Prototyping the Processing Code

  • Connecting to a Kafka topic
  • Retrieving JSON from data source using Paw
  • Variations and additional processing

Streaming the Code

  • Job control variables
  • Defining values to match
  • Functions and conditions

Acquiring Stream Output

  • Counters
  • Kafka output (matched and non-matched)

Troubleshooting

Summary and Conclusion

Sites Published:

United Arab Emirates - Spark Streaming with Python and Kafka

Qatar - Spark Streaming with Python and Kafka

Egypt - Spark Streaming with Python and Kafka

Saudi Arabia - Spark Streaming with Python and Kafka

South Africa - Spark Streaming with Python and Kafka

Brasil - Spark Streaming with Python and Kafka

Canada - Spark Streaming with Python and Kafka

中国 - Spark Streaming with Python and Kafka

香港 - Spark Streaming with Python and Kafka

澳門 - Spark Streaming with Python and Kafka

台灣 - Spark Streaming with Python and Kafka

USA - Spark Streaming with Python and Kafka

Österreich - Spark Streaming with Python and Kafka

Schweiz - Spark Streaming with Python and Kafka

Deutschland - Spark Streaming with Python and Kafka

Czech Republic - Spark Streaming with Python and Kafka

Denmark - Spark Streaming with Python and Kafka

Estonia - Spark Streaming with Python and Kafka

Finland - Spark Streaming with Python and Kafka

Greece - Spark Streaming with Python and Kafka

Magyarország - Spark Streaming with Python and Kafka

Ireland - Spark Streaming with Python and Kafka

Luxembourg - Spark Streaming with Python and Kafka

Latvia - Spark Streaming with Python and Kafka

España - Spark Streaming with Python and Kafka

Italia - Spark Streaming with Python and Kafka

Lithuania - Spark Streaming with Python and Kafka

Nederland - Spark Streaming with Python and Kafka

Norway - Spark Streaming with Python and Kafka

Portugal - Spark Streaming with Python and Kafka

România - Spark Streaming with Python and Kafka

Sverige - Spark Streaming with Python and Kafka

Türkiye - Spark Streaming with Python and Kafka

Malta - Spark Streaming with Python and Kafka

Belgique - Spark Streaming with Python and Kafka

France - Spark Streaming with Python and Kafka

日本 - Spark Streaming with Python and Kafka

Australia - Spark Streaming with Python and Kafka

Malaysia - Spark Streaming with Python and Kafka

New Zealand - Spark Streaming with Python and Kafka

Philippines - Spark Streaming with Python and Kafka

Singapore - Spark Streaming with Python and Kafka

Thailand - Spark Streaming with Python and Kafka

Vietnam - Spark Streaming with Python and Kafka

India - Spark Streaming with Python and Kafka

Argentina - Spark Streaming with Python and Kafka

Chile - Spark Streaming with Python and Kafka

Costa Rica - Spark Streaming with Python and Kafka

Ecuador - Spark Streaming with Python and Kafka

Guatemala - Spark Streaming with Python and Kafka

Colombia - Spark Streaming with Python and Kafka

México - Spark Streaming with Python and Kafka

Panama - Spark Streaming with Python and Kafka

Peru - Spark Streaming with Python and Kafka

Uruguay - Spark Streaming with Python and Kafka

Venezuela - Spark Streaming with Python and Kafka

Polska - Spark Streaming with Python and Kafka

United Kingdom - Spark Streaming with Python and Kafka

South Korea - Spark Streaming with Python and Kafka

Pakistan - Spark Streaming with Python and Kafka

Sri Lanka - Spark Streaming with Python and Kafka

Bulgaria - Spark Streaming with Python and Kafka

Bolivia - Spark Streaming with Python and Kafka

Indonesia - Spark Streaming with Python and Kafka

Kazakhstan - Spark Streaming with Python and Kafka

Moldova - Spark Streaming with Python and Kafka

Morocco - Spark Streaming with Python and Kafka

Tunisia - Spark Streaming with Python and Kafka

Kuwait - Spark Streaming with Python and Kafka

Oman - Spark Streaming with Python and Kafka

Slovakia - Spark Streaming with Python and Kafka

Kenya - Spark Streaming with Python and Kafka

Nigeria - Spark Streaming with Python and Kafka

Botswana - Spark Streaming with Python and Kafka

Slovenia - Spark Streaming with Python and Kafka

Croatia - Spark Streaming with Python and Kafka

Serbia - Spark Streaming with Python and Kafka

Bhutan - Spark Streaming with Python and Kafka

Nepal - Spark Streaming with Python and Kafka

Uzbekistan - Spark Streaming with Python and Kafka