Course Code: sparkstreaming
Duration: 7 hours
Prerequisites:
  • 具有 Python 和 Apache Kafka 的经验
  • 熟悉流处理平台

观众

  • 数据工程师
  • 数据科学家
  • 程序员
Overview:

Apache Spark 流式处理是一种可扩展的开源流 处理系统,允许用户处理来自受支持源的实时数据。Spark Streaming 支持对数据流进行容错处理。

这种以讲师为主导的现场培训(现场或远程)针对希望使用Spark Streaming功能处理和分析实时数据的数据工程师,数据科学家和程序员。

在本次培训结束时,参与者将能够使用 Spark Streaming 处理实时数据流,以便在数据库、文件系统和实时仪表板中使用。

课程形式

  • 互动讲座和讨论。
  • 大量的练习和练习。
  • 在现场实验室环境中实际实施。

课程定制选项

  • 如需申请此课程的定制培训,请联系我们进行安排。
Course Outline:

介绍

Spark Streaming 功能和体系结构概述

  • 支持的数据源
  • 核心 API

准备环境

  • 依赖
  • Spark 和流式处理上下文
  • 连接到 Kafka

处理消息

  • 将入站消息解析为 JSON
  • ETL 过程
  • 启动流式处理上下文

执行窗口化 Stream Processing

  • 滑动间隔
  • 检查点传送配置
  • 启动环境

对处理代码进行原型设计

  • 连接到 Kafka 主题
  • 使用 Paw 从数据源检索 JSON
  • 变更和额外处理

流式传输代码

  • 作业控制变量
  • 定义要匹配的值
  • 功能和条件

获取流输出

  • 计数器
  • Kafka 输出(匹配和不匹配)

故障 排除

总结和结论

Sites Published:

United Arab Emirates - Spark Streaming with Python and Kafka

Qatar - Spark Streaming with Python and Kafka

Egypt - Spark Streaming with Python and Kafka

Saudi Arabia - Spark Streaming with Python and Kafka

South Africa - Spark Streaming with Python and Kafka

Brasil - Spark Streaming with Python and Kafka

Canada - Spark Streaming with Python and Kafka

中国 - Spark Streaming with Python and Kafka

香港 - Spark Streaming with Python and Kafka

澳門 - Spark Streaming with Python and Kafka

台灣 - Spark Streaming with Python and Kafka

USA - Spark Streaming with Python and Kafka

Österreich - Spark Streaming with Python and Kafka

Schweiz - Spark Streaming with Python and Kafka

Deutschland - Spark Streaming with Python and Kafka

Czech Republic - Spark Streaming with Python and Kafka

Denmark - Spark Streaming with Python and Kafka

Estonia - Spark Streaming with Python and Kafka

Finland - Spark Streaming with Python and Kafka

Greece - Spark Streaming with Python and Kafka

Magyarország - Spark Streaming with Python and Kafka

Ireland - Spark Streaming with Python and Kafka

Luxembourg - Spark Streaming with Python and Kafka

Latvia - Spark Streaming with Python and Kafka

España - Spark Streaming with Python and Kafka

Italia - Spark Streaming with Python and Kafka

Lithuania - Spark Streaming with Python and Kafka

Nederland - Spark Streaming with Python and Kafka

Norway - Spark Streaming with Python and Kafka

Portugal - Spark Streaming with Python and Kafka

România - Spark Streaming with Python and Kafka

Sverige - Spark Streaming with Python and Kafka

Türkiye - Spark Streaming with Python and Kafka

Malta - Spark Streaming with Python and Kafka

Belgique - Spark Streaming with Python and Kafka

France - Spark Streaming with Python and Kafka

日本 - Spark Streaming with Python and Kafka

Australia - Spark Streaming with Python and Kafka

Malaysia - Spark Streaming with Python and Kafka

New Zealand - Spark Streaming with Python and Kafka

Philippines - Spark Streaming with Python and Kafka

Singapore - Spark Streaming with Python and Kafka

Thailand - Spark Streaming with Python and Kafka

Vietnam - Spark Streaming with Python and Kafka

India - Spark Streaming with Python and Kafka

Argentina - Spark Streaming with Python and Kafka

Chile - Spark Streaming with Python and Kafka

Costa Rica - Spark Streaming with Python and Kafka

Ecuador - Spark Streaming with Python and Kafka

Guatemala - Spark Streaming with Python and Kafka

Colombia - Spark Streaming with Python and Kafka

México - Spark Streaming with Python and Kafka

Panama - Spark Streaming with Python and Kafka

Peru - Spark Streaming with Python and Kafka

Uruguay - Spark Streaming with Python and Kafka

Venezuela - Spark Streaming with Python and Kafka

Polska - Spark Streaming with Python and Kafka

United Kingdom - Spark Streaming with Python and Kafka

South Korea - Spark Streaming with Python and Kafka

Pakistan - Spark Streaming with Python and Kafka

Sri Lanka - Spark Streaming with Python and Kafka

Bulgaria - Spark Streaming with Python and Kafka

Bolivia - Spark Streaming with Python and Kafka

Indonesia - Spark Streaming with Python and Kafka

Kazakhstan - Spark Streaming with Python and Kafka

Moldova - Spark Streaming with Python and Kafka

Morocco - Spark Streaming with Python and Kafka

Tunisia - Spark Streaming with Python and Kafka

Kuwait - Spark Streaming with Python and Kafka

Oman - Spark Streaming with Python and Kafka

Slovakia - Spark Streaming with Python and Kafka

Kenya - Spark Streaming with Python and Kafka

Nigeria - Spark Streaming with Python and Kafka

Botswana - Spark Streaming with Python and Kafka

Slovenia - Spark Streaming with Python and Kafka

Croatia - Spark Streaming with Python and Kafka

Serbia - Spark Streaming with Python and Kafka

Bhutan - Spark Streaming with Python and Kafka

Nepal - Spark Streaming with Python and Kafka

Uzbekistan - Spark Streaming with Python and Kafka