Course Code: platformengrdia
Duration: 21 hours
Prerequisites:
  • An understanding of basic data structures and algorithms
  • Experience with Java, Scala, or Python programming
  • Familiarity with basic concepts of databases and SQL

Audience

  • Software developers
  • Data engineers
  • Technical leads
Overview:

Platform Engineering is the discipline that combines software development, systems operations, and architecture to create large-scale data platforms.

This instructor-led, live training (online or onsite) is aimed at intermediate-level software developers and data engineers who wish to build and maintain robust platforms for data-intensive applications.

By the end of this training, participants will be able to:

  • Understand the challenges and solutions in platform engineering for big data.
  • Design and implement scalable data processing pipelines.
  • Utilize big data frameworks like Hadoop and Spark effectively.
  • Develop real-time analytics solutions using streaming data.
  • Orchestrate complex data workflows with tools like Apache Airflow.

Format of the Course

  • Interactive lecture and discussion.
  • Lots of exercises and practice.
  • Hands-on implementation in a live-lab environment.

Course Customization Options

  • To request a customized training for this course, please contact us to arrange.
Course Outline:

Foundations of Data-Intensive Platform Engineering

  • Introduction to data-intensive applications
  • Challenges in platform engineering for big data
  • Overview of data processing architectures

Data Modeling and Management

  • Principles of data modeling for scalability
  • Data storage options and optimization
  • Managing data lifecycle in a distributed environment

Big Data Processing Frameworks

  • Overview of big data processing tools (Hadoop, Spark, Flink)
  • Batch vs. stream processing
  • Setting up a big data processing pipeline

Real-Time Analytics Platforms

  • Architecting for real-time analytics
  • Stream processing engines (Kafka Streams, Apache Storm)
  • Building real-time dashboards and visualizations

Data Pipeline Orchestration

  • Workflow management with Apache Airflow and others
  • Automating data pipelines for efficiency
  • Monitoring and alerting for data pipelines

Platform Security and Compliance

  • Security best practices for data platforms
  • Ensuring data privacy and regulatory compliance
  • Implementing secure data access controls

Performance Tuning and Optimization

  • Techniques for optimizing data throughput and latency
  • Scaling strategies for data-intensive platforms
  • Performance benchmarking and monitoring

Case Studies and Best Practices

  • Analyzing successful data platform implementations
  • Lessons learned from industry leaders
  • Emerging trends in data-intensive platform engineering

Capstone Project

  • Designing a platform solution for a data-intensive application
  • Implementing a prototype of the data processing pipeline
  • Evaluating the platform's performance and scalability

Summary and Next Steps

Sites Published:

United Arab Emirates - Platform Engineering for Data-Intensive Applications

Qatar - Platform Engineering for Data-Intensive Applications

Egypt - Platform Engineering for Data-Intensive Applications

Saudi Arabia - Platform Engineering for Data-Intensive Applications

South Africa - Platform Engineering for Data-Intensive Applications

Brasil - Platform Engineering for Data-Intensive Applications

Canada - Platform Engineering for Data-Intensive Applications

中国 - Platform Engineering for Data-Intensive Applications

香港 - Platform Engineering for Data-Intensive Applications

澳門 - Platform Engineering for Data-Intensive Applications

台灣 - Platform Engineering for Data-Intensive Applications

USA - Platform Engineering for Data-Intensive Applications

Österreich - Platform Engineering for Data-Intensive Applications

Schweiz - Platform Engineering for Data-Intensive Applications

Deutschland - Platform Engineering for Data-Intensive Applications

Czech Republic - Platform Engineering for Data-Intensive Applications

Denmark - Platform Engineering for Data-Intensive Applications

Estonia - Platform Engineering for Data-Intensive Applications

Finland - Platform Engineering for Data-Intensive Applications

Greece - Platform Engineering for Data-Intensive Applications

Magyarország - Platform Engineering for Data-Intensive Applications

Ireland - Platform Engineering for Data-Intensive Applications

Luxembourg - Platform Engineering for Data-Intensive Applications

Latvia - Platform Engineering for Data-Intensive Applications

España - Platform Engineering for Data-Intensive Applications

Italia - Platform Engineering for Data-Intensive Applications

Lithuania - Platform Engineering for Data-Intensive Applications

Nederland - Platform Engineering for Data-Intensive Applications

Norway - Platform Engineering for Data-Intensive Applications

Portugal - Platform Engineering for Data-Intensive Applications

România - Platform Engineering for Data-Intensive Applications

Sverige - Platform Engineering for Data-Intensive Applications

Türkiye - Platform Engineering for Data-Intensive Applications

Malta - Platform Engineering for Data-Intensive Applications

Belgique - Platform Engineering for Data-Intensive Applications

France - Platform Engineering for Data-Intensive Applications

日本 - Platform Engineering for Data-Intensive Applications

Australia - Platform Engineering for Data-Intensive Applications

Malaysia - Platform Engineering for Data-Intensive Applications

New Zealand - Platform Engineering for Data-Intensive Applications

Philippines - Platform Engineering for Data-Intensive Applications

Singapore - Platform Engineering for Data-Intensive Applications

Thailand - Platform Engineering for Data-Intensive Applications

Vietnam - Platform Engineering for Data-Intensive Applications

India - Platform Engineering for Data-Intensive Applications

Argentina - Platform Engineering for Data-Intensive Applications

Chile - Platform Engineering for Data-Intensive Applications

Costa Rica - Platform Engineering for Data-Intensive Applications

Ecuador - Platform Engineering for Data-Intensive Applications

Guatemala - Platform Engineering for Data-Intensive Applications

Colombia - Platform Engineering for Data-Intensive Applications

México - Platform Engineering for Data-Intensive Applications

Panama - Platform Engineering for Data-Intensive Applications

Peru - Platform Engineering for Data-Intensive Applications

Uruguay - Platform Engineering for Data-Intensive Applications

Venezuela - Platform Engineering for Data-Intensive Applications

Polska - Platform Engineering for Data-Intensive Applications

United Kingdom - Platform Engineering for Data-Intensive Applications

South Korea - Platform Engineering for Data-Intensive Applications

Pakistan - Platform Engineering for Data-Intensive Applications

Sri Lanka - Platform Engineering for Data-Intensive Applications

Bulgaria - Platform Engineering for Data-Intensive Applications

Bolivia - Platform Engineering for Data-Intensive Applications

Indonesia - Platform Engineering for Data-Intensive Applications

Kazakhstan - Platform Engineering for Data-Intensive Applications

Moldova - Platform Engineering for Data-Intensive Applications

Morocco - Platform Engineering for Data-Intensive Applications

Tunisia - Platform Engineering for Data-Intensive Applications

Kuwait - Platform Engineering for Data-Intensive Applications

Oman - Platform Engineering for Data-Intensive Applications

Slovakia - Platform Engineering for Data-Intensive Applications

Kenya - Platform Engineering for Data-Intensive Applications

Nigeria - Platform Engineering for Data-Intensive Applications

Botswana - Platform Engineering for Data-Intensive Applications

Slovenia - Platform Engineering for Data-Intensive Applications

Croatia - Platform Engineering for Data-Intensive Applications

Serbia - Platform Engineering for Data-Intensive Applications

Bhutan - Platform Engineering for Data-Intensive Applications

Nepal - Platform Engineering for Data-Intensive Applications

Uzbekistan - Platform Engineering for Data-Intensive Applications