Course Code:
sparktensorflowbespoke
Duration:
21 hours
Course Outline:
- Scala primer
- A quick introduction to Scala
- Labs : Getting know Scala
- Spark Basics
- Background and history
- Spark and Hadoop
- Spark concepts and architecture
- Spark eco system (core, spark sql, mlib, streaming)
- Labs : Installing and running Spark
- First Look at Spark
- Running Spark in local mode
- Spark web UI
- Spark shell
- Analyzing dataset – part 1
- Inspecting RDDs
- Labs: Spark shell exploration
- RDDs
- RDDs concepts
- Partitions
- RDD Operations / transformations
- RDD types
- Key-Value pair RDDs
- MapReduce on RDD
- Caching and persistence
- Labs : creating & inspecting RDDs; Caching RDDs
- Spark API programming
- Introduction to Spark API / RDD API
- Submitting the first program to Spark
- Debugging / logging
- Configuration properties
- Labs : Programming in Spark API, Submitting jobs
- Spark SQL
- SQL support in Spark
- Dataframes
- Defining tables and importing datasets
- Querying data frames using SQL
- Storage formats : JSON / Parquet
- Labs : Creating and querying data frames; evaluating data formats
- Mlib
- mlib intro
- mlib algorithms
- Labs : Writing mlib applications
- GraphX
- GraphX library overview
- GraphX APIs
- Labs : Processing graph data using Spark
- Spark Streaming
- Streaming overview
- Evaluating Streaming platforms
- Streaming operations
- Sliding window operations
- Labs : Writing spark streaming applications
- Spark and Hadoop
- Hadoop Intro (HDFS / YARN)
- Hadoop + Spark architecture
- Running Spark on Hadoop YARN
- Processing HDFS files using Spark
- Spark Performance and Tuning
- Broadcast variables
- Accumulators
- Memory management & caching
- Spark Operations
- Deploying Spark in production
- Sample deployment templates
- Configurations
- Monitoring
- Troubleshooting
Machine Learning and Recursive Neural Networks (RNN) basics
- NN and RNN
- Backprogation
- Long short-term memory (LSTM)
TensorFlow Basics
- Creation, Initializing, Saving, and Restoring TensorFlow variables
- Feeding, Reading and Preloading TensorFlow Data
- How to use TensorFlow infrastructure to train models at scale
- Visualizing and Evaluating models with TensorBoard
TensorFlow Mechanics 101
- Prepare the Data
- Download
- Inputs and Placeholders
- Build the Graph
- Inference
- Loss
- Training
- Train the Model
- The Graph
- The Session
- Train Loop
- Evaluate the Model
- Build the Eval Graph
- Eval Output
Advanced Usage
- Threading and Queues
- Distributed TensorFlow
- Writing Documentation and Sharing your Model
- Customizing Data Readers
- Using GPUs
- Manipulating TensorFlow Model Files
TensorFlow Serving
- Introduction
- Basic Serving Tutorial
- Advanced Serving Tutorial
- Serving Inception Model Tutorial