Course Code: bigdata
Prerequisites:
  • A general understanding of math
  • A general understanding of programming
  • A general understanding of databases
Overview:

Participants who complete this course will gain a solid understanding of Big Data and its related technologies, methodologies and tools.

Participants will have the opportunity to put their new-gained knowledge into practice by way of exercises and tests. Group interaction and instructor feedback make up an important component of the class.

The course starts with an introduction to elemental concepts of Big Data, then progresses into the programming languages and methodologies used to perform Data Analysis. Finally, we discuss the tools that enable Big Data storage, Distributed Processing, and scalability.

Audience

  • Developers / programmers
  • IT consultants

Format of the course

  • Part lecture, part discussion, hands-on practice

Course Outline:

DAY 01

Data Analysis and Big Data concepts - a brief review

  1. VVVV (Velocity, Volume, Variety, Veracity) definition
  2. Limits to traditional data processing capacity
  3. Distributed Processing
  4. Statistical Analysis
  5. Machine Learning Analysis Types
  6. Data Visualization
  7. Distributed Processing (e.g. map-reduce)

Languages used for Data Analysis

  • R language (intermediate-to-advanced)

DAY 02

Languages used for Data Analysis

  • Python (crash course)

DAY 03

Approaches to Data Analysis

  1. Statistical Analysis
    1. Time Series analysis
    2. Forecasting with Correlation and Regression models
    3. Inferential Statistics (estimating)
    4. Descriptive Statistics in Big Data sets (e.g. calculating mean)

DAY 04

Approaches to Data Analysis

  1. Machine Learning
    1. Supervised vs unsupervised learning
    2. Classification and clustering
    3. Estimating cost of specific methods
    4. Filter

DAY 05

Approaches to Data Analysis

  1. Natural Language Processing
    1. Processing text
    2. Understaing meaning of the text
    3. Automatic text generation
    4. Sentiment/Topic Analysis
  2. Computer Vision

DAY 06

Big Data tooling

  1. Data storage solution (SQL, NoSQL, hierarchical, object oriented, document oriented)
    1. MySQL, Cassandra, MongoDB, Elasticsearch, HDFS, etc.
    2. Choosing right solution to the problem

DAY 07

Big Data tooling

  1. Distributed Processing
    1. Spark
    2. Machine Learning with Spark (MLLib)
    3. Spark SQL

DAY 08

Big Data tooling

  1. Scalability
    1. Public cloud (AWS, Google, etc...)
    2. Private cloud (OpenStack, Cloud Foundry)
    3. Auto-scalability

Closing remarks