Introduction Hadoop Big Data Course

• Introduction to the Course

Top Ubuntu commands

Understand NameNode, DataNode, YARN and Hadoop Infrastructure

Hadoop Install

• Hadoop Installation & HDFS Commands

• Java based Mapreduce

# Hadoop 2.7  / 2.8.4

Learn HDFS commands

Setting up Java for mapreduce

Intro to Cloudera Hadoop & studying Cloudera Certification

SQL and NoSQL

• SQL, Hive and Pig Installation (RDBMS world and NoSQL world)

• More Hive and SQOOP (Cloudera – Sqoop and Hive on Cloudera.

• JDBC drivers.   

• Pig

• Intro to NoSQL, MongoDB, Hbase Installation

Understanding different databases 

Hive : 

1. Hive Partitions and Bucketing

2. Hive External and Internal Tables

Spark Scala Python

• Spark Installations and Commands

• Spark Scala Scala Sheets

• Hadoop Streaming Python Map Reduce

• PySpark – (Python – Basics). RDDs.

Running Spark-shell and importing data from csv files

PySpark – Running RDD

  Mid Term Projects

1. Pull data from csv online and move to Hive using hive import

2. Pull data from spark-shell and run map reduce for fox news first page

3. Create Data in MySQL and using SQOOP move it to HDFS

4. Using Jupyter Anaconda and Spark Context run count on file that has Fox news first page

5. Save raw data using delimiter comma, space, tab and pipe and move that into spark-context and spark shell

  Broadcasting Data – stream of data 

Kafka Message Broadcasting