Master Big Data Ingestion and Analytics with Flume, Sqoop, Hive and Spark

Video description

Complete course on Sqoop, Flume, and Hive: Great for CCA175 and Hortonworks Spark Certification preparation

About This Video

Learn Sqoop, Flume, and Hive and successfully achieve CCA175 and Hortonworks Spark Certification
Understand the Hadoop Distributed File System (HDFS), along with exploring Hadoop commands to work effectively with HDFS

In Detail

In this course, you will start by learning about the …

Master Big Data Ingestion and Analytics with Flume, Sqoop, Hive and Spark

Video description

Complete course on Sqoop, Flume, and Hive: Great for CCA175 and Hortonworks Spark Certification preparation

About This Video

Learn Sqoop, Flume, and Hive and successfully achieve CCA175 and Hortonworks Spark Certification
Understand the Hadoop Distributed File System (HDFS), along with exploring Hadoop commands to work effectively with HDFS

In Detail

In this course, you will start by learning about the Hadoop Distributed File System (HDFS) and the most common Hadoop commands required to work with HDFS. Next, you'll be introduced to Sqoop Import, which will help you gain insights into the lifecycle of the Sqoop command and how to use the import command to migrate data from MySQL to HDFS, and from MySQL to Hive.

In addition to this, you will get up to speed with Sqoop Export for migrating data effectively, along with using Apache Flume to ingest data. As you progress, you will delve into Apache Hive, external and managed tables, working with different files, and Parquet and Avro. Toward the concluding section, you will focus on Spark DataFrames and Spark SQL.

By the end of this course, you will have gained comprehensive insights into big data ingestion and analytics with Flume, Sqoop, Hive, and Spark.

Publisher resources

Download Example Code

Chapter 1 : Hadoop Introduction

HDFS and Hadoop Commands 00:09:16

Chapter 2 : Sqoop Import

Sqoop Introduction 00:08:07

Managing Target Directories 00:02:38

Working with Different File Formats 00:04:26

Working with Different Compressions 00:06:17

Conditional Imports 00:04:26

Split-by and Boundary Queries 00:08:27

Field delimeters 00:03:18

Incremental Appends 00:03:13

Sqoop Hive Import 00:03:32

Sqoop List Tables/Database 00:02:45

Sqoop Import Practice1 00:04:57

Sqoop Import Practice2 00:04:17

Sqoop Import Practice3 00:03:32

Chapter 3 : Sqoop Export

Export from Hdfs to Mysql 00:03:39

Export from Hive to Mysql 00:02:30

Chapter 4 : Apache Flume

Flume Introduction & Architecture 00:02:32

Exec Source and Logger Sink 00:03:42

Moving data from Twitter to HDFS 00:09:25

Moving data from NetCat to HDFS 00:04:40

Flume Interceptors 00:01:56

Flume Interceptor Example 00:04:54

Flume Multi-Agent Flow 00:06:49

Flume Consolidation 00:06:11

Chapter 5 : Apache Hive

Hive Introduction 00:03:42

Hive Database 00:03:05

Hive Managed Tables 00:06:24

Hive External Tables 00:02:27

Hive Inserts 00:05:31

Hive Analytics 00:04:22

Working with Parquet 00:03:25

Compressing Parquet 00:04:27

Working with Fixed File Format 00:03:04

Alter Command 00:06:13

Hive String Functions 00:06:22

Hive Date Functions 00:05:41

Hive Partitioning 00:07:26

Hive Bucketing 00:03:46

Chapter 6 : Spark Introduction

Spark Introduction 00:03:46

Resilient Distributed Datasets 00:02:53

Cluster Overview 00:06:52

Directed Acyclic Graph (DAG) & Stages 00:10:07

Chapter 7 : Spark Transformations & Actions

Map/FlatMap Transformation 00:04:29

Filter/Intersection 00:04:01

Union/Distinct Transformation 00:02:23

GroupByKey/ Group people based on Birthday months 00:05:54

ReduceByKey / Total Number of students in each Subject 00:06:44

SortByKey / Sort students based on their rollno 00:06:03

MapPartition / MapPartitionWithIndex 00:06:20

Change number of Partitions 00:03:34

Join / Join email address based on customer name 00:03:06

Spark Actions 00:06:06

Chapter 8 : Spark RDD Practice

Scala Tuples 00:03:05

Extract Error Logs from log files 00:10:23

Frequency of word in Text File 00:08:35

Population of each City 00:03:53

Orders placed by Customers 00:09:21

Movie Average Rating greater than 3 00:07:04

Chapter 9 : Spark Dataframes & Spark SQL

Dataframe Intro 00:02:17

Dafaframe from Json Files 00:04:46

Dataframe from Parquet Files 00:01:41

Dataframe from CSV Files 00:08:05

Dataframe from Avro/XML Files 00:04:54

Working with Different Compressions 00:06:34

DataFrame API Part1 00:04:51

DataFrame API Part2 00:06:24

Spark SQL 00:01:33

Working with Hive Tables in Spark 00:01:29

Start your Free Trial

Self paced

Go to the Course
We have partnered with providers to bring you collection of courses, When you buy through links on our site, we may earn an affiliate commission from provider.