Scala & Spark-Master Big Data with Scala and Spark

Table of Contents

Chapter 1 : Introduction

Why Scala

Scala Applications

About the Instructor

Introduction to Course

Projects Overview

Chapter 2 : Overview

What is Scala

Scala Setup (Local Machine)

Hello World

Scala Setup (Online)

Variables in Scala

Arithmetic Operations on Variables-1

Arithmetic Operations on Variables-2

Quiz (Arithmetic Operations)

Solution (Arithmetic Operations)

Quiz (Strings)

Solution (Strings)

Type Casting

Taking Input from User

Quiz (User Input and Type Casting)

Solution (User Input and Type Casting)

Chapter 3 : Flow Control

Overview of Control Statements

If else statements

Conditions in If

Quiz (if statement)

Solution (if statement)

Nested If Else

Quiz (Nested If Else)

Solution (nested if else)

Logical Operators

Quiz (Logical Operators)

Solution (Logical Operators)

If Else If

Quiz (If Else If)

Solution (If Else If)

Overview of Loops

Overview of While Loop

While Loop

Quiz (While Loop)

Solution 1 (While Loop)

Solution 2 (While Loop)

Do While Loop

For Loop

Quiz (For Loop)

Solution (For Loop)

Quiz(For Loop)

Solution(For Loop)

Break

Break Fix

Project Overview

Project Solution Design

Project Solution Code 1

Project Solution Code 2

Project Solution Code 3

Project Solution Code 4

Chapter 4 : Functions

Overview of Functions

Writing Addition Function

Quiz (Basic Functions)

Solution (Basic Functions)

Functions Common Issues

Named Arguments

Quiz (String Concatenation Function)

Solution (String Concatenation Function)

Quiz (Dividing Code in Functions)

Solution (Dividing Code in Functions)

Default Arguments

Quiz (Default Arguments)

Solution (Default Arguments)

Anonymous Functions

Quiz (Anonymous Functions)

Solution (Anonymous Functions)

Scopes

Project Overview

Checking Credentials

Prompting the Menu

Basic Functions

Breaking Code in More Functions

Final Run

Chapter 5 : Classes

Introduction to Classes

Creating Class

Class Constructor

Functions and Classes

Project Overview

Basic Structure

Final Run

Chapter 6 : Data Structures

Introduction of Data Structures

Lists introduction

Lists Create and Delete Elements

Lists Take

ListBuffer Introduction

Add Data in ListBuffer

Remove Data from ListBuffer

Take Data from ListBuffer

Project Overview

Project Architecture Discussion

Project Architecture Implementation

User Input for Objects

Implementing the Control Flow

Creating Required Functions Inside Class

Overview of Maps

Creating Maps

Check Key in Map

Update Value in Map

Add and Remove Items from Maps

Iterating in Maps

Project Overview

Project Architecture

Project Structure Code

Using Maps for Word Count

Final Run

Sets Overview

Add and Remove Item from the Set

Set Operations

Overview of Stack

Push and Pop in Stack

Stack Attributes

Project Overview

Project Architecture

Extra Closing Bracket Use Case

Extra Starting Bracket Use Case

Chapter 7 : Project

Project Introduction

Why Spark

Hadoop Ecosystem

Spark Architecture

Spark Ecosystem

DataBricks Account

Setting Up DataBricks Cluster

Spark Local Setup

Spark Hadoop Setup

Spark RDDs

Spark RDDs (textFile, collect)

Spark Local Run

Understanding Map

Understanding Flat Map

Understanding Reduce by Key

Word Count Example

Spark DFs

Spark DF Read Data

Spark Print Schema, Select

Spark Group By

Spark DF Write

Creating S3 Bucket

Creating Database in RDS

Performing ETL