Scalable Machine Learning on Big Data using Apache Spark
This course will empower you with the skills to scale data science and machine learning (ML) tasks on Big Data sets using Apache Spark. Most real world machine learning work involves very large data sets that go beyond the CPU, memory and storage limitations of a single computer. Apache Spark is an open source framework that leverages cluster computing and distributed storage to process extremely large data sets in an efficient and cost effective manner. Therefore an applied knowledge of working with Apache Spark is a great asset and potential differentiator for a Machine Learning engineer.
After completing this course, you will be able to:
- gain a practical understanding of Apache Spark, and apply it to solve machine learning problems involving both small and big data
- understand how parallel code is written, capable of running on thousands of CPUs.
- make use of large scale compute clusters to apply machine learning algorithms on Petabytes of data using Apache SparkML Pipelines.
- eliminate out-of-memory errors generated by traditional machine learning frameworks when data doesn’t fit in a computer’s main memory
- test thousands of different ML models in parallel to find the best performing one – a technique used by many successful Kagglers
- (Optional) run SQL statements on very large data sets using Apache SparkSQL and the Apache Spark DataFrame API.
Enrol now to learn the machine learning techniques for working with Big Data that have been successfully applied by companies like Alibaba, Apple, Amazon, Baidu, eBay, IBM, NASA, Samsung, SAP, TripAdvisor, Yahoo!, Zalando and many others.
NOTE: You will practice running machine learning tasks hands-on on an Apache Spark cluster provided by IBM at no charge during the course which you can continue to use afterwards.
Prerequisites:
- basic python programming
- basic machine learning (optional introduction videos are provided in this course as well)
- basic SQL skills for optional content
The following courses are recommended before taking this class (unless you already have the skills)
https://www.coursera.org/learn/python-for-applied-data-science
or similar
https://www.coursera.org/learn/machine-learning-with-python
or similar
https://www.coursera.org/learn/sql-data-science
for optional lectures
None
Syllabus
Syllabus - What you will learn from this course
Week 1
Week 1: Introduction
Week 2
Week 2: Scaling Math for Statistics on Apache Spark
Week 3
Week 3: Introduction to Apache SparkML
Week 4
Week 4: Supervised and Unsupervised learning with SparkML
FAQ
When will I have access to the lectures and assignments?
Access to lectures and assignments depends on your type of enrollment. If you take a course in audit mode, you will be able to see most course materials for free. To access graded assignments and to earn a Certificate, you will need to purchase the Certificate experience, during or after your audit. If you don't see the audit option:
What will I get if I subscribe to this Certificate?
When you enroll in the course, you get access to all of the courses in the Certificate, and you earn a certificate when you complete the work. Your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile. If you only want to read and view the course content, you can audit the course for free.
Reviews
Course was nice and avoided peer-graded assignments (which I appreciate) but code was written in Python 2 which led to un-maintained code.
There are some issues with the normalization of the distribution moments. Everything else is good material to learn how to use apache-spark for the first time.
Really really REALLY enjoyed this course! The instructor does a masterful job of going from simple examples and building up complexity in a very logical and thorough way.
I found difficult to understand the concepts, for sure I must have to review the class.
Thanks for the dedication in helping us.
Itamar
