Get Started with Natural Language Processing Using Python, Spark, and Scala

Table of Contents

Introduction

Course Introduction

About The Author

Getting Started: Basic String Processing In Python

String Operations

Working With Unicode

Converting Text To Symbols: Tokenization In NLTK and spaCy

Splitting Documents

Splitting Sentences

Filtering Stop Words

Going Subsymbolic: Vector Representations

tf-idf Gensim

Word Vectors

Google Word Vectors

Learn Word Vectors

Finding The Structure Of Text: Parsing In spaCy

Dependency Parsing

Sentence Head

Named Entities

Determining How The Writer Feels: Sentiment Analysis In VADER

Sentiment Analysis Intro

Sentiment In VADER

Making Decisions: Text Classification

Text Classification Intro

Classification With TextBlob

Classification With scikit-learn

Indentifying Discussed Topics: LDA In Gensim

LDA Introduction

LDA Gensim

LDA pyLDAvis

Toward Machine Reading: Entity Extraction And Linking

Entity Linking

pyspotlight

FRED

Conclusion

Conclusion

Part 1: Introduction

Welcome to the Course

Natural Language Understanding in Examples

Part 2: NLP Pipelines

Building an NLP Pipeline

Part 3 - Annotators

Commonly Used Annotators

Detecting Positive, Negative Speculative Polarity

Machine Learned Annotators

Part 4: Custom Annotators

NLP Pipelines are Domain Specific

Unified Medical Language System (UMLS)

Coding Custom Annotators

Part 5: Machine Learned Annotators

Training Using Machine Learned Annotators

Part 6: Ontology Enrichment

The Need for Learned and Updated Ontologies

Learning New Medical Concepts and Relationships

Part 7: Architecture

An End-to-End Reference Architecture

Spark, SparkSQL, Cassandra Workflow

ElasticSearch SparkSQL

Part 8: Parting Advice

Language is Source and Domain-Specific

Welcome to the Course

Part 1: Building a natural language processing and entity extraction pipeline on Scala Spark

Notebook 1: Introduction

Annotation Library

Basic Annotators

Vocabulary Analysis

Exercise: Building a stopword annotator

Part 2: Machine Learning Applications for Statistical Natural Language Understanding at Scale

Notebook 2: Introduction

Model-based Annotators

Creating a Binary Classifier

Exercise: Predicting score or popularity

Part 3: Topic Modeling on Natural Language with Scala, Spark and MLLib

Notebook 3: Introduction

K-Means clustering

LDA topic modeling

Exercise: Using topics for score or popularity prediction

Part 4: Deep Learning Applications for Natural Language Understanding with Scala, Spark and MLLib

Notebook 4: Introduction

Word2Vec

Expanding genre entity lists

Exercise: Using Word2Vec based features for score or popularity prediction