This course is designed for those who want to learn how to automatically collect information from various sites, that is, scraping. Basic knowledge of Python is required to complete the course.

The lectures will cover the basic approaches to web scraping using libraries such as requests, beautifulsoup4 and Splash, as well as the Scrapy and Selenium frameworks.

This course covers the basic approaches to web scraping. At the end of the course, students will have a basic understanding of how to scrape various sites using Python.

This course will not be very interesting for those who already have a basic understanding of what web scraping is and how to create scripts for automated data collection.

After completing the course, students will know:

• How to find and install the required libraries for parsing data.

• What is ethical parsing, how not to violate the rights and the law when scraping data.

• What is the robots.txt file for?

• How to get the HTML of the site to extract the necessary information.

• How to use xpath to specify the exact location of elements on a page.

• How to navigate through pages on a multi-page site.

• How to create a project and run Scrapy spiders.

• How to scrape all information about each product on the site.

• How to navigate to the page of each product.

• How to scrape sites that use JavaScript on load.

• How to log in to the site using the Selenium framework.

• How to scrape sites with infinite scrolling by imitating user actions on the page.