Web Scraping with Python

• Programming experience, preferably in Python. If participants have programming experience in a language other than Python, the training can be extended to include more introductory Python exercises.

Audience

• Developers

Web Scraping is a technique for extracting data from a website then saving it to local file or database.

This instructor-led, live training (online or onsite) is aimed at developers who wish to use Python to automate the process of crawling many websites to extract data for processing and analysis.

By the end of this training, participants will be able to:

• Install and configure Python and all relevant packages.
• Retrieve and parse data stored across many websites.
• Understand how websites work and how their HTML is structured.
• Construct spiders to crawl the web at scale.
• Use Selenium to crawl AJAX-driven web pages.

Format of the Course

• Interactive lecture and discussion.
• Lots of exercises and practice.
• Hands-on implementation in a live-lab environment.

Course Customization Options

• This course assumes knowledge of programming.
• To request a customized training for this course, please contact us to arrange.

Introduction

Setting up the Development Environment

Python Primer: Data Structures, Conditionals, File Handling, etc.

Python Packages for Web Scraping: Scrapy and BeautifulSoup

How a Website Works

How HTML is Structured

Making a Web Request

Scraping an HTML Page

Working with XPath and CSS

Filtering Data Using Regular Expressions

Creating a Web Crawler

Crawling AJAX and JavaScript Pages with Selenium.

Web Scraping Best Practices

Troubleshooting

Summary and Conclusion