Adrian Horning – The Ultimate Web Scraping Course 2024-8

Published on: 2024-09-12 14:37:22

Categories: 28

Descriptions

The Ultimate Web Scraping Course, I’ve made a living for 3+ years from just web scraping, made over $125k, have 25k+ social media followers, and in this course I’m going to reveal exactly how I scrape sites, and how to get clients/grow on social media. I don’t go over how to scrape behind logins. Everything I scrape will be in front of the login. The code is in JavaScript and I am using a Mac with VS Code editor. I only briefly talk about Puppeteer (there are MUCH better ways). Scraping isn’t magic you do have to have a source to scrape (a website). Like you can’t scrape people’s bank accounts or something. That would be more hacking, which is a different field.

What you’ll learn

How websites get their data (how API’s work)
My process for scraping ANY site
How to find where data is coming from
How to avoid getting blocked
All about proxies
Scraping by web framework (React, Next.js, static HTML/server rendered, ASP.NET, Java)
How to reverse engineer sites with good blocking
Scraping sites with Cursor Based Pagination (YouTube comments)
Scraping a chrome extension for free emails
How to scrape mobile apps
How to skip trace (find random people’s phone numbers and emails)
How to get Puppeteer running on AWS Lambda
How to intercept requests with Puppeteer
Solving captchas with Puppeteer

Specificatoin of The Ultimate Web Scraping Course

Publisher : Adrian Horning
Teacher : Adrian Horning
Language : English
Level : All Levels
Number of Course : 48
Duration : 10 hours and 40 minutes

Content of The Ultimate Web Scraping Course

Is Web Scraping Legal?
Hackers vs Scrapers
My go to packages, editor, language, proxies, etc
How websites get their data/how API’s work
My Process for Scraping Any Site
How to find where data is coming from
How to set up proxies
How to make money web scraping
Scraping Next.js sites (angi, imdb)
Zillow – Scraping Hidden API’s
ESPN – Scraping API’s
Amazon – Straight up HTML page
Indeed – Straight Up HTML
Sarpy – ASP.NET (long version)
Sarpy – ASP.NET Cliff Notes
Kitsap – Java Site
Google Search
Google Maps
Linkedin
Shopify
They can’t hide anything from you…..
Reverse Engineering Realtor API
Lynchburg exposing Auth Token
Beacons.ai
Twitter
Instagram
Arcgis
Cursor Based Pagination (YouTube Comments)
How to scrape Chrome Extensions (free emails)
Reverse Engineering a Private API with mitmproxy
Scraping Mobile Apps
Skip Tracing
Getting Puppeteer Up and Running on AWS Lambda
Getting Tiktok Videos with Puppeteer
Solving CAPTCHAs with Puppeteer
Sitemaps (crunchbase)
Google Tricks
Decoding Imperva Security Measures
Leveraging Google’s Web Cache to get around Cloudflare and Datadome
Tiktok API
SC Courts, use undetected chrome driver
Exploring Supabase for Efficient Database Management
Where to host the scrapers? Render.com
Quick note on Concurrency in Web Scraping
How to get web scraping clients
What do you charge clients??
What should you scrape for $$?
I found a way to scrape Ohio State Student Emails

Requirements

You should have a good understanding of JavaScript, or some programming language, to get the most out of this course.
If you don’t know how to code at all, this is not the course for you.
I don’t teach the basics of coding or JavaScript in this course.