Too Long; Didn't Read
Using Node v12, we will build a simple scraper and crawler from scratch using Javascript.
We'll use scrapeme.live as an example, a fake website prepared for scraping. We use Axios to get the HTML. Then we will pass the HTML to cheerio and query it as we would in a browser environment. We'll query for the two things we want right now: paginator links and products.
For cases when we want to run JS, Playwright will do. Once everything is working fine, we will scale it by launching crawls async.
Share Your Thoughts