Web Scraping with Javascript and Node.jsby@anderrv
477 reads
477 reads

Web Scraping with Javascript and Node.js

by Ander16mOctober 1st, 2021
Read on Terminal Reader
Read this story w/o Javascript
tldt arrow

Too Long; Didn't Read

Using Node v12, we will build a simple scraper and crawler from scratch using Javascript. We'll use scrapeme.live as an example, a fake website prepared for scraping. We use Axios to get the HTML. Then we will pass the HTML to cheerio and query it as we would in a browser environment. We'll query for the two things we want right now: paginator links and products. For cases when we want to run JS, Playwright will do. Once everything is working fine, we will scale it by launching crawls async.

Companies Mentioned

Mention Thumbnail
Mention Thumbnail
featured image - Web Scraping with Javascript and Node.js
Ander HackerNoon profile picture
Ander

Ander

@anderrv

Web developer who has been working for startups for +10 years. Engineer turned entrepreneur.

About @anderrv
LEARN MORE ABOUT @ANDERRV'S
EXPERTISE AND PLACE ON THE INTERNET.

Share Your Thoughts

About Author

Ander HackerNoon profile picture
Ander@anderrv
Web developer who has been working for startups for +10 years. Engineer turned entrepreneur.

TOPICS

THIS ARTICLE WAS FEATURED IN...

Permanent on Arweave
Read on Terminal Reader
Read this story in a terminal
 Terminal
Read this story w/o Javascript
Read this story w/o Javascript
 Lite
Also published here
L O A D I N G
. . . comments & more!