Hey, fellow devs 👋 In this tutorial, I will be showing you guys how to scrape the latest Tesla stock prices using Node.js and puppeteer. Let’s get started! First of all, you will need to install the puppeteer using . Now if you don’t have , , and , a great tutorial. npm i puppeteer npm package.json node_modules setup here’s After you’ve installed puppeteer, create a new javascript file and require puppeteer on the first line: puppeteer = ( ); const require 'puppeteer' Then create the async function in which we are going to write our main code: puppeteer = ( ); {
}
start(); const require 'puppeteer' async ( ) function start Now we’re ready to start scraping. First of all, you need to initiate a new browser instance, as well as define the url which your web-scraper is going to be visiting: puppeteer = ( ); { url = ; browser = puppeteer.launch({ : });
} const require 'puppeteer' async ( ) function start const 'https://finance.yahoo.com/quote/TSLA?p=TSLA&.tsrc=fin-srch' const await headless false Next, you need to call the function to open a new page in the browser, and go to the url that we defined using the function: newPage() goto() puppeteer = ( ); { url = ; browser = puppeteer.launch({ : }); page = browser.newPage(); page.goto(url);
} const require 'puppeteer' async ( ) function start const 'https://finance.yahoo.com/quote/TSLA?p=TSLA&.tsrc=fin-srch' const await headless false const await await For this next step, you will have to go to , right-click on the current stock price, and click on inspect: https://finance.yahoo.com/quote/TSLA?p=TSLA&.tsrc=fin-srch A pop-up will appear on the right of your window, you will need to find the stock price element: Next, you will need to right-click on the stock price element and click on “copy full Xpath”. This will give us a way of accessing the stock price element: Once we have the Xpath of the stock price element, we can add these 3 lines of code into our function: element = page.waitForXPath( ) price = page.evaluate( element.textContent, element); .log(price); var await "put the stock price Xpath here" var await => element console The function will locate the stock price element. Next, the function will get the text contents of the stock price element which would then be printed by the function. page.waitForXPath() page.evaluate console.log() At this point, your code should look something like this: puppeteer = ( ); { url = ; browser = puppeteer.launch({ : }); page = browser.newPage(); page.goto(url); element = page.waitForXPath( ) price = page.evaluate( element.textContent, element); .log(price);
}
start() const require 'puppeteer' async ( ) function start const 'https://finance.yahoo.com/quote/TSLA?p=TSLA&.tsrc=fin-srch' const await headless false const await await var await "/html/body/div[1]/div/div/div[1]/div/div[2]/div/div/div[5]/div/div/div/div[3]/div[1]/div[1]/span[1]" var await => element console If you were to execute your current code, you will find that when going to the url that you defined earlier, a pop-up will appear: To get around this, plug these 2 lines of code into your function before defining the “element” variable: accept = ( ); page.click(accept) var "#consent-page > div > div > div > form > div.wizard-body > div.actions.couple > button" await This will locate the “Accept All” button and click it to make the popup go away. Now you will have a working function that goes to your defined url, scrapes the latest Tesla stock price, and prints it in your terminal. To go one step further, you can put these lines of code in a for loop: ( k = ; k < ; k++){ element = page.waitForXPath( ) price = page.evaluate( element.textContent, element); .log(price); page.waitForTimeout( );
    } for var 1 2000 var await "/html/body/div[1]/div/div/div[1]/div/div[2]/div/div/div[5]/div/div/div/div[3]/div[1]/div[1]/span[1]" var await => element console await 1000 The function will wait 1000 milliseconds(1 second) before repeating the for a loop. page.waitForTimeout(1000) And finally, function after the for loop to close the browser and finish your code execution: add a browser.close() puppeteer = ( ); { url = ; browser = puppeteer.launch({ : }); page = browser.newPage(); page.goto(url); accept = ( ); page.click(accept); ( k = ; k < ; k++){ element = page.waitForXPath( ); price = page.evaluate( element.textContent, element); .log(price); page.waitForTimeout( );
    }
    browser.close();
}
start(); const require 'puppeteer' async ( ) function start const 'https://finance.yahoo.com/quote/TSLA?p=TSLA&.tsrc=fin-srch' const await headless false const await await var "#consent-page > div > div > div > form > div.wizard-body > div.actions.couple > button" await for var 1 2000 var await "/html/body/div[1]/div/div/div[1]/div/div[2]/div/div/div[5]/div/div/div/div[3]/div[1]/div[1]/span[1]" var await => element console await 1000 Get the hottest programming: tweets, Reddit posts, Github Repos, and memes of the week directly in your inbox every Friday via my newsletter ! Byeeeeeee 👋 Als published . here

Tesla

Web Scraping Using Node.js

Get the top programming stuff of the week in your inbox every Friday via my newsletter!

Scraping Tesla Stock Prices with Node.js and Puppeteer

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Declutter Your Code!

The Noonification: Reduce Javascript: Master the Basics (1/11/2023)

The Noonification: Satoshi Might Be a Bitcoin ETF Skeptic (1/13/2024)

The Noonification: Small Puddle of Freedom (11/25/2022)

The Noonification: How to Use AI for Your B2B Marketing (11/11/2022)

105 Stories To Learn About Functional Programming

Declutter Your Code!

The Noonification: Reduce Javascript: Master the Basics (1/11/2023)

The Noonification: Satoshi Might Be a Bitcoin ETF Skeptic (1/13/2024)

The Noonification: Small Puddle of Freedom (11/25/2022)

The Noonification: How to Use AI for Your B2B Marketing (11/11/2022)

105 Stories To Learn About Functional Programming

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps