\
In this post, we will learn to scrape Google Shopping Results using Node JS.

\
## Requirements:

### Web Parsing with CSS selectors

Searching for the tags from HTML files is not only a difficult thing to do but also a time-consuming process. It is better to use the [CSS Selectors Gadget](https://chrome.google.com/webstore/detail/selectorgadget/mhjhnkcfbdhnjickkkdbjoemdmbfginb) for selecting the perfect tags to make your web scraping journey easier.

\
This gadget can help you come up with the perfect CSS selector for your need. Here is the link to the [tutorial](https://selectorgadget.com/), which will teach you to use this gadget for selecting the best CSS selectors according to your needs.

### User Agents

User-Agent is used to identify the application, operating system, vendor, and version of the requesting user agent, which can save help in making a fake visit to Google by acting as a real user.

\
You can also rotate User Agents, read more about this in this article: [How to fake and rotate User Agents using Python 3](https://www.scrapehero.com/how-to-fake-and-rotate-user-agents-using-python-3/).

\
### Install Libraries

To scrape Google maps reviews, we need to install some NPM libraries so we can move forward.

\

1. [Unirest JS](https://www.npmjs.com/package/unirest)
2. [Cheerio JS](https://www.npmjs.com/package/cheerio)

   \

So before starting, we have to ensure that we have set up our Node JS project and installed both the packages - Unirest JS and Cheerio JS. You can install both packages from the above link.

## Target:

 ![Scrape Google Shopping Results 2](https://cdn.hackernoon.com/images/JQw8tOXTHed5LWWisLwgsiVMAEb2-2022-12-09T14:35:59.131Z-clbgm22vw00070as6dk543zu2 "Scrape Google Shopping Results 2")

We will target to scrape the shopping results of Nike shoes.

## Process:

\
We have installed all the things which we will need for our scraper. Now we will hit our target URL using Unirest JS to get our HTML data, and then we will parse our extracted HTML data with the help of Cheerio JS.

\
We will target this URL:

\
```
https://www.google.com/search?q=nike shoes&tbm=shop&gl=us
```

\
Look at the `tbm` parameter and its value(`shop`, here). This value `shop` will tell Google that we are looking for shopping results.

\
Open this URL in your browser. Inspect the code. You will see that every organic shopping result is inside this tag `.sh-dgr__gr-auto`.

\
 ![Scrape Google Shopping Results 3](https://cdn.hackernoon.com/images/JQw8tOXTHed5LWWisLwgsiVMAEb2-2022-12-09T14:35:59.136Z-clbgm22w000080as6fadh0nbq "Scrape Google Shopping Results 3")

\
Now, we will search the tags for title, product link, price, rating, reviews, delivery, and source.

\
 ![Scrape Google Shopping Results 4](https://cdn.hackernoon.com/images/6sWrtbrOmsOIbrWzrG88lYfV4ch1-7d93shj.png)      ![Scrape Google Shopping Results 5](https://cdn.hackernoon.com/images/6sWrtbrOmsOIbrWzrG88lYfV4ch1-gta3swd.png)The above images are in the pattern of two at the top and one at the bottom.

We have completed our search for tags of organic shopping results. Now, we will search for the tags of ad results.

 ![Scrape Google Shopping Results 6](https://cdn.hackernoon.com/images/JQw8tOXTHed5LWWisLwgsiVMAEb2-2022-12-09T14:35:59.148Z-clbgm22wc000d0as6gkuf3mm4 "Scrape Google Shopping Results 6")

If you inspect the ad results, you will see that all the ad results are inside the tag `.sh-np__click-target`. This tag contains all the information about the title, link, price, and source.

All the above things make our code look like this:

\
```javascript
 const unirest = require("unirest");
    const cheerio = require("cheerio");
    
    const getShoppingData = () => {
      
      try
      {
      return unirest
        .get("https://www.google.com/search?q=nike shoes&tbm=shop&gl=us")
        .headers({
          "User-Agent":
            "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.54 Safari/537.36",
        })
        .then((response) => {
          let $ = cheerio.load(response.body);
    
    
        let ads = [];
        
        $(".sh-np__click-target").each((i,el) => {
         ads.push({
            title: $(el).find(".sh-np__product-title").text(),
            link: "https://google.com" + $(el).attr("href"),
            source: $(el).find(".sh-np__seller-container").text(),
            price: $(el).find(".hn9kf").text(),
            delivery: $(el).find(".U6puSd").text(),
         })
         if($(el).find(".rz2LD").length)
         {
          let extensions = []
            extensions = $(el).find(".rz2LD").text()
            ads[i].extensions = extensions
         }
         })
    
        for (let i = 0; i < ads.length; i++) {
            Object.keys(ads[i]).forEach(key => ads[i][key] === "" ? delete ads[i][key] : {});  
        }
    
        let shopping_results = [];
    
        $(".sh-dgr__gr-auto").each((i,el) => {
            shopping_results.push({
                title: $(el).find(".Xjkr3b").text(),
                link: $(el).find(".zLPF4b .eaGTj a.shntl").attr("href").substring($(el).find("a.shntl").attr("href").indexOf("=")+1),
                source: $(el).find(".IuHnof").text(),
                price: $(el).find(".XrAfOe .a8Pemb").text(),
                rating: $(el).find(".Rsc7Yb").text(),
                reviews: $(el).find(".NzUzee div").attr("aria-label") ? $(el).find(".NzUzee div").attr("aria-label").substring(0,$(el).find(".NzUzee div").attr("aria-label").indexOf(" ")) : "",
                delivery: $(el).find(".vEjMR").text()
            })
            if($(el).find(".Ib8pOd").length)
            {
                let extensions = [];
                extensions = $(el).find(".Ib8pOd").text();
                shopping_results[i].extensions = extensions
            }
        })
    
        for (let i = 0; i < shopping_results.length; i++) {
            Object.keys(shopping_results[i]).forEach(key => shopping_results[i][key] === "" ? delete shopping_results[i][key] : {});  
         }
         
         console.log(ads)
         console.log(shopping_results)
        })
      }
     catch(e)
     {
        console.log(e)
     }
    }
    
    
    getShoppingData();
```

\
You can also check some of my other Google scrapers in my Git Repository: <https://github.com/Darshan972/GoogleScrapingBlogs>

\
## Result:

 ![Scrape Google Shopping Results 7](https://cdn.hackernoon.com/images/JQw8tOXTHed5LWWisLwgsiVMAEb2-2022-12-09T14:35:59.147Z-clbgm22wb000c0as65ohfhpas "Scrape Google Shopping Results 7")

Our result should look like this 👆🏻.

## With Google Shopping API

\
If you don't want to code and maintain the scraper in the long run and don't want to work with complex URLs and HTML, then you can try this [Google Search API](https://serpdog.io/).

 \n Serpdog | Google Search API solves all the problem of captchas and proxies and allow developers to scrape Google Search Results smoothly. Also, the pre-cooked structured JSON data can save you a lot of time.

\
```javascript
 const axios = require('axios');
    
    axios.get('https://api.serpdog.io/shopping?api_key=APIKEY&q=shoes&gl=us')
      .then(response => {
        console.log(response.data);
      })
      .catch(error => {
        console.log(error);
    });
          
```

## Result:

 ![Scrape Google Shopping Results 8](https://cdn.hackernoon.com/images/JQw8tOXTHed5LWWisLwgsiVMAEb2-2022-12-09T14:35:59.146Z-clbgm22wa000b0as6cr2b06jf "Scrape Google Shopping Results 8")

## Conclusion:

In this tutorial, we learned to scrape Google Shopping Results using Node JS. Feel free to [message me](https://drift.me/darshankhandelwal12) anything you need clarification on. Follow me on [Twitter](https://twitter.com/serpdogAPI). Thanks for reading!

\
*Also published [here.](https://serpdog.io/blog/scrape-google-shopping-results)*

\
## Additional Resources


1. [Scrape Google Images Results](https://hackernoon.com/scrape-google-images-with-node-js)
2. [Scrape Google Maps Reviews](https://serpdog.io/blog/scrape-google-maps-reviews)

## Frequently Asked Questions

### **Q. How do I get Google Shopping results?**

You can get Google Shopping Results by using Serpdog [Google Shopping API](https://docs.serpdog.io/google-shopping-api) without any problem with proxies and CAPTCHAs. This data is one of the great sources for data miners for competitor price tracking, sentimental analysis, etc.

\
## Author:

My name is Darshan, and I am the founder of Serpdog. I love to create scrapers. I am currently working for several MNCs to provide them with Google Search Data through a seamless data pipeline.

Google

Mozilla

Nike

Essential Guide to Scraping Google Shopping Results

Too Long; Didn't Read

Companies Mentioned

@darshan12

RELATED STORIES