Tips On Web Scraping to Find Amazon’s Bestselling Products

There’s no doubt that in order to make a decent profit on Amazon, it is essential to choose the best product to sell. To find out which product sells the best, we need to conduct product research to understand the market.

In this article, I will discuss 5 tips to find the best selling product on Amazon using web scraping.

Golden Rule for Best-Sellers on Amazon

We all want to choose a product that will become a huge success after we put it on the shelf. But what aspects should we take into consideration? The golden rule for selling things online is simple: more demand, less competition.

Here are ways we can narrow down to what we want:

1. A product that has a few big brands in the fielda product that has a huge demand and can be sold year-round

2. A product that still has room for improvementa product that has a decent profit margin but not too expensive (ideal price range: $25-$50)

3. A product that is lightweight and small (for easy packaging and shipping)…

Luckily, with the vast amounts of data available online, we can do some analysis to get a general idea of the market. Let’s take a look at how we can use data to fuel our knowledge base for selecting the best items.

Scrape Amazon Product Data

For example, I want to sell shavers on Amazon. To conduct well-rounded market research, I analyze it. The data I scraped includes the ASINs, prices, seller names, brands, a number of reviews, review texts, shipping weights, and so on.

Since there were at least thousands of shavers being listed on Amazon, it was not feasible to collect the data manually. To make my life easier, I used an automated web scraping tool called Octoparse to scrape data in bulk from the web pages. It helped me extract valuable information about the first 232 shavers that appeared on the search results page.

How Many Big Brands are there in the Market?

The first and most important aspect to consider is always the competitors, especially the big ones, as people all tend to trust big brands over a new/small brand that they’ve never heard of.

Among the 232 products, there are 88 brands in total. Philips Norelco (16.5%), Remington (10%), and Braun (10%) are the top 3 players with the most items, followed by Panasonic (3.9%), Wahl (3.9%) and Rozlapro (3.5%). As we can see from the pie chart below, the shaver market isn’t solely occupied by the big brands as there are dozens of small brands taking their share. This indicates that there is still some room left for newcomers to compete in this race.

Which Products are in High Demand on Amazon

?

The second important thing we want to make sure is that there are a number of people wanting this item, and are willing to pay for it.

A shortcut to estimate overall demands for a product is using keyword research tools like Google Trends, since when people are interested in something, they search for it first. There are many other keyword tools available as well, and we can take advantage of a few of them to collect as much information as possible. By looking at the trends over a period of time, we can determine whether this is a seasonal product and when is a good time to sell it.

The keyword “shaver” has a relatively smooth curve that doesn’t seem to be influenced by seasons. In other cases, for example, let’s say we are selling fashionable clothes. It would be critical for us to do some googling and check the fashion trends before selling any clothing.

The downside of this method is that it only gives a vague idea of the market demand, and it doesn’t equal to the number of potential buyers on Amazon. A more accurate way to deduce the actual demand is by looking at the numbers of reviews of each product listing.

Usually, a brand with more than 10,000 reviews means that there is a huge market for their item(s). After collecting the number of reviews of each item, we can see the top 4 brands all surpassed 20,000 reviews. And the number of reviews for each brand averages around 3,000. From there, we can conclude that there is a huge demand for shavers. However, it would be not wise to compete directly with the top players as they possess a bigger part of the market pie.

Scrape Customer Ratings

So we just took a look at the number of reviews, now we can dig into the ratings. The product ratings tell us in a straightforward way how consumers are perceiving these brands and the products.

If a single brand has tons of reviews with an extremely high average rating, it means that it is a monopoly brand. Obviously, everybody loves it and thinks this is an awesome choice. On the contrary, if the ratings of all products in a market are very low, it means that all the brands share unavoidable defects in common. What I just described are two extreme situations, and it would be unwise to enter either market.

Now let’s go back to our case and compare the average product ratings of all brands. From left to right, the average ratings go down and the brands with the most reviews concentrate in the middle. As shown in the chart below, the shavers with the most reviews are rated between 4.0–4.4, which indicates that there is no single brand monopolizing the market and this is not a bad battlefield to compete in.

Amazon Reviews as part of Product Research

Besides scraping the number of reviews, we can also pull the review text off Amazon directly. To choose the best product that beats the others, knowing what the customers are thinking about the existing products is the key.

By scraping the Amazon reviews of the products and analyzing the word frequency, we can easily pinpoint what our customers need and know what they care the most about the products.

Above is the Wordcloud based on the shaver reviews on Amazon. In this case, we can make a conclusion that customers usually prefer a high-quality shaver with a long battery life that gives a clean, close and smooth shave. And this is just scratching the surface of review analysis. With all the review data in hand, one can analyze the sentiments hidden behind the words and uncover public opinions towards the brands/products.

Ideal Price Points for your Product/s

We have already done our research on the major competitors and the market demand. Everything looks great and we are ready to cut into the market with our product. Yet another big question emerges: how should we price the product?

A product, when priced too high compared to those of the competitors, will be ignored by consumers and run into a dead end. But if it is priced too low, there won’t be a profitable margin to generate enough revenue to cover all the costs.

This is where product pricing data come into play. As the web scraper Octoparse has pulled the prices of all products, to analyze them in Excel is easy-peasy. After careful calculation, the average price for these shavers is $53.92, with the most expensive one selling at $309.99. Almost half of the shavers fall in the price range of $25-$50, while a quarter of them are cheaper than 25 bucks.

Note that Amazon sellers are constantly changing their prices, so monitoring competitors’ prices is essential in the fierce competition. The good news is that with Octoparse it can schedule regular data extractions from Amazon into Excel, which enables us to make up-to-date pricing strategies.

Bigger Brands vs. Small Brands on the Amazon Marketplace

Knowing market prices makes us one step closer to giving birth to a best-seller, but it is not enough. One needs to find out the relationship between the prices and their brands so as to determine the best market position.

With all the data we have, it is not hard to do so. I compared the average selling prices among brands. Data shows that in general, the top players have relatively higher prices compared to those of the rest. Considering that big brands appeal to customers naturally, in this case, it may be a good strategy to start at a lower-than-average price.

Calculate the Weight of your Amazon Products

Brands, demand and price are the 3 big aspects to consider. But to become a smart seller on Amazon, we still need to pay attention to details that will influence the sales and profits. One small aspect often neglected by sellers is the size and weight of the product.

How does shipping weight influence a product’s performance, or online business? High shipping weight leads to high shipping costs and FBA fees, which can cut in our profit margin. Be careful that going too small can also be a disadvantage since small items can easily be crushed by big packages falling on to them.

I unified the units of the shipping weights of all shavers to analyze them. As shavers are small and light-weighted (less than 1 pound on average), they have low shipping charges and take up less space. This kind of product tends to generate more profits and is easier to manage. But when picking products, avoid the ones with fragile inner parts, like glass or delicate electronics. It is better to choose something sturdy, durable and easy to hold.

Wrapping Up

The tips above guide us in choosing which product to sell on Amazon. With a web scraping tool, the research process is no longer tedious and time-consuming, as it automates the data scraping process and makes data collection easier for us. If we have a product in mind already, you can follow the scraping and analysis steps I covered in this post to see whether it is the best choice. The research should provide us with all the information we need to succeed.

Originally published at http://www.dataextraction.io on June 10, 2020.