In this tutorial, we are going to create our own e-commerce search API with support for both eBay and Etsy without using any external APIs. With the power of and , we are able to achieve this goal in fewer than 20 lines of Python code for each site. I recommend reading my last about AutoScraper if you haven’t done so yet. AutoScraper Flask article Requirements Install the required libraries using pip: pip install -U autoscraper flask Let’s Do It First, we are going to create a smart scraper to fetch data from eBay’s search results page. Let’s say we want to get the title, price, and product link of each item. Using AutoScraper, it would be easily done by just providing some sample data: Note that if you want to copy and run this code, you may need to update the . wanted_list Now let’s get the results grouped by scraping rules: scraper.get_result_similar(url, grouped= ) True From the output, we’ll know which rule corresponds to which data, so we can use it accordingly. Let’s set some aliases based on the output, remove redundant rules, and save the model so we can use it later: Note that the rule IDs will be different for you if you run the code. OK, we’ve got eBay covered. Let’s add support for Etsy search results too. We’ll start by building its scraper. This time, we will use instead of . It will automatically set aliases for us: wanted_dict wanted_list As the links are generated with a unique ID on Etsy each time, we added one sample product ID to the so we can create the link from it. Also, we provided two samples for title and price, as the structure of items on Etsy search result pages is different and we want the scraper to learn them all. wanted_dict After analyzing the output, let’s keep our desired rules, remove the rest, and save our model: Now that we have our scrapers ready, we can create our fully functioning API for both sites in fewer than 40 lines: Here, we are defining an API with the parameter as our search query. We will get and join eBay and Etsy search results and return them as a response. Note that we are to the scraper to get the results grouped by our defined aliases. q passing group_by_alias=True By running this code, the API server will be up listening on port 8080. So let’s test our API by opening in our browser: http://localhost:8080/?q=headphone Some results from eBay Some results from Etsy Voila! We have our e-commerce API ready. Just replace in the URL with your desired query to get its search results. headphone Final Notes The final code for this tutorial is available on . GitHub This is a development setup suitable for developing and testing. Flask’s built-in server is not suitable for production. For production usage, please check . Flask’s deployment options This tutorial is intended for personal and educational use. If you want to scrape websites, you can check their policies regarding the scraping bots. I hope this article is useful and helps to bring your ideas into code faster than ever. Happy coding!
