

Web scraping should be simple!
Learn how to automate searches using Puppeteer + Apify
One of our clients at ZScrape needed a way to get instant alerts for new items matching his criteria, specifically Pokémon tradable cards and collectables for his personal collection and online store.
The built-in Craigslist alert feature is limited and you cannot configure the frequency for every hour or even every minute! It can also be unreliable and miss new results. Our client did not want to miss another opportunity because he was not quick enough to find new posts. He also wanted to be alerted when existing posts are updated such as price changes.
To build this scraper we will use Node.js, Puppeteer, and a cloud hosting solution to host the script. ZScrape provides hosting and a hassle-free no setup solution — learn more! In this tutorial we will use one of our favorite services: Apify. They offer a FREE tier to get started! Sign up below.
First we need to extract the data from the search page — this includes Image, Title, Description, Price, City, Link URL, and Date posted. To do this will use Chrome Dev tools to examine the XPaths. There is an excellent tutorial here by Riley Predum.
span.result-price, a.result-title, span.result-hood, span.nearby, a.result-image
Putting it together into a javascript function:
Now we need to save the results so the next time the script is run we can compare if there are new posts or any new changes. Let’s use Apify’s built in storage capability. Create a new data store and save results from the current run.
Since our data is in a dictionary, we can easily compare the previous and new results and only capture the new or updated items. Results are sent via Email or SMS using Apify.call(‘apify/send-mail’)
This function is used to format the results. You can customize as you wish.
Finally we need to add in the search query. We can utilize Apify’s Apify.getInput() call.
Putting it all together we have the final script:
Login to Apify and navigate to the Actors tab and Create new.
Click the Source tab, choose Node.js 12 on Alpine Linux (apify/actor-node-basic) and paste the Source code.
Scroll down to Input and enter the input email and search criteria of the search. For SMS, use phone#@carrier.
Example:
{
"email": "XXX@XXX.com",
"email_subject": "Alert! New updates matching your search",
"search": "pokemon",
"zip": "94023",
"distance": "10",
"min_price": "25",
"max_price": "200"
}
Then click Run! Your script will build and run for the first time.
Navigate to the Schedules tab and Create new. Under settings, set the Cron expression to a schedule such as ‘daily’ or ‘hourly’.
Note: keep in mind more frequent runs means more compute units used
Click the Actor sub tab next to Settings and Add actor. Select the actor you created from the Actor dropdown menu. Set Body to the same input from above. Click Save.
Another satisfied client 😃 If you need help setting this up or require a custom solution to fit your project, use our live chat at ZScrape and we will be happy to help!
Web scraping should be simple!
Note: The information contained in this article is for educational and informational purposes only.
Also published at https://medium.com/@zscrapesolutions/never-miss-another-deal-on-craigslist-setup-custom-email-sms-alerts-4f9c459e4c8c
Create your free account to unlock your custom reading experience.