Working in a B2B company, you have a whole specter of scraping tools to choose from. Anyone with either a bit coding skills or some extra cash to spend can find a given person’s business email address.

The formula here is very simple, you either install something like skrapp.io that can crawl LinkedIn to get the company name + the first and last name and from there you can literally guess the email address of everyone in the company. Sure, the pattern varies for every company but it’s a very limited number of choices. After you have let’s say 30 combinations of an email address, you can use an SMTP server that can check the validity and go from there. That’s it.

So my spoiled ass took this for granted before I started working for a B2C company. Being used to easily make a cold outreach campaign and start experimenting I promised my company that we are going to grow our revenue by 2x in the next month.

Boy was I wrong…

We had an Instagram agency that offered any kind of Instagram automation including tactics that were very popular back then like unlimited story viewing, the mother-slave tactic and so on.

Our primary personas were fitness instructors and photographers. So I jumped into LinkedIn and just to quickly find out that most of the target audience wasn’t very active on LinkedIn. Also, a ton of the people worked for a company that wasn’t so easily found because they were small.

Fuck.

I learned the hard way that our audience was almost exclusively active on Instagram (with some activity in Facebook groups). But what could I do here? How could I go from a targeted Instagram profile to an email address?

While looking at some targeted people and engaging in a chat I found that the button that Instagram added a while ago could give me the email address of a person if I click it. Cool. Some people had their email in their bio. Also cool. I started the manual process of visiting Instagram profiles one by one and writing the data down in a google sheet. I for all intent and purposes was the real-life Instagram Scraper we needed. After days of hard manual labor, I had about 300 profiles that I could run with.

I was praying that the campaign wouldn’t work so I didn’t have to do all that work again. On the other hand, if this did work I am sure the whole company would be impressed.

And the results were…. (drum roll) 🥁🥁🥁

Amazing. 25% clicked on my email!! Out of those about 30% actually signed up for a trial. This was huge!!

We had to scale our Instagram email scraper. There weren’t any Instagram Email scraper tools out there and this was clearly working!

With our Instagram scraper tool, we were already working with the unofficial Instagram API ie the mobile endpoints. That meant that to achieve any action, you had to simulate a device and log in to the Insta profile. From there, you just call the same endpoints as you would using a normal phone.

To get the emails scraped, we learned how to parse the returned JSON file and look up the email address. Sounds easy but it’s not. The problem was that our Instagram scrapers were getting blocked left and right. The data was amazing, but it just wasn’t sustainable.

I called my coding prodigy Victor Petrovski to see if we can figure something out together. As an android developer, he was a crucial part in getting this solved. We narrowed our problem down to two essential parts:

Problem number 1: We have to use Instagram accounts to do the crawling. The Instagram API has a very, very limited API calls limit. The crawling took ages.

Solution number 1: Create 10 Instagram account and try to split the work between them.

Problem number 2: There were emails in the comments, description of pictures, on the profile as a button, link to a website that has an email of the person there. We wanted to get that all of that.

Solution number 2: Create 1000 Instagram accounts and crawl the entire activity of a particular user. Hunter.io has a great API for finding emails from profiles that have a website link included so we used them too.

People were clicking and replying left and right. We had 5x the trials that we got from ANY other marketing efforts including a lot of money spent on Facebook Ads. But the day came when Instagram said, enough, no more automation. We couldn’t do anything other than use our Instagram Email scraper. No auto follows, comments, likes — NOTHING. I still remember the day when the entire office was dead silent. We knew there is nothing to be done and we might lose the company. And we just raised a seed round of 3M. But that was just the initial reaction. We were not defeated, we just had to pivot into something else.

What else could we use these emails for? Well, a lot of things. We were definitely not the first company to sell to consumers. We googled some great e-commerce stores and promised that we can make an email campaign that was going to have an ROI of above 600%. One said “Yes, let’s try it” and we got to work.

We quickly figured out how to find the ideal customer persona, started crawling a lot and then filtered the data using python. Our filters were done using the data points that we could get from Instagram + some external APIs that we found like determining the gender of a profile even though Instagram does not provide that.

From there we set up the campaign, wrote the copy, tested it and launched at scale. The results were unbelievable. We became experts in cold outreach reaching a scale of whooping 100 000 emails sent per day and growing our client to 2.5M in ARR! The whole tooling setup and how we did that are written in a separate blog post here.

We have no problem sharing what we did and how we did that for a specific reason. It’s very hard to replicate without spending a lot of time developing it and that costs a ton.

So here is how I would sum this up:

If you have a very early stage startup and have a developer in-house that can build this — go for it. You don’t need to reach a scale of scraping 5 million profiles per day like us. Heck, I even started collecting emails by hand so you can do it too. If you decide to build it here are my suggestions:

Use a python library (any really from git) I have found some good PHP repos too but I’d go with Python.

Find great sources for crawling — This should be followers of a given account, locations or hashtags. Pick the ones that are very specific to your business.Use aged Instagram profiles and have phone numbers that you can validate them with otherwise you’ll lose them almost immediately.

Do NOT use your personal profile.

Validate your emails — About 15% of your emails will be either spam traps, catch-all emails or invalid. If your email server ie Google sees that you just send random blasts to invalid emails, you’ll end up in spam and never get out of it.

If your business reached a scale of getting some revenue and you are looking for another growth channel that can scale fast, book a call with me and we can figure out whether or how we can help you with that.

Hopefully, you learned something today. Let me know if you have any questions about our this that I can help with, email me at [email protected]