How to Write a Python Script to Scale Your Data Science Job Applications

Background

Back in Spring 2020, when the Coronavirus just hit the US, I was enrolled in a Master's degree at Worcester Polytechnic Institute. With less than six months left to graduate, the urgency to get a job felt very real.

For those of you interested, I was looking to get a job in the Data Science field, as a Data Analyst or Data Scientist.

I had been applying to jobs for quite some time and was really struggling to get any interviews. I felt I did check most of the requirements on the job description. I had been doing projects and coursework for about two years until graduation, but I still could not get a job.

So I was convinced that there was something wrong in the way I was applying to jobs. Like any sane person, I would apply to job postings that I could find on LinkedIn or Indeed. On any given day I was applying to about ten companies. I would find the posting on the job board, locate the same posting on the careers page of the company and apply on the company’s career page.

Pretty straightforward right? If this task was that simple you wouldn’t be reading this. I recalled what Daniel Bourke said:

Job portals are dead. If I click that button, it's always a no.

I learned that if I apply through a job portal then it is almost certain that I would not get an interview.

Don’t get me wrong, people do get interviews this way too but I am convinced it is a highly unlikely gamble especially for a fresh graduate like me. Anyway, I think I should have another article out in the future detailing my struggles finding a job fresh out of college.

My Epiphany

My next move was to come up with a different approach to get a job. I came across this YouTube video one day where someone talked about how he would send out emails to recruiters making a pitch for himself for a job that was posted by that company recently.

That is how he got his first job in Data Science. I thought to myself “great, I’d never heard of something like this before!”. I mean being able to circumvent the process of filling up a form and waiting for it to be picked by the robot that was “parsing” my resume felt a lot like waiting for an H1B application to be picked in the lottery (my fellow Indian and Chinese students in the US would get this analogy) but with even worse odds.

That is when I decided to try this approach myself. I would have loved to share that Youtube video but I could never find that video again — funny how YouTube recommendations and searches work.

Doing the Deed

Alright, I was excited to try this new approach. Except there were several challenges.

1. How do I know who is the recruiter at the company I am interested in working for?
2. How do I find the work email addresses of these recruiters?
3. Sending personalized emails to every recruiter was cumbersome. Especially considering that every company often had multiple recruiters. How do I make this process fast enough?

Well, I did my research. Here are the solutions I came up with, answered in the same order as the questions above:

1. Use LinkedIn. Get a premium account if needed. But search for “Recruiter” or “Technical recruiter” or “HR” in the search bar in the “People” section. Then filter by “Company” and “Location” of the job posting. View LinkedIn profiles of the people from the search result and decide if they seemed like the recruiter who could be hiring for the position you are interested in.
   
2. Again I never knew a google search could help me answer so many of my queries. It really is about asking the right questions. I found that there are services/websites like “RocketReach” or “Hunter.io” that have a database of email addresses for most companies out there. I personally ended up using “Hunter.io” for its easy-to-use interface. All I had to do was type the name of the company and then it would tell me the domain name of the email addresses associated with the company.
   
   In addition, it would also tell me the format of the email addresses at that company. For instance, if I type in “Facebook” then it would return something like “80% of the email addresses at Facebook use the format {first}.{last}@facebook.com”.
   
   So if I know that there is a “Archit Kumar” working at Facebook then that employee’s work email address would be [email protected]. It involves hit and trial but in my experience, it works most of the time. Great! All I had to do was substitute the names of the recruiters I found from step 1 and voila I have the email address of a recruiter!
   
3. This was a difficult task. First I had to complete steps 1 and 2. Then I had to write personalized emails to every recruiter. No matter how fast I tried I couldn’t manage more than 1 application per day. There had to be a better way. I gave this problem a lot of thought. It came to me that I knew how to write code in Python and Python happened to be one of the best computer programming languages for scripting.

By now all you computer programmers probably know what I did next. I wrote a Python script!

It is important to note that I did not come up with these solutions in an instant. It took me a few days to formulate these solutions. Most of this I understood from just searching on the internet.

I broke down the Python script into three sub-tasks.

1. Have a source/file where you have the information for every recruiter.

For the source file, I decided on using Google sheets, since I didn’t want to buy MS Excel to use on my Mac. I manually searched for the jobs I was interested in on job boards like LinkedIn and Indeed. I then entered the information into my sheet as seen below. Note that I have used dummy names and emails here, but you get the idea.

2. For the email body I created a .txt file with Python template strings so that I can substitute information for every individual recruiter.

As you can see below the email body accounts for substitutions for the name of the recruiter (REC_FIRST_NAME), name of the company (COMPANY_NAME), name of the position (POSITION_NAME), and name of the location for the position(POSITION_LOCATION). This is just an initial template I created. Feel free to tweak the content to your liking.

This way I could customize the content of my email to each recruiter and not come across as a generic email that the recruiter thinks I probably sent out to multiple recruiters/companies.

3. Write Python code that can parse the source file, make relevant substitutions in the email body and send out emails to the recruiters.

The code can be found in my GitHub repository here.

Here are the Python libraries I used :

Note that I was using outlook email to send out emails. The above libraries should work if you plan on sending out emails using Outlook. However, I did try sending out emails with Gmail but I don’t think it worked.

Instead of using the “smtplib” and “email” libraries I used “yagmail” which works fine with Gmail. I have a separate script for using Gmail on my GitHub. There are additional settings like the two-factor security/authentication that Gmail offers which you might have to change in order for the script to use your gmail to send emails. Just something to keep in mind.

Then I created a function called “read_template” that will read the txt file containing the email body into a “Python Template” — see below embedded code. This allows the Python script to make substitutions in the email body and utilize the template strings later on. Note that I did not parse the google sheet directly (although it is possible with a little tweak) but instead downloaded the sheet as an “.xlsx” file onto my local machine.

I created another function called “get_contact_dict” to parse the excel file (using the “openpyxl” library). This will extract the information from the excel sheet that is required down the line when substituting it into the template(email body).

As you can see, I did type in a few “checks” into the function so that when it gets called at the time of running the script then it prompts me to check if I put in the right pathname of the .xlsx file or the right sheet number within the workbook. Also, I would parse only a part of the entire .xlsx file (I send out emails to only those recruiters that I shortlisted on a given day) which variables “start_row” and “end_row” take care of.

Then I do some string manipulation because in my excel sheet the names and email addresses of recruiters are within one cell, separated by commas.

With the functions defined and libraries imported then I wrote the main program that sets up the variables, calls these functions, and sends out the email.

Above you can see that I set up the variables required by the script that decide what part of the .xlsx file to parse, where to find the .xlsx file, path to my resume, email_id, password and smtp details.

Make the function calls.

Set up the smtp server.

Then I substituted the values in the email body template. Finally I entered the details for the email like “from”, “to”, “Subject”, attached my Resume and sent out the email!

Conclusion

By now you might be asking did all of this effort actually work?

Well yes and no. This will not guarantee you a job or even an interview for that matter. But it did get me an interview with an established investment firm based in New York for a Data Analyst position that required about 2–3 years of work experience.

Would I have gotten this interview with my earlier approach of applying through the job portal? No.

The beauty of this was that the position I interviewed for was not even posted on the careers page of the company. I actually sent out an email (using this script of course) for a Data Visualization Analyst position at the same company.

The recruiter who I sent the email to never replied back and I assumed this was a dead end. But then two weeks after sending out the application, another recruiter from the same company reached out to me for the Data Analyst position. I can only surmise that the recruiter who I initially sent out an email to probably forwarded my resume to the other recruiter.

I was later told during the interview by the CTO of the company that he picked my resume because I had a project that worked with the kind of data that was relevant to the data the company used. Did I get the job? I have no clue. I went through about four different rounds, the last one being with the CTO and MD of the company. After that, I was ghosted, even though I tried to follow up.

Long story short I was able to circumvent the filter where a robot reads my resume. I was able to reach a human being, a recruiter, who went through my resume or at least my email. From getting automated replies rejecting my application, I was able to get a recruiter to reach out to me inviting me to interview for a role. I‘d say that is an achievement.

Anyway, I’d be happy to listen to your challenges with finding jobs, or if someone actually used the script I shared then whether it worked for you or not. What other improvements can you guys think of? I will keep you guys posted with future articles as to what other methods I tried. I hope this read was of some value or at the very least an entertaining read. Either way good luck with your job search!