I Wrote a Cover Letter Using Machine Learning and GPT-2 and The Results Were...Not That Bad 😱

Ah yes we’ve all been there. Need a job, but not desperate enough to beg random strangers on the street to trade hookups for meals and rent money yet. Instead of just using a bot to apply to every job on the internet, we’re relegated to looking at the actual ad and write 😮 a cover letter.

My first attempts ended up with me browsing memes and making memes instead of writing a cover letter to the first job I found on LinkedIn. Doing nothing but writing code and crappy memes for years has me a bit dull in professional writing department so in my hour of need I turned to writing bots to outsource this labor for me. I’ve wanted to make a project using GPT-2, but I had not found something that needed that much power until now. 

Fig 1. The lifecycle of finding and getting a job

Most of the cover letter templates I found online involved LinkedIn stalking or asking around to find the person hiring for the position and yielded a worse cover letter, so instead of inputting that into GPT-2, I input portions of job ad with this first section that I took from a template.

Fig 2. The only part written by a person

GPT-2 is not very good at generating long sequences and there might be some recent advancements in the field that might solve these problems, but instead I used a few tricks to get around this. The end result was a couple pages and all I had to do then was omit all the portions I did not do. Compared to older full attention models from 2016–2017, the quality is much better because of the pre-training weights.

I didn’t get a callback for the job, but I was still pretty impressed with GPT-2. Unlike the cover letter, I did not write this post with a bot, so it is a pretty short post. 😜