Sign up as a competitor or employer for👩🏾💻NYC Engineering Tournament👨🏻💻 (Apr 20–23)
Product management is one of the most coveted and difficult to land roles in tech. It is a multidisciplinary role which requires technical know-how, product insight, organizational chops, and people skills—and can bring huge success to a company in certain hands, and major harm in the wrong ones.
For candidates, it can be hard to definitively prove your ability to do the job.
For employers, it can be paralyzing to hire for a role that’s so important yet so hard to define, measure, or articulate.
This means most product management offers go to people who already have the the title “Product Manager” on their resume, with experience at a well-known tech firm or educational institution, or to an internal hire (usually an engineer or designer) with the hope of coaching them on the skills they’re missing.
This makes it almost impossible for people to break into the field.
I faced these challenges myself in breaking into product (you can read the whole story in FastCompany) and that’s part of what inspired me to co-found Headlight, a performance hiring platform. We help employers recruit, screen, and hire engineers, product managers, marketers, and other roles.
One of our company’s key tenets is that we should hire for what people can actually do, and not get distracted by the shiny things they have or don’t have on their resume. The truth is, decades of research has shown that work samples are better predictors of success on the job than years of experience or education.
Our own research on tech hiring has found that rather than people / culture skills, the biggest reasons most tech candidates don’t get jobs is lack of (or perceived lack of) technical skills. While take-home assignments are gaining popularity in screening developers, PM candidates haven’t had as many opportunities to demonstrate their ability to succeed in the role.
At the end of 2017, Headlight launched PM Take-Homes, a free library of assessments / mini-projects that employers could use to screen PM candidates. We got some great feedback from folks who tried out these assignments but what we found was that they were still heavily relying on resumes and thus cutting down their candidate pool way too early.
I had run a one-day product workshop a few months earlier and found an enormous hunger within the NYC tech community for more opportunities to develop their skills and get recognized for their work.
So we decided to take things further and designed perhaps what was the first ever online product manager tournament.
In January of 2018, we announced the NYC Product Tournament in partnership with Day100, a psychology-based hiring tech platform. The goal of the tournament was to help identify who were the top PM’s in the greater NYC area and help tech employers find their next product hire.
We wanted to create a product challenge that was interesting, meaty, and open-ended, to give people a flexibility to interpret and address the problem in their own way.
We had a little fun with it and turned the challenge into a bit of a Marvel fan-fiction.
The challenge asked competitors to develop a product plan (in presentation format) that would be shown to Tony himself when he arrived in the office a few hours later.
They were given a template deck with about 15 different slide formats (big quote, list, 2x2 framework, etc) that they could use to get started. About half of finishers elected to use this and that’s why some of the decks have a similar style.
Here‘s a 5 minute video of the product challenge we presented to competitors. They had three hours to submit their presentation.
To incentivize folks to participate, beyond just the glory of victory of course, we put together some prizes for top finishers. From a tech reading set of Reset by Ellen K. Pao, Hooked by Nir Eyal, and Sapiens by Yuval Noah Harari, to the latest innovation from one of the world’s most successful product organizations: Airpods, to a roundtrip flight anywhere inside the United States, we wanted to have this tournament to have some tangible stakes and outcomes.
We talked before about the challenge of figuring out who might succeed in the product role as being difficult because it requires so many different competencies.
For the sake of the tournament, we broke that down into four categories:
Since we were experimenting with a new kind of hiring process where we hid candidate’s names and even resumes, we wanted to give employers a chance to learn more about candidates, so we turned to our friends at Day100, a psychology-based hiring technology platform that works off the Big Five personality framework that is most widely accepted by psychologists.
Each competitor completed a Day100 assessment, which was reviewed against an “ideal PM temperament” that we created at Headlight. This wasn’t perfect as every company and manager has a different idea of what they’re looking for, but it at least helped reveal more about what a candidate was like.
To bring it all together, we needed to have capable and respected folks reviewing the work of the participants. We assembled a panel of folks from four fields: product, engineering, marketing, and design. By having multiple judges review the work, we’d have a more balanced and healthy view of the participant’s ability.
We recruited judges from different size companies — startups, midsized tech co’s, and publicly traded firms — and in different industries — ecommerce, SaaS, media, developer tools, and social networking.
I have to give a huge shout out to Cliff Kim, Kate Zasada, Veronica Ray, Frantz Joseph, Jordan Obi, Joel Califa, Sam Yeh, and Victor Gamez for giving up their Sunday to churn through dozens of competitor submissions, providing feedback and debating the merits of top finishers to get to our final list. #YouTheRealMVP
On January 4th, we announced the NYC Product Tournament. We shared the tournament with folks in our networks, on private Slack and Facebook groups, on social media, and through our email lists.
In just over two weeks, we had 153 PM’s register to compete!
Most of them were already product managers, with a majority of the remainder in a role adjacent to product like engineering or product marketing.
They were nearly evenly split between men and women and 75% of them were between 25 and 34 years old. About half were Asian-American, another 35% were White, and 16% were Black or Latinx.
Making tech a more inclusive place where people of all backgrounds can thrive is something we care a lot about and we’ll continue to work on making our tournaments accessible, especially to traditionally underrepresented groups.
One final stat — the majority of respondents were actively looking for a new job (60%) while the remainder were open to new opportunities. This isn’t far off from our survey findings which indicated that 75% of tech workers had either changed jobs or seriously considered a move in the past year.
We reached out to a couple dozen NYC employers with PM openings and about 14 folks said they were interested in seeing top candidates.
Over the course of Jan 20–21, we had 53 people work on the Product Challenge. Each competitor was given only three (3) hours to review the prompt, develop a plan, and put a presentation together to convey their ideas.
Every submission was reviewed by at least two judges, and the top scoring presentations (normalized for tough vs easy graders) were identified.
Because we emphasized blind review, judges had nothing to base their judgement on except for the candidate’s work. We even code named these candidates with planets from Star Trek so their race and gender were hidden.
The top 14 of competitors were identified, which represented the top quartile of finishers.
We had all judges look through the top 14 finishers and together they all independently voted for their Grand and First Prize winners. So far, one of those PM’s have gone on to be hired by an employer via the Tournament.
While I’d love to tell you about why all our top finishers (top 25% of all competitors!) are awesome, I’ll stick with just our top four for brevity’s sake.
Below you can see an Airtable of all our top finishers. Interested in meeting any of them? Shoot us a note at firstname.lastname@example.org with your company and a relevant job description and we’ll be in touch!
We started this journey wondering if it would be possible to run a hackthon-esque competition for product managers and I think the answer: definitely!
Overall our competitors found the product prompt to be clear, appropriately challenging and interesting. That said, there were definitely still areas we could improve:
All in all, the Product Tournament was a lot of work, but I’m really happy we were able to pull it off. Probably the best feedback is that 81% of competitors would recommend something like this to their colleagues / peers.
We’re excited to take what we’ve learned from this Tournament and apply it to our next event — the NYC Engineering Tournament 2018.
If you’re interested in having better onsite interviews and reducing time spent in the screening process, check out Headlight. And please give this piece 👏 so others can find it too!
Have thoughts, questions, feedback? Find me at @jasonshen.
Create your free account to unlock your custom reading experience.