I recently quit my hedge fund job to pursue entrepreneurship. My hypothesis is that in the long term entrepreneurship outperforms high paying jobs, and that most risk of failure can be minimized when starting a business by choosing conservative (easy) problems, and applying rational thinking as well as creativity to business problems. I started chronicling my success and failures here, this is article 4. The next article is here
My first project is a group email business. I’m going to go into how I tried to make my landing page better with Mechanical Turk.
GroupMail enables users to easily create email groups by forwarding existing threads. There is a built in viral component, where people create groups with their friends, and hopefully their friends create more groups with more friends, etc. However converting from my website is my first starting point. Up until now I didn’t have very much traffic, but enough to know that no one is converting and my landing page must be terrible.
If you’ve never used Mechanical Turk before, Mechanical Turk is an Amazon product where you can pay random people on the internet to do something for you for a fixed fee (I was offering 5 cents). It’s an incredibly cheap way to do market research, but the downside is there is some selection bias in who you’re going to get. I setup a survey which asks the following
I’m only offering 5 cents per survey, so I got back some ridiculous answers. For example for question 1 about what this business does — here are some of the nonsense answers I got.
The good news is, most people who weren’t giving me garbage did know what this business does. There was also a small sample that confuses GroupMail with online marketing mailing lists like MailChimp.
So the problem is there is no quantitative benchmark for what is a good score here this would be amazing as a conversion rate but since these are paid survey takers it’s hard to anchor these results. Slightly less than half said they would use GroupMail . 24/102 people said they don’t use email groups, and 23/102 people said they didn’t see any value to the service. I don’t think there is much I can do for those people.
I was able to get some interesting qualitative information. Here are the interesting reasons for rejection.
I’m surprised that someone who runs their own mail server still wants to fill out my survey for cash, but there were a few key takeaways
I also wanted to baseline these results, so I created another survey from the perspective of a competitor, gaggle email. I asked the following questions
If they answer yes
If they answer no
This was pretty interesting to me because it allows me to benchmark the performance of my landing page against the performance of a competitor and find out what sets them apart from me.
Slightly less than half said they would use the service — which is basically the same performance as GroupMail.
Only 36 percent would switch to GroupMail — which is pretty damaging to me considering my service is free. Here are the reasons cited:
Wow, so I have a very tiny impact on people who don’t like gaggle.email.
One person got what I was going for!
Groupmail feels soothing to the nominal of all the painful google’s and gaggle just seems to add on to it.
Unfortunately it’s only one person.
Here are my takeaways from this exercise
Based on this feedback, I’ve made text changes to differentiate GroupMail from common alternatives, both client side email groups, as well as mailing lists from Google and Yahoo. I haven’t made any changes regarding trustworthiness or professionalism, as those require much more work.
So Mechanical Turk can be a very useful way to get feedback — both on your site, and how you compare against competitors. Extracting any quantitative information out of mechanical turk for these purposes is a bit difficult however, since there is a significant selection bias and since these people paid.
If you want to do this yourself, here are a few things boring details to know. If you will never do this, skip this paragraph. Amazon Mechanical Turk requires workers to give some sort of “answer” to amazon to get credit for their work. Most people who use Turk for surveys generate a code per survey and have Turkers submit that so you can verify that they did the job. Google Forms does NOT let you vary the code, you just have to output some code at the end of the survey. If you click the gears icon that will allow you to choose what is displayed after submission.
You can also have a question asking for their amazon worker ID (several people online recommend this). That would allow you to confirm who has taken your survey, and if anyone clearly did not actually take your survey you can refuse payment. However if you’re only charging 5 cents, it’s not worth your time to do this (the next time I do this, I’m not going to ask for worker IDs). DO NOT ask for email addresses. That is a violation of policy and you will get banned. Finally Jobs you create in mechanical turk are called batches.
Both batches I have are setup for 50 tasks. I can edit one of them, and ask for more tasks, or change the price per task. The important thing is that if you want to gather more results — do NOT create a new batch. Amazon has controls in place so you can prevent workers from taking multiple tasks from a batch(I do not want the same person taking my survey twice). Creating multiple batches circumvents this control
Thanks for reading!. My past projects are about simple easy email groups, and getting help for open source projects. My current project provides cloud hosted Jupyter notebooks. My next article is about moving from groupmail to a different business — a support model for open source developers