What can we really learn from failure?

I’ve been urged, directly or indirectly, to learn from failure many, many times. I read it in articles, I hear from people I speak with, I hear it in presentations.

But failure doesn’t hold that much information. Specifically, why you failed tells you very little. It eliminates one of thousands of possible scenarios. This makes intuitive sense, but there is also some careful research supporting it: here.

Null Hypothesis Logic

Hypothesis testing is a common idea in statistics. You have a set of data, a hypothesis, and a statistical test. The end result is to decide whether you can reject the null hypothesis in favor of a specific alternative. The null hypothesis is almost always an equality condition (i.e., something of interest equals something else.) Sometimes it is a range (i.e., something of interest is between two numbers in value). If you fail to reject the null hypothesis, you don’t know that the null is true, you just know that the particular alternative hypothesis you tested on that particular dataset in that particular way didn’t reject the null.

The null is a sort of default value, or at least it is often set up that way. In life, and especially in startups, the same logic applies. The default value is failure. To reject the null is to succeed. But if you don’t succeed, it may have been for many reasons. You may have had a good idea but didn’t market it well, or didn’t implement the product well. Perhaps you ran out of money. Perhaps investors didn’t think it would work and so never gave money in the first place. Perhaps you executed perfectly and investors thought it would succeed, but the market didn’t want it. It’s tempting to say ‘I failed because of X,’ but it’s dangerous. It would have been a mistake had Mark Zuckerburg decided that Facebook would fail because MySpace failed.

Success teaches you more than failure. You know exactly what you did, and you know that it worked. However, even success doesn’t teach as much as one might think, because context matters a lot. In a recent talk, Peter Reinhart of Segment.io quoted a study that put some hard figures on this: people who had failed before had roughly a 20% chance of success on a new startup, which is the same rate as novices. People who had succeeded before had a 34% chance of success on a new venture. Success teaches more than failure, but not that much. It’s hard to conclude a great deal from one study, but it’s suggestive.

So it’s tough to learn from failure. But I think it can be done, and I want to explain how. I’ll be concrete and use a recent failure of mine: Dreamless, a pillow subscription service. You can see my post a couple weeks ago about Dreamless to learn more:

Hacking Together a Business in 2 Weeks_Note: Dreamless is a pillow subscription service. It’s live on Indiegogo right now. I’d love it if you’d consider…_hackernoon.com

Failures Contain Small Successes

The first principle is simple: if you try something, it can usually be broken into steps. If you can measure results in a more granular fashion, successes often appear. Don’t lose successes by binding them up with the grand failure.

In the case of Dreamless, I wanted to raise money to get a pillow subscription service going. I failed, miserably. The campaign page shows I reached 4% of my $8000 goal, and all of that was from friends and family. I didn’t get a single backer that didn’t know me personally before the project.

A friend of mine commented that he thought such a service could work under the right conditions. Supposing he’s correct, it’s hard for me to know exactly which things I did wrong that led Dreamless to its grand failure. However, if I break things down, some things are pretty clear.

I can learn from small things I did right, and I can learn from things that went wrong (even if I can’t necessarily say which things caused the failure). I wanted to get at least 500 visits, I wanted to see how different traffic sources performed, and I wanted to understand consumer reactions to the idea better than I have on previous ideas. I succeeded on those three things.

My Google Analytics for the Indiegogo Campaign page:

So, 874 sessions is better than 500. That’s pretty good. Here is the traffic breakdown:

If you aren’t familiar with this language, Direct means traffic from Indiegogo itself. Social is from big social websites like Facebook, Instagram, Twitter, and Reddit. Referral is from a link provided on a site other than the ones listed above. Organic search is via Google, generally.

There is a lot of useful information here: 808 different users came and went very quickly. By about the 4th day, almost all of the traffic had come and gone. In my experience, you can usually tell within the first 24 hours whether you have a success or failure on your hands.

The most useful information is that direct traffic is such a large group. This shows you the value of validating ideas on a crowdfunding platform. That direct traffic accounts for so much is a really nice thing for campaign owners, because it is organic, and free (sort of). It also tells you that your thumbnail image on the platform and marketing tagline are the two most important things for driving traffic.

It also means that marketing for crowdfunding projects should be aimed directly at crowdfunding enthusiasts. Direct traffic on crowdfunding is comprised of a certain kind of person. Some products that backers like wouldn’t do as well in traditional markets, because the customer base would change so much.

One unanticipated gem of learning came from an experience with Reddit. On a previous campaign I was involved with (indirectly), I learned of a particular subreddit where people go to hate on crowdfunding campaigns. I learned because our previous campaign made it there, and there were some serious traffic spikes. This time around, for Dreamless, I had a friend of mind post Dreamless to this thread, just to see what they’d say. I learned something valuable: Reddit loves to hate things so much, they will give you detailed product evaluations for free. There are many nice things about this: it’s free to post on Reddit, and the upvoting system filters the feedback for you. What are the most negative things about your product? Of course, it comes with a lot of jokes and irrelevant stuff, but it’s easy to pick through. Here is the feedback for Dreamless. The most upvoted comment that isn’t a joke about the video is:

Or I could buy my own pillow when I feel necessary?

That tells you a lot. I believe this is why most people didn’t think Dreamless was compelling. They don’t see a problem with their pillow purchasing frequency. I happen to think most people don’t change their pillows enough, but I failed to make the case, and Reddit told me loud and clear.

In the future, I have a useful trick for getting feedback. One important note about Reddit though, don’t pay attention to the positivity/negativity of the tone. Reddit hates almost everything. (In fact, the original campaign that Reddit hated so much ended up succeeding beyond our expectations.) The value is in the why. Your conversion rate tells you whether the market reaction is positive or negative, Reddit can help you understand why. Reddit hate can be informative even on successful campaigns: in a previous instance, we decided to offer a new variant because of some Reddit comments (even though we were doing fine) and it increased our revenue by around 15%.

Learn from Other’s Successes

There are three big sources of information about other people’s successes:

1. Biographies
2. Success/Fail Laboratories like Kickstarter and Amazon
3. Online Media

In the case of Dreamless, I didn’t read any relevant biographies, or online media that I can remember. But I did learn something from successful crowdfunding campaigns. Like this one:

This campaign attacks the same problem as Dreamless from a different angle. But Silvon Home sheets knocked it out of the park. One of the reasons, I believe, is that people like an x-factor. In the case of Silvon Home, the x-factor is silver. Dreamless had nothing. I even said in the campaign copy that backers would receive ‘standard polyfill pillows.’ (Now, I have a particularly exacting conscience, and so I wanted to be extremely transparent. But transparency can really hurt. I’m still trying to learn how to strike the balance between writing appealing copy and making unfounded claims.)

Silvon Home is not the first successful campaign that I’ve noticed with some kind of x-factor. It is probably not a sufficient condition for success, but it seems to promote success to a large degree.

Fail Faster

Rather than trying to extract a great deal of knowledge from each failure, take note of the big things you can learn, and then move on. Even if your chance of future success remains at 20%, that’s not so bad. It’s one out of five. So cut your losses, and go the next idea. The relation to Dreamless here is clear: I’m putting that idea to bed, and this post with it.