The Risk of a Scientific Creative Process

Sometimes we create as a form of play, for fun, or to satisfy unguided curiosity. Other times we create with a purpose, an intent to move the world in a direction we want it to go.

As I explain in my post The Fundamental Tension in Product, there is a paradoxical nature to creating for change. Changing the world seems to simultaneously require bending the world and bending to it. Products, artworks, and political movements achieve minimal impact, regardless of their genius, when they are too far away from public consciousness, economic realities, or technological feasibility.

To fit our work with current reality, we are compelled to adopt a scientific sensibility; a mindset of running experiments and dispassionately discarding ideas shown to be false. We take on the perspective of the skeptic, looking at our own intuitions with suspicion. We require seeing evidence before believing hypotheses to be true.

Within your organization or movement, it may be the case that you are a minority voice advocating for a more scientific approach. Frustrated by watching investment in seemingly arbitrary directions, perhaps you’re begging your peers to read The Lean Startup. But for now, let’s assume the value of testing ideas is a given.

As the pendulum of our creative process swings towards science, a new risk emerges. With the scientific approach, we methodically weed out bad assumptions, continuously conforming our creations to living behavioral patterns. However, as our creations transform, after countless iterations and discoveries, we risk losing touch with the original vision we sought to make real at the outset — we crossover from changing the game to mastering its rules. Venkatesh Rao characterizes this phenomenon as visioning-process debt.

A scientific approach, while optimal in many scenarios, is insufficient for changing the world. The most impactful creations seem to come, in part, due to an unwavering vision that willfully ignores the apparent workings of society.

It might appear on the surface that holding an “unwavering vision” is incompatible with the scientific sensibility, but this is not necessarily the case. If you were to stay committed to a hypothesis that was proven false, there is surely a problem. But this does not imply that a hypothesis must be proven true before you invest in pursuing it.

In his 1896 paper, The Will to Believe, pragmatist philosopher William James argues for the necessity of believing some hypotheses before you have proof. His overall agenda is to argue for the compatibility of religion and the scientific sensibility. Religion is the archetypal untestable hypothesis. Belief in an afterlife cannot be falsified until you’re dead.

But there are less extreme cases that require believing hypotheses before they’ve been tested. Consider what it’s like to work with a team. You must believe that other members of your team will be genuinely committed to the same cause. If you require evidence of commitment before you contribute your part, your skepticism will cause the hypothesis to fail. Your team members, seeing your disengagement, will also disengage.

James writes on religion and other cases where a hypothesis must be believed prior to testing:

We cannot escape the issue by remaining sceptical and waiting for more light, because, although we do avoid error in that way if religion be untrue, we lose the good, if it be true, just as certainly as if we positively chose to disbelieve… Scepticism, then, is not avoidance of option; it is option of a certain particular kind of risk. Better risk loss of truth than chance of error,-that is your faith-vetoer’s exact position. He is actively playing his stake as much as the believer is; he is backing the field against the religious hypothesis, just as the believer is backing the religious hypothesis against the field. To preach scepticism to us as a duty until ‘sufficient evidence’ for religion be found, is tantamount therefore to telling us, when in presence of the religious hypothesis, that to yield to our fear of its being error is wiser and better than to yield to our hope that it may be true.

While for me, as James puts it, religion is a dead hypothesis, I think his sentiment sheds light on how great creators persevere against the social grain. On the one hand, science is underused in how people strive to spark change. Science should creep into more and more creative domains. On the other hand, experiencing the power of the scientific sensibility can bias us against less testable directions. I designed the below maze to illustrate this bias.

After entering the maze, you quickly reach a fork in the road. If your goal is to efficiently solve the maze, you will likely look ahead in both directions before committing to a path.

If your eyes follow the path on the right, you immediately see avoidable dead-ends. This path is intended to evoke the scientific sensibility, a process centered around avoiding falsehoods. The hope is that methodically avoiding errors will lead to the discovery of a meaningful new truth, but this is not guarenteed.

The left path, in contrast, winds around with no foreseeable dead-ends. But it very well could ultimately lead to a dead-end. Finding out will take time. Heading this direction requires a “leap of faith.” You will get no feedback, positive or negative, until you reach victory or failure. It’s intended to be a metaphor for staying committed to an unwavering vision, regardless of evidence, or lack thereof.

So which road do you feel inclined to pick? Either one comes, as James argues, with potential upside and risk. However, as the scientific sensibility increasingly takes over our mindset, we risk becoming biased to take the path on the right, overlooking the opportunity presented on the left.