Why Personalization Projects Fail

It happens alarmingly often; massive projects, meant to personalize site experience, either fail or stop dead after making only small inroads into the problem.

At the heart of most of these failures are a few common mistakes, and at the heart of these mistakes is a failure to understand the personalization loop.

You cannot create a brilliant personalized experience if you don’t have meaningful insights about the human who will use it.

You can’t build meaningful insights about humans without rich, identifiable data.

You won’t have access to rich, identifiable data unless you’re capturing it from your experiences.

Once this loop is primed though, it can be incredibly virtuous. An average experience can generate data, which produces insights, to make a slightly better experience, which generates slightly better data, which produces slightly better insights, which makes for a much better experience…

The challenge, of course, is priming these loops. And this is where a huge number of early personalization solutions flop.

Mistake 1: Chasing the biggest segment

Most web and mobile businesses begin with a single, generalized experience.

These general experiences don’t need personalization, and can deliver substantial value without it. It’s a solid business model.

Developing a system with this business model still requires data though, and one the most popular ways to gather data in a generalized experience is to use generalized tracking tools like Google Analytics (GA). Instead of a true personalization loop, the business will have a loop a bit more like this:

This kind of solution is limited, because it only serves the dominant segments of your audience.

Say you’ve prototyped a variation on your experience, and 80% of users find it more helpful, while the remaining 20% find it less so. The product manager will see these results and decide that all users should see this new experience, for a net gain of 60%.

Unfortunately, the 20% go away disappointed.

In a personalized system however, the 80% who preferred the new experience get it, and the 20% who didn’t, don’t. Perhaps they get the original experience unmodified, or perhaps they just see the new one less often. The key though, is remembering who did and didn’t like the experience, and reflecting their most effective experience back to them.

You can’t build this type of system using only generic data. That data needs to be augmented or replaced with individually identifiable data.

Your systems must remember your users between sessions. Encouraging user membership is one of the most common ways to do this, but fundamentally, all users need unique, reproducible identifiers. Every part of your system needs to be fluent in these identifiers.

Then, all of those parts of the system need to be generating and storing usage data against those identifiers. That way, when you generate insights and build amazing experiences, you can make those experiences adapt to every individual you build them for. If you’re very lucky, you might also find that the portion of your audience which was originally 20% ends up growing to be one of your largest segments — one you might have lost entirely if you’d just only chased the biggest segment.

Mistake 2: Designing experiences without insights or data

Nice UIs are an obvious thing to try to build or replicate, because they are visible and easy to understand. That temptation becomes doubly strong when faced with the amount of effort required to fill out the other two thirds of the personalization loop.

Unfortunately, even teams with a strong design and user focus can easily get caught up in trying to replicate a Netflix style user experience, or investing in good UX to share personally relevant content on their website, only to be horribly let down when the content shared through these experiences makes for an average, mediocre result. Effectively, all the bells and whistles in the world won’t hold up without substance to back it up.

The largest value of personalization comes not from how it looks or operates, but how good the insights are at delighting the user. When designing any new, personalized experience, it’s imperative to also design (or at least consider) the insights that will power it.

Insights can be large, complicated machine learning beasts, or they can be simple if-this-then-that logic. Both ends of the spectrum are perfectly valid. What matters is that you have a way of turning your individual users’ data into a meaningful insight, and applying that insight all the way through to the beautiful experience you’ve designed.

Mistake 3: Treating insights as second class citizens

Once your system has started to accumulate data, there’s a strong instinct to want to immediately pull that data into new experiences. While this is fine for prototyping, it leads quickly to dead ends in personalization.

This happens often when an experience is designed, but the insights aren’t clearly understood yet. Consequently, the insights are calculated within the experience itself.

This works fine while the experience is under active development, but once it is released and the team moves onto other parts of the ecosystem, that insight stagnates. When new data becomes available, the existing experience languishes, unloved. The algorithms are left in their original state, not taking advantage of new techniques or data sources which could make it more valuable.

The original insight is also trapped. Because it is built into a specific experience, it cannot be easily reused in other, new projects. That slows the development of new features, and makes it harder to learn from previous insights. Everything is starting from zero every time.

Insights need to be treated as first class citizens in your design and engineering process. They should exist and be maintained as systems in their own right, so that multiple experiences can be built on top of them.

As the insights evolve and grow, the experiences built on top of them will become richer by default. This acts as a multiplying factor for every personalized experience that has already been developed.

Solving for personalization

The personalization loop is only really effective when all parts of it are respected and considered at every step of the way.

Balance your portfolio of data, insight and experience creation, to make sure that you’re getting the most (and the best) out of every part of your system.

In time, the personalization loop will pay you back, with more and more returns for less and less development effort.