2017: A Data Odyssey

I’ve been designing, developing and marketing things on the web for the last decade. In that time, I’ve probably worked on 50–100 projects. My single most persistent frustration has been our collective relationship with analytics and data.

When it comes to data, I’ve observed that people fall into one of two ideological camps:

1. The Gut-Driven Camp — “I intuitively know what’s good or bad. I’ll use my instincts to drive this project forward.”
2. The Data-Driven Camp — “We can’t possibly know what will work or fail, so let’s make no assumptions and A/B test everything.”

Both approaches have merit but are flawed if they exclude the other camp. Intuition typically drives innovation and ideation. Data focuses and guides us towards our stated goals. We need both. We need the confidence to assert our beliefs and the modesty to adapt as we learn.

Most analytics tools and frameworks support one camp while leaving the other in the cold. I’ve been thinking about a way to encourage teams to experiment productively by comparing their ‘gut driven’ initiatives to their ‘data driven’ KPIs (key performance indicators).

Tractorbeam is my data odyssey. It’s a journey to see if we can unite these two camps. The concept started to take hold during my time in the Techstars Cloud program. We used to have these weekly meetings at 9:09pm to gather around, drink beer and go over our company KPIs. We all found it challenging to not only present our KPIs but also to articulate what we’d do to move the needle.

V1 of Tractorbeam is now live. Its a milestone and campaign tracking tool. Try it out and let me know what you think.

Examples of how your team could use Tractorbeam:

Launching and Growing an App

There is no right way to build a successful app. But here are a few phases many apps go through.

1. Design a prototype — At this stage, you don’t really have KPIs. Doing user interviews and surveys can give you signals that influence how you think about your MVP. I’d suggest not getting too deep into tracking metrics over time at this stage. Use your gut, make bold assumptions and take the leap.
2. Launch an MVP — Now you need to start thinking about KPIs. Don’t get too deep into the funnel yet because your product is likely somewhat janky and the few users you convince to pay are unlikely to be representative. Instead, focus more on the top of the funnel. Can you attract people to your idea and get them to essentially put their name on a list saying they’re interested. I’d focus mostly on traffic and signups here while starting to think about KPIs deeper in the funnel.
3. Iterate on the product — Start focusing more on the middle of the funnel. Are people going beyond the basic signup and actually using your product? Daily or weekly active users are a great measure of success but don’t be afraid to track more specific usage metrics.
4. Grow — Once your product is getting real usage, start taking the bottom of the funnel more seriously. Ultimately, most apps are going to want to track revenue as a primary KPI.

With Tractorbeam, each graph allows you to look at 2 metrics. The intention here is to let you look at a primary metric based on the current stage you’re optimizing for. The secondary metric is intended to show a deeper underlying metric. So say you are on stage 3, you might want to track ‘Weekly Active Users’ as the primary metric and ‘Revenue’ as the secondary metric.

Over time, you might find that a different stage 3 metric has a stronger correlation to a stage 4 metric. In this case maybe ‘Documents Created’ is a better measure of activity so you focus on optimizing for ‘Documents Created’ instead of ‘Weekly Active Users’.

Personal Analytics

Tractorbeam wasn’t designed specifically with personal analytics or quantified self in mind. But I think this is a good example to show an alternative use case. It’s also very relevant considering I’m writing this on Jan 2nd, 2017.

So you’ve got a New Year’s resolution to be more fit. How might you make that goal less ambiguous? Lose 25lbs, drop a few inches from your waist size? Being more specific gives you something to work towards but how do you know what is working and what isn’t? Are you just going to run 1 mile every day and hope to drop 25lbs? If so, you’ll probably get off to a good start and run every day for a month. Then you’ll weigh yourself and find little progress and get discouraged.

What I’d recommend is an approach that’s a little more experimental. Make a list of 4–5 things that have the potential to influence your weight:

1. Workout for an hour every day
2. Stop drinking soda
3. Stop drinking alcohol
4. Replace snack X with snack Y
5. No desserts after dinner

Now weigh yourself every day to create line chart of your weight. Now start running 2+ week campaigns designed to influence your weight. Maybe the first campaign is to workout an hour every day. You commit to doing that for 2–3 weeks, then come back and overlay the campaign over the weight metric to see how the number changed. Maybe you’ll find that you only dropped 1–2 lbs and while that is going in a positive direction, it isn’t good enough. So then start a new campaign to stop drinking soda and see what that does to your weight after a few weeks.

Now you’ve got a feedback loop. Keep experimenting by launching new campaigns and seeing what has the most influence on moving the number you care about. Do that for a year and I’ll bet you’ll make real progress!

It’s very early days at Tractorbeam HQ. I hope TB helps your team explore a new frontier. If not, good luck in finding your own data Odyssey.

— Dane Lyons