How I built my first Machine Learning Software-As-A-Service

And guess what? It didn’t need any funding. But it did need humans (and still does)

My first payment from a real customer finally cleared. Product Pix just made $65.36. I feel like I can finally write a bit about my journey so far in building a SaaS (software as a service). I can finally tell for sure that someone out there thinks my site is generating real value.

And the site still doesn’t have any payment or subscription page! The transaction arose from the client reaching out to me and asking how much the service costs, because they needed to use my service at scale.

That’s a good sign, right? So elegant. Here I am, a machine learning consultant, writing some code on my spare time, and I’m getting people to pay me for using it. Sheer elegance. Here’s a flow chart:

Well, turns out there’s a few pieces missing from this flow chart.

I just learned the hard way that marketing, product design and full stack development are all very real skills. The good news they’re all very fun to pick up, and as long as you keep trying and changing your approach, you’re bound to get better at them.

How it Started

What I love to do most is build machine learning products. I like doing it so much, that it’s my full time job as a freelancer. But, I thought, wouldn’t it be cool to build a product from A to Z, where machine learning is the core component? Wouldn’t it be cool to launch a web page, have customers stroll in and pay a little for code that I wrote?

So I set out to see what people might pay for and might be solvable with machine learning.

The search for a problem begins

Having recently left LinkedIn, I thought helping people identify their most professional photo from the pool of photos they have on their camera roll might be a good direction. Then I tried hacking a proof of concept on my own phone and found out my friends and I had 0 professional photos on our camera rolls, no matter how you crop them.

I went back to the drawing board like that a couple of times, and finally settled on something I liked. Looks like people who sell online is kind of a big deal (Shopify having reached $20B valuation is a case in point). And all these sellers have to do tedious manual labor in removing background from product photos. Not only that, some of them pay people to do this.

A machine can do that! Let’s build a service that removes the background of a product image!

Fast forward and — hey it works!

Let the Fun Begin

A guy with a hammer sees only nails. So I immediately set out to build my first machine learning model to remove backgrounds from photos. Mind you, I did this before I had a website, before I figured out whether I have any way to bring users to that website, and before I knew what sort of pictures they’d be uploading.

In other words, I started out by doing the one thing I’m an expert in. There’s any number of reasons why this is a silly priority. Most notably, it’s not the best place to start because I’m not revealing answers to great uncertainties in this project (who will pay for background removal, what sort of photos they’re taking, and how do I find those users).

But in my defense: a part of the motivation for building a bootstrapped SaaS was for fun. And if toying around with semantic segmentation models is my idea of fun, can you really judge me?

Alright Fine. Judge me.

I’ll probably write a long post about the numerous iterations I’ve done in perfecting the machine learning model here. I like to geek out about machine learning quite a lot. For now I’ll say I started out by synthesizing data: downloaded a bunch of backgrounds from the internet, and embedded product pictures inside these artificial backgrounds to teach my model how to remove the background.

So I’d take a bunch of backgrounds like these:

artificial backgrounds

And I’d just stick random product photos on top of them, and teach my model to get rid of the background.

Paste these photos on top of the backgrounds above. Now you’ve got yourself a machine learning dataset!

So I’m basically imagining what sort of problem I might need to solve, and letting my model solve these synthetic examples of fake products with fake backgrounds. Surprisingly, it sort of worked. These sort of half-baked results got me excited:

Only an optimist can do machine learning. These poor results got me excited that the problem is looking solvable!

Once I saw the model is sort of starting to give sane results, I lost a couple of weekends trying to improve it. And improve it had.

Getting there…

But at some point my modeling efforts got stuck. I soon realized that improvements would be hard to come by without real data to supplement the synthetic data I was training on.

And yes, I tried harvesting the web for “product photo with backgrounds”. You’ll find it’s very hard to get realistic “before” photos.

No one uploads photos of furniture with white drapes behind them, or shoes in a bathtub — but as I later found out, that’s the kind of photos real users who try to sell stuff are uploading. Only live users can give me this data.

In any case, I was so happy with these results I wanted to share them with the world. Surely once they see what my model can do, they’ll be impressed!

Build a website (with a bit of help from my friends)

I’m lucky enough to count amount my friends a crazy talented full stack that’s overworked and not interested in joining pointless adventures. But he was like “yeah I’ll give you a skeleton website”. Wham, a couple of hours later I have something that looks like a reasonable website for me to modify and hook up to my model.

I never wrote a line of Javascript or Ruby before, but again — learning new skills was half the goal here. So I modified away. Launched the website, posted about it on a bunch of reddit groups. And…

drumroll…

A few people dropped by the website, I even paid a couple of dozens of dollars for google ads. People would drop in, upload a photo, and never come back. Eventually a reddit user explained it to me with a very redditesque level of politeness:

Brutal honesty: your app app sucks. You keep posting about your lame shit but it doesn’t work. Stop spamming about it cause no one want to hear it.

Stand by Your Product

It took a while, but eventually I had to come to terms with reality. Something that almost, sometimes works is impressive for an engineer that sees the potential, but unacceptable for a user that needs to get a problem solved. The point in time where I moved from 0 to 1 in terms of “are there users that keep coming back”, was when I teamed up with a a first graphic designer. Any user uploads now stream into a queue for graphic designers to fix up. If it’s not a product photo, they drop it. If it is — the designers remove the background, and the user gets a perfect result by mail.

The beauty of it: the user gets something of value, and my model gets gold: a real product photo with a real label for how a cleaned photo should look like!

And Force Your Users to Benefit from It

My initial thinking was: let’s show the user the automatic result, and let the user optionally type in an email address if the result isn’t perfect, with the promise that the user will get a manually fixed result once it’s done. Surely if the user gets a result that she’s not satisfied with, she’ll type in her email to get a perfect result? Right?

Wrong. 4.5% of users typed in their email address.

I talked about it with people who know about marketing. And they had a very insightful suggestion: that I hide the automatic result. Want to see your automatic result? Please type in an email, and then you’ll see it. The result? Five fold increase in likelihood that a user will give her email. And that user will always get back an email with perfect, hand-crafted results with perfect background removal.

Only 4.5% of users opted to type in their user email — until I held their photos hostage

Talk To Your Users

I’m definitely not the first to say this, but witnessing first hand the power of talking to your users was impressive. After a while I got into the habit of sending emails to users that upload a nice chunk of photos, or those that uploaded a couple of photos and then went away.

There was nothing systematic in these emails. I had no branding , no template — just something along the lines of “hi, I’m Uri and I built the website, wondered how do you feel about it”. I wrote the email from scratch every time, and I still don’t know why I didn’t write one standard template and repeat it — but I’m glad I didn’t.

Almost everyone responded. And they told me lots of things:

• Users told me they needed higher resolution (so I changed it)
• That they need a dashboard to see their results in one place (so I built one)
• That they need to be able to batch-download photos (so I added that)
• That they need to be able to batch-upload more than 12 photos (it’s 500 now)
• They told me they don’t know how to download the photos from my email as they were embedded in the mail (so I added instructions in the email)

Finally, months after the brutal reddit message that notified me that my app is shit, I started getting messages like these:

Thank you, nice person on reddit!

If You Build It They Will Come (but only if you persist)

I know it’s not sustainable to offer manual background removal for free for the long term. But one thing I found already is that when you talk to your users, and when you give them real value — they’ll pay you back, quite literally.

My first paid conversions are entirely voluntary, by users who just feel like it’s only fair that they chip in for manual image background removal. Long term, I’ll have to start charging at some point. But honestly, I wish I had a few $10Ks to spend here on delighting my customers and building up a monstrous dataset of real products and their background removed variations. Because behind the scenes, my model is improving at a rapid pace, and already a nice chunk of results that user get by email are automatically generated — completely indistinguishable from human-made result.

Onwards!

So my first payment wasn’t this clean “user walks up to code and pays for code’s output”. It was a combination of automatic results, human reviews, and code that helps humans serve results with consistent quality to the user. A far cry from my users -> code -> money flowchart.

But that’s perhaps the biggest lesson I’ve learned. Just because I want to build a machine learning model, doesn’t mean the path to success necessarily starts by perfecting a model. I sometimes imagine an alternative world where I kept at it, iterating my machine learning model without launching, not talking to my users, not knowing the reason no one uses my service is low resolution and lack of a dashboard. I’d still be imagining that maybe all I need is a somewhat better model and then users will come rushing in!

Still, I believe the potential of automation here is huge. And I can’t wait to see how the model improves now that it has a seedy stream of data that reflects exactly what sort of problems my users want to solve.

Oh, and by the way: If you or anyone you know might need to remove the background from a product picture — go ahead and give Product Pix a try. Every product photo you upload makes it somewhat better.