paint-brush
This AI Prevents Bad Hair Daysby@whatsai
267 reads

This AI Prevents Bad Hair Days

by Louis BouchardJune 13th, 2021
Read on Terminal Reader
Read this story w/o Javascript
tldt arrow

Too Long; Didn't Read

AI can transfer a new hairstyle and/or color to a portrait to see how it would look like before committing to the change. It can change not only the style of your hair but also the color from multiple image examples. The results are very superficial compared to more useful applications, such as hairstylists. It is a step forward towards "seeing in the future" using AI, which is pretty cool, and I'll let you watch my other videos where I explained how it works in-depth.

Companies Mentioned

Mention Thumbnail
Mention Thumbnail
featured image - This AI Prevents Bad Hair Days
Louis Bouchard HackerNoon profile picture

Could this be the technological innovation that hairstylists have been dying for? I'm sure a majority of us have had a bad haircut or two. But hopefully, with this AI, you'll never have to guess what a new haircut will look like ever again.

This AI can transfer a new hairstyle and/or color to a portrait to see how it would look like before committing to the change. Learn more about it below!

Watch the video

References:

►The full article: https://www.louisbouchard.ai/barbershop/

►Peihao Zhu et al., (2021), Barbershop, https://arxiv.org/pdf/2106.01505.pdf

►Project link: https://zpdesu.github.io/Barbershop/

►Code: https://github.com/ZPdesu/Barbershop

Video Transcript

00:00

This article is not about a new technology in itself.

00:03

Instead, it is about a new and exciting application of GANs.

00:06

Indeed, you saw the title, and it wasn't clickbait.

00:10

This AI can transfer your hair to see how it would look like before committing to the

00:15

change.

00:16

We all know that it may be hard to change your hairstyle even if you'd like to.

00:19

Well, at least for myself, I'm used to the same haircut for years, telling my hairdresser

00:24

"same as last time" every 3 or 4 months even if I'd like a change.

00:29

I just can't commit, afraid it would look weird and unusual.

00:33

Of course, this all in our head as we are the only ones caring about our haircut, but

00:38

this tool could be a real game-changer for some of us, helping us to decide whether or

00:43

not to commit to such a change having great insights on how it will look on us.

00:48

Nonetheless, these moments where you can see in the future before taking a guess are rare.

00:53

Even if it's not totally accurate, it's still pretty cool to have such an excellent approximation

00:57

of how something like a new haircut could look like, relieving us of some of the stress

01:02

of trying something new while keeping the exciting part.

01:06

Of course, haircuts are very superficial compared to more useful applications.

01:10

Still, it is a step forward towards "seeing in the future" using AI, which is pretty cool.

01:17

Indeed, this new technique sort of enables us to predict the future, even if it's just

01:22

the future of our haircut.

01:24

But before diving into how it works, I am curious to know what you think about this.

01:28

In any other field: What other application(s) would you like to see using AI to "see into

01:34

the future"?

01:38

It can change not only the style of your hair but also the color from multiple image examples.

01:44

You can basically give three things to the algorithm:

01:47

a picture of yourself a picture of someone with the hairstyle you

01:51

would like to have and another picture (or the same one) of the hair

01:55

color you would like to tryand it merges everything on yourself realistically.

01:59

The results are seriously impressive.

02:02

If you do not trust my judgment, as I would completely understand based on my artistic

02:06

skill level, they also conducted a user study on 396 participants.

02:12

Their solution was preferred 95 percent of the time!

02:17

Of course, you can find more details about this study in the references below if this

02:21

seems too hard to believe.

02:22

As you may suspect, we are playing with faces here, so it is using a very similar process

02:27

as the past papers I covered, changing the face into cartoons or other styles that are

02:33

all using GANs.

02:34

Since it is extremely similar, I'll let you watch my other videos where I explained how

02:39

GANs work in-depth, and I'll focus on what is new with this method here and why it works

02:45

so well.

02:46

A GAN architecture can learn to transpose specific features or styles of an image onto

02:52

another.

02:53

The problem is that they often look unrealistic because of the lighting differences, occlusions

02:58

it may have, or even simply the position of the head that are different in both pictures.

03:04

All of these small details make this problem very challenging, causing artifacts in the

03:09

generated image.

03:10

Here's a simple example to better visualize this problem, if you take the hair of someone

03:11

from a picture taken in a dark room and try to put it on yourself outside in daylight,

03:12

even if it is transposed perfectly on your head, it will still look weird.

03:13

Typically, these other techniques using GANs try to encode the pictures' information and

03:15

explicitly identify the region associated with the hair attributes in this encoding

03:21

to switch them.

03:22

It works well when the two pictures are taken in similar conditions, but it won't look real

03:27

most of the time for the reasons I just mentioned.

03:30

Then, they had to use another network to fix the relighting, holes, and other weird artifacts

03:36

caused by the merging.

03:38

So the goal here was to transpose the hairstyle and color of a specific picture onto your

03:43

own picture while changing the results to follow the lighting and property of your picture

03:49

to make it convincing and realistic all at once, reducing the steps and sources of errors.

03:55

If this last paragraph was unclear, I strongly recommend watching the video at the end of

03:56

this article as there are more visual examples to help to understand.

03:57

To achieve that, Peihao Zhu et al. added a missing but essential alignment step to GANs.

04:01

Indeed, instead of simply encoding the images and merge them, it slightly alters the encoding

04:07

following a different segmentation mask to make the latent code from the two images more

04:12

similar.

04:13

As I mentioned, they can both edit the structure and the style or appearance of the hair.

04:18

Here, the structure is, of course, the geometry of the hair, telling us if it's curly, wavy,

04:24

or straight.

04:25

If you've seen my other videos, you already know that GANs encode the information using

04:30

convolutions.

04:31

This means it uses kernels to downscale the information at each layer and makes it smaller

04:37

and smaller, thus iteratively removing spatial details while giving more and more value to

04:43

general information to the resulting output.

04:46

This structural information is obtained, as always, from the early layers of the GAN,

04:52

so before the encoding becomes too general and, well, too encoded to represent spatial

04:58

features.

04:59

Appearance refers to the deeply encoded information, including hair color, texture, and lighting.

05:05

You know where the information is taken from the different images, but now, how do they

05:10

merge this information and make it look more realistic than previous approaches?

05:15

This is done using segmentation maps from the images.

05:18

And more precisely, generating this wanted new image based on an aligned version of our

05:24

target and reference image.

05:26

The reference image is our own image, and the target image the hairstyle we want to

05:31

apply.

05:32

These segmentation maps tell us what the image contains and where it is, hair, skin, eyes,

05:38

nose, etc.

05:40

Using this information from the different images, they can align the heads following

05:44

the target image structure before sending the images to the network for encoding using

05:49

a modified StyleGAN2-based architecture.

05:52

One that I already covered numerous times.

05:55

This alignment makes the encoded information much more easily comparable and reconstructable.

06:00

Then, for the appearance and illumination problem, they find an appropriate mixture

06:05

ratio of these appearances encodings from the target and reference images for the same

06:11

segmented regions making it look as real as possible.

06:15

Here's what the results look like without the alignment on the left column and their

06:19

approach on the right.

06:21

Of course, this process is a bit more complicated, and all the details can be found in the paper

06:26

linked in the references.

06:27

Note that just like most GANs implementations, their architecture needed to be trained.

06:32

Here, they used a StyleGAN2-base network trained on the FFHQ dataset.

06:38

Then, since they made many modifications, as we just discussed, they trained a second

06:42

time their modified StleGAN2 network using 198 pairs of images as hairstyle transfer

06:50

examples to optimize the model's decision for both the appearance mixture ratio and

06:55

the structural encodings.

06:57

Also, as you may expect, there are still some imperfections like these ones where their

07:02

approach fails to align the segmentation masks or to reconstruct the face.Still, the results

07:08

are extremely impressive and it is great that they are openly sharing the limitations.

07:13

As they state in the paper, the source code for their method will be made public after

07:18

an eventual publication of the paper.

07:21

The link to the official GitHub repo is in the references below, hoping that it will

07:25

be released soon.

07:27

Thank you for watching!