Could this be the technological innovation that hairstylists have been dying for? I'm sure a majority of us have had a bad haircut or two. But hopefully, with this AI, you'll never have to guess what a new haircut will look like ever again. This AI can transfer a new hairstyle and/or color to a portrait to see how it would look like before committing to the change. Learn more about it below! Watch the video References: ►The full article: https://www.louisbouchard.ai/barbershop/ ►Peihao Zhu et al., (2021), Barbershop, https://arxiv.org/pdf/2106.01505.pdf ►Project link: https://zpdesu.github.io/Barbershop/ ►Code: https://github.com/ZPdesu/Barbershop Video Transcript 00:00 This article is not about a new technology in itself. 00:03 Instead, it is about a new and exciting application of GANs. 00:06 Indeed, you saw the title, and it wasn't clickbait. 00:10 This AI can transfer your hair to see how it would look like before committing to the 00:15 change. 00:16 We all know that it may be hard to change your hairstyle even if you'd like to. 00:19 Well, at least for myself, I'm used to the same haircut for years, telling my hairdresser 00:24 "same as last time" every 3 or 4 months even if I'd like a change. 00:29 I just can't commit, afraid it would look weird and unusual. 00:33 Of course, this all in our head as we are the only ones caring about our haircut, but 00:38 this tool could be a real game-changer for some of us, helping us to decide whether or 00:43 not to commit to such a change having great insights on how it will look on us. 00:48 Nonetheless, these moments where you can see in the future before taking a guess are rare. 00:53 Even if it's not totally accurate, it's still pretty cool to have such an excellent approximation 00:57 of how something like a new haircut could look like, relieving us of some of the stress 01:02 of trying something new while keeping the exciting part. 01:06 Of course, haircuts are very superficial compared to more useful applications. 01:10 Still, it is a step forward towards "seeing in the future" using AI, which is pretty cool. 01:17 Indeed, this new technique sort of enables us to predict the future, even if it's just 01:22 the future of our haircut. 01:24 But before diving into how it works, I am curious to know what you think about this. 01:28 In any other field: What other application(s) would you like to see using AI to "see into 01:34 the future"? 01:38 It can change not only the style of your hair but also the color from multiple image examples. 01:44 You can basically give three things to the algorithm: 01:47 a picture of yourself a picture of someone with the hairstyle you 01:51 would like to have and another picture (or the same one) of the hair 01:55 color you would like to tryand it merges everything on yourself realistically. 01:59 The results are seriously impressive. 02:02 If you do not trust my judgment, as I would completely understand based on my artistic 02:06 skill level, they also conducted a user study on 396 participants. 02:12 Their solution was preferred 95 percent of the time! 02:17 Of course, you can find more details about this study in the references below if this 02:21 seems too hard to believe. 02:22 As you may suspect, we are playing with faces here, so it is using a very similar process 02:27 as the past papers I covered, changing the face into cartoons or other styles that are 02:33 all using GANs. 02:34 Since it is extremely similar, I'll let you watch my other videos where I explained how 02:39 GANs work in-depth, and I'll focus on what is new with this method here and why it works 02:45 so well. 02:46 A GAN architecture can learn to transpose specific features or styles of an image onto 02:52 another. 02:53 The problem is that they often look unrealistic because of the lighting differences, occlusions 02:58 it may have, or even simply the position of the head that are different in both pictures. 03:04 All of these small details make this problem very challenging, causing artifacts in the 03:09 generated image. 03:10 Here's a simple example to better visualize this problem, if you take the hair of someone 03:11 from a picture taken in a dark room and try to put it on yourself outside in daylight, 03:12 even if it is transposed perfectly on your head, it will still look weird. 03:13 Typically, these other techniques using GANs try to encode the pictures' information and 03:15 explicitly identify the region associated with the hair attributes in this encoding 03:21 to switch them. 03:22 It works well when the two pictures are taken in similar conditions, but it won't look real 03:27 most of the time for the reasons I just mentioned. 03:30 Then, they had to use another network to fix the relighting, holes, and other weird artifacts 03:36 caused by the merging. 03:38 So the goal here was to transpose the hairstyle and color of a specific picture onto your 03:43 own picture while changing the results to follow the lighting and property of your picture 03:49 to make it convincing and realistic all at once, reducing the steps and sources of errors. 03:55 If this last paragraph was unclear, I strongly recommend watching the video at the end of 03:56 this article as there are more visual examples to help to understand. 03:57 To achieve that, Peihao Zhu et al. added a missing but essential alignment step to GANs. 04:01 Indeed, instead of simply encoding the images and merge them, it slightly alters the encoding 04:07 following a different segmentation mask to make the latent code from the two images more 04:12 similar. 04:13 As I mentioned, they can both edit the structure and the style or appearance of the hair. 04:18 Here, the structure is, of course, the geometry of the hair, telling us if it's curly, wavy, 04:24 or straight. 04:25 If you've seen my other videos, you already know that GANs encode the information using 04:30 convolutions. 04:31 This means it uses kernels to downscale the information at each layer and makes it smaller 04:37 and smaller, thus iteratively removing spatial details while giving more and more value to 04:43 general information to the resulting output. 04:46 This structural information is obtained, as always, from the early layers of the GAN, 04:52 so before the encoding becomes too general and, well, too encoded to represent spatial 04:58 features. 04:59 Appearance refers to the deeply encoded information, including hair color, texture, and lighting. 05:05 You know where the information is taken from the different images, but now, how do they 05:10 merge this information and make it look more realistic than previous approaches? 05:15 This is done using segmentation maps from the images. 05:18 And more precisely, generating this wanted new image based on an aligned version of our 05:24 target and reference image. 05:26 The reference image is our own image, and the target image the hairstyle we want to 05:31 apply. 05:32 These segmentation maps tell us what the image contains and where it is, hair, skin, eyes, 05:38 nose, etc. 05:40 Using this information from the different images, they can align the heads following 05:44 the target image structure before sending the images to the network for encoding using 05:49 a modified StyleGAN2-based architecture. 05:52 One that I already covered numerous times. 05:55 This alignment makes the encoded information much more easily comparable and reconstructable. 06:00 Then, for the appearance and illumination problem, they find an appropriate mixture 06:05 ratio of these appearances encodings from the target and reference images for the same 06:11 segmented regions making it look as real as possible. 06:15 Here's what the results look like without the alignment on the left column and their 06:19 approach on the right. 06:21 Of course, this process is a bit more complicated, and all the details can be found in the paper 06:26 linked in the references. 06:27 Note that just like most GANs implementations, their architecture needed to be trained. 06:32 Here, they used a StyleGAN2-base network trained on the FFHQ dataset. 06:38 Then, since they made many modifications, as we just discussed, they trained a second 06:42 time their modified StleGAN2 network using 198 pairs of images as hairstyle transfer 06:50 examples to optimize the model's decision for both the appearance mixture ratio and 06:55 the structural encodings. 06:57 Also, as you may expect, there are still some imperfections like these ones where their 07:02 approach fails to align the segmentation masks or to reconstruct the face.Still, the results 07:08 are extremely impressive and it is great that they are openly sharing the limitations. 07:13 As they state in the paper, the source code for their method will be made public after 07:18 an eventual publication of the paper. 07:21 The link to the official GitHub repo is in the references below, hoping that it will 07:25 be released soon. 07:27 Thank you for watching!
