This new model by Google Research and Tel-Aviv University is incredible. MyStyle is a very powerful deepfake model that can do basically anything. Take a hundred pictures of any person and you have its persona encoded to fix, edit or create any realistic picture you want. This is both amazing and scary, if you ask me, especially when you look at the results. Watch the video to see more results and understand how the model works! Watch the video References ►Read the full article: ►Nitzan, Y., Aberman, K., He, Q., Liba, O., Yarom, M., Gandelsman, Y., Mosseri, I., Pritch, Y. and Cohen-Or, D., 2022. MyStyle: A Personalized Generative Prior. arXiv preprint arXiv:2203.17272. ►Project link: ►Code (coming soon): ►My Newsletter (A new AI application explained weekly to your emails!): https://www.louisbouchard.ai/mystyle/ https://mystyle-personalized-prior.github.io/ https://mystyle-personalized-prior.github.io/ https://www.louisbouchard.ai/newsletter/ Video Transcript 0:00 this new model by google research and 0:02 tel aviv university is incredible you 0:05 can see it as a very very powerful deep 0:07 fake that can do anything take a hundred 0:10 pictures of any person and you have its 0:12 personnel encoded to fix edit or create 0:15 any realistic picture you want this is 0:18 both amazing and scary if you ask me 0:20 especially when you look at the results 0:23 just take a second to admire them 0:36 [Music] 0:50 the model simply uses a pre-trained 0:52 style gun architecture which i covered 0:54 in numerous videos so i won't enter into 0:56 the detail of this network quickly star 0:58 gun takes an image encodes it using 1:01 convolutional neural networks and is 1:03 trained to regenerate the same image if 1:05 this sounds like another language to you 1:08 just take two minutes to watch the video 1:10 i made covering style gun 1:12 then when you have it well trained with 1:15 a big data set of many people you can 1:17 teach it to transform the image directly 1:20 from the encoded space as i explained in 1:22 my videos so you don't need to fade it 1:24 images anymore you can simply play with 1:27 what we call the generator this means 1:29 you can teach it to change the whole 1:31 picture like a style transfer 1:33 application where you would for example 1:35 take a realistic picture and encode it 1:38 or start right from the encoding and 1:40 transform it into an anime like digital 1:43 image trained and manipulated properly 1:45 you can also change only some local 1:48 features like the color of the hair or 1:50 any other edits to make you look your 1:52 best 1:53 so this new model called my style uses 1:56 the style gun base model and modifies it 1:59 to achieve not only a style transfer 2:01 task but any task that can be associated 2:04 with your face as i said it literally 2:06 learns how you look and can do pretty 2:08 much anything in painting super 2:11 resolution or editing imagine painting 2:13 is where you'd have some object in the 2:15 shot covering your face and you'd remove 2:17 the subject from the picture and make 2:19 your face reappear just like if you 2:22 enable transparency in a video game to 2:24 see through their walls image super 2:27 resolution is an incredibly challenging 2:29 task when trying to generalize to many 2:31 different faces but much easier when you 2:33 focus on one person here the goal is to 2:36 take a very low definition image and 2:38 upscale it to a high resolution one so 2:41 you basically have this a blurry image 2:43 of yourself and you try to make it look 2:46 like this you can see how these two 2:48 applications are quite challenging for a 2:50 machine as it needs to understand the 2:52 person in order to fill in big gaps or 2:55 add pixels to make the face look sharper 2:57 the model basically has to be both a 2:59 very close friend of yours and a great 3:02 artist at the same time as it needs to 3:04 know what your face looks like from any 3:06 angle as well as be able to draw it 3:08 realistically while i will always do the 3:11 most i can to be the best friend 3:13 possible forget about me drawing an 3:15 accurate version of your face if you 3:17 want good results this is just another 3:19 level so taking this style gun basis 3:22 train with a huge general data set of 3:24 thousands of people and a hundred 3:26 pictures of yourself my style will learn 3:29 an encoded space unique to your face it 3:32 will basically find you in the included 3:35 representation of all faces and be 3:37 retrained to push the model to focus on 3:39 your different features then you will be 3:42 able to feed it incomplete or failed 3:44 pictures of yourself and ask it to fix 3:46 it for you how cool is that it requires 3:49 quite a lot of images of yourself but a 3:52 hundred pictures just mean a big day 3:54 outside with a friend and your phones to 3:56 have much better results than the 3:58 general models that try to generalize to 4:00 everyone it's also much cheaper than 4:02 hiring a professional on photoshop and 4:05 asking to edit all your future pictures 4:08 still you can see how this kind of model 4:10 can be dangerous for famous people or 4:12 those with a lot of instagram pictures 4:15 someone could use them to train a model 4:17 and basically create super realistic 4:19 pictures of yourself in compromising 4:21 situations this is why i often say that 4:24 we can't trust what we see anymore 4:26 especially on the internet let's not 4:29 think about all the possible issues when 4:31 it will also be in the real world with 4:33 augmented reality glasses nonetheless 4:36 the results are fantastic and much 4:38 better than what we've seen before 4:40 considering it only requires a hundred 4:43 pictures of yourself instead of hours of 4:45 video shooting for older deep face and 4:48 has much fewer artifacts than those 4:50 requiring fewer images performing only a 4:53 single task and voira this is how my 4:57 style a new model from google research 4:59 and tel aviv university is able to 5:01 perform imaging painting image super 5:04 resolution and image editing using a 5:06 single architecture and training scheme 5:09 compared to other approaches as it 5:11 focuses on the person instead of the 5:13 task itself which makes it much more 5:16 accurate realistic and generalizable to 5:18 you i hope you enjoyed this video let me 5:21 know what you think of this shorter and 5:22 simpler format if you like it or not of 5:25 course this was just an overview of this 5:27 new paper and i strongly recommend 5:29 reading the paper for a better 5:31 understanding of their training scheme 5:33 and the model i will see you next week 5:35 with another amazing paper [Music]

The World's Most Powerful Deepfake Model was Just Released by Google

Too Long; Didn't Read

Louis Bouchard

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...