Google recently announced a new model for automatically generating summaries using machine learning, released in Google Docs that you can already use. The model will try to understand the whole document and generate a short summary of the piece—something some movie professionals clearly still can’t do. The model needs to achieve two things to achieve that, which you will learn in the video below! Watch the video References ►Read the full article: ►Google's blog post: ►GPT-3 video: ►Attention video: ►What are RNNs?: ►My Newsletter (A new AI application explained weekly to your emails!): https://www.louisbouchard.ai/google-docs-summary/ https://ai.googleblog.com/2022/03/auto-generated-summaries-in-google-docs.html https://youtu.be/gDDnTZchKec https://youtu.be/QcCJJOLCeJQ https://youtu.be/Z0pb3LjeIZg https://www.louisbouchard.ai/newsletter/ Video Transcript 0:00 do you find it hard to quickly summarize 0:02 a movie you just watched or a book you 0:04 read a few weeks ago sometimes you love 0:06 a book and if you can't manage to 0:09 remember its content which i often can't 0:11 you may end up boring your friend for 0:12 talking for an hour describing the many 0:15 chapters and important parts while your 0:17 friend just want to have a quick and 0:18 concise summary this is because doing a 0:21 great summary is challenging even for us 0:23 but necessary how useful it is to be 0:26 able to quickly know what the book is 0:28 about before buying it or simply to help 0:30 you go through all your emails and 0:32 documents in seconds taking the time to 0:34 recap and summarize your work is also 0:36 very important if you want to be taken 0:38 seriously at your job and weights and 0:41 biases the sponsor of this video will 0:43 help you with that with my favorite 0:45 feature weights and biases reports it 0:48 helps you create beautiful reports about 0:50 your previous runs with your hyper 0:51 parameters and any matrix tracked in 0:54 seconds it even allows you to create 0:56 clear and dynamic plots comparing the 0:58 runs your team members can leave 1:00 comments right on the report if this 1:02 doesn't make you the coolest ml engineer 1:05 on your team i don't know what will your 1:07 teammates will understand what you are 1:09 working on in a split second it will 1:11 allow them to help you out if they can 1:13 saving valuable time and possible 1:15 misunderstandings try out weights and 1:17 biases for free with the first link 1:19 below and profit from these benefits by 1:22 stepping up your engineering game 1:27 just like for creating clear reports you 1:29 need to have a great understanding of a 1:32 book movie or any content you are trying 1:34 to summarize to do it well omit all 1:36 unnecessary information while keeping 1:39 the essential making something as 1:41 concise as possible can be really 1:42 complicated or even impossible here i 1:46 try to explain research in a few minutes 1:48 and i often can't manage to make it 1:49 shorter than 5 minutes even if it's only 1:52 a summary of a 20 page piece it requires 1:55 hours of work and fine tuning and now i 1:57 may be replaced by an ai that does that 2:00 better in milliseconds indeed google 2:03 recently announced a new model for 2:05 automatically generating summaries using 2:07 machine learning released in google docs 2:09 that you can already use the model will 2:12 try to understand the whole document and 2:14 generate a short summary of the piece 2:16 something some movie professionals 2:18 clearly still can't do the model needs 2:21 to achieve two things understand the 2:23 text in the document called natural 2:25 language understanding and generate 2:28 coherent sentences using a natural 2:30 language or in other words perform 2:33 natural language generation but how can 2:36 you achieve that you guessed it with a 2:39 lot of data and compute power luckily 2:41 enough this is google research they 2:44 trained our model to replicate our 2:46 thought process for generating summaries 2:48 using way too many documents with 2:50 manually generated summaries seeing all 2:52 these examples the model does like any 2:55 good student and ends up being able to 2:57 generate relatively good summaries for 2:59 similar documents as it has seen during 3:02 its training phase you can see why we 3:04 need good quality data here the model 3:06 will learn from them it may only be as 3:09 good as the data that was used for 3:11 training it it will be like having a 3:13 really bad coach that doesn't know 3:15 anything about basketball trying to 3:17 teach a new player how could this new 3:19 player become any good if the coach 3:21 doesn't know anything about the sport 3:24 the newcomer's talent won't be optimized 3:26 and might be wasted only because of the 3:28 poor coaching the challenge comes with 3:31 generalizing to new documents 3:33 generalizing is something even difficult 3:35 for students that only learned how to 3:37 perform the given examples but did not 3:40 understand how to apply the formulas 3:42 it's the same thing here the model faces 3:45 difficulties as it cannot remember all 3:47 documents and summaries by heart it has 3:50 to understand them or at least know 3:52 which words to put its attention on in 3:55 order to produce a summary that reflects 3:57 the document well the latter will most 4:00 likely happen as the model doesn't 4:02 really understand the document it only 4:04 understands how to perform the task 4:06 which is unfortunately still far from 4:09 human level intelligence but well enough 4:11 for such a task i just mentioned 4:13 attention well this was not a 4:16 coincidence attention may be the most 4:18 important concept behind this model 4:21 indeed just like gpt3 this new model 4:24 also uses the transformer architecture 4:26 and attention mechanisms this is where 4:28 high computation is required as you know 4:31 transformers are big and powerful 4:34 networks but most time a bit too big for 4:36 fast and efficient tools that need to be 4:38 available online in seconds transformers 4:41 computation complexity also scales with 4:43 the input size which means that the 4:45 longer the input the heavier the 4:47 computation will be causing big issues 4:50 when we want to summarize a whole book 4:52 gpt3 works well for small inputs like 4:55 question and sharing tasks but the same 4:57 architecture won't be able to process 4:59 whole books efficiently instead they had 5:01 to use some tricks in order to have a 5:03 smaller and more efficient model while 5:06 keeping high quality results this 5:08 optimization was achieved by merging 5:10 transformers with rnns or recurrent 5:13 neural networks which are two concepts i 5:16 explained in previous videos that i 5:18 highly recommend watching for a better 5:19 understanding both videos are linked in 5:22 the description below in short it will 5:24 act similar to gpt3 which you should 5:26 understand by now from my video about it 5:29 but with a smaller version of the model 5:31 iterating over and over until the model 5:34 finishes the book the transformer part 5:37 of the architecture will be responsible 5:39 for understanding a small section of the 5:41 text and producing an encoded version of 5:44 it the rnn will be responsible for 5:46 stacking and keeping this knowledge in 5:48 memory iterating through the whole book 5:51 to end up with the most concise way of 5:53 summarizing its content working together 5:56 the attention mechanism added to the 5:58 recurrent architecture will be able to 6:00 go through long documents and find the 6:02 most important features to mention in 6:04 the summary as any human will do of 6:07 course the model is not perfect since 6:09 even professional writers aren't perfect 6:11 at summarizing their work but the 6:13 results are quite impressive and 6:15 produced extremely efficiently i'd 6:17 strongly recommend trying it for 6:19 yourself in google docs to make up your 6:21 mind about it 6:22 and voila this is how google docs 6:25 automatically summarize your documents 6:27 with their new machine learning based 6:29 model i hope you enjoyed this video if 6:32 so please take a second to leave a like 6:34 and comment your thoughts on the video 6:36 and on this new model will you use it 6:39 thank you for watching until the end and 6:41 i will see you next week with another amazing paper
