Since their inception, search engines have gone from basic search agents to sophisticated algorithms based on artificial intelligence (AI) and machine learning (ML). These innovative technologies affect the Search Engine Optimization (SEO) space from two completely opposite perspectives.
On the one hand, it has become much more challenging to promote websites and push them to the top of SERP due to new AI-based ranking algorithms able to perform a very in-depth scan beyond meta. On the other hand, as the overall quality of search results has improved significantly, it is more difficult now to manipulate them using different kludges and black hat practices (albeit still possible which I'll show you below).
All in all, artificial intelligence has fundamentally changed the approach to SEO. Let's dive deep into how AI is used in search engine marketing and how tech-savvy marketers use it to better meet their goals and improve crucial performance indicators.
What's Referred to as "AI Technologies"?
AI technologies are categorized in terms of their ability to mimic human behavior and capabilities. Using these characteristics, all AI technologies – both existing and hypothetical ones – can be divided into three types:
1) Artificial Narrow Intelligence (ANI), or weak AI.
It provides a narrow range of abilities. These systems can only be trained to perform specific tasks. Examples are Google's Rankbrain, Apple's Siri, or Amazon's Alexa.
2) Artificial General Intelligence (AGI), or strong AI.
It mirrors human capabilities, is versatile, capable of solving many problems, and learning from experience.
3) Artificial Super Intelligence (ASI), or hypothetical AI.
It's supposed to surpass the human intellectual ability.
ANI is the only type of AI that has been successfully implemented by humans so far.
Why is Machine Learning Needed?
Machine learning is an application of AI that can automatically learn and improve from the experience without being explicitly programmed to do so. ML occurs as a result of the analysis of ever-growing amounts of data, so the underlying algorithms do not change, but the internal weights and biases of the code used to select a specific answer change. Of course, it's not that simple.
Data scientists often refer to the technologies used to implemen ML as algorithms. An algorithm is a series of step-by-step operations, usually computations, that solve a specific problem in a finite number of steps. In machine learning, algorithms use a series of finite steps to solve a problem by learning from data.
Although ML algorithms learn, it is often difficult to find the exact meaning of the term "learning" because there are different ways to extract information from data, depending on how the ML algorithm is built. Typically, the learning process requires huge amounts of data that provide the expected response given a certain input. Each input / output pair is an example, and additional examples make the algorithm easier to learn. This is because each input / output pair corresponds to a row, cluster, or other statistical view that defines the problem area.
ML is the process of optimizing a model, which is a mathematical generalized representation of the data itself, allowing it to predict or otherwise determine the appropriate response, even when it receives input that it has not seen before. The more accurately the model can provide correct answers, the better the model learns from the input provided. The algorithm fits the model to the data, and this fitting process is learning.
How ML Works
The central idea of machine learning is that you can represent reality with a mathematical function that the algorithm does not know in advance, but which it can guess after looking at some data (always in the form of paired inputs and outputs). You can express reality and all its complexity in terms of the unknown math functions that machine learning algorithms find and make available as a modification of their internal math function. That is, each machine learning algorithm is built on a modifiable mathematical function.
Depending on the expected result and the type of input data, you can classify the algorithms according to the learning style. The style you choose depends on the type of data you have at your disposal and the expected result.
Four learning styles are used to create algorithms:
Supervised learning – algorithms try to model the relationships and dependencies between the target prediction output and the input functions so that we can predict the outputs for new data based on the relationships it has learned from previous datasets.
Unsupervised learning – the computer is trained with unlabelled data. The computer can teach you something new after it learns the patterns in the data. These algorithms are especially useful in cases where we don't know what to look for in the data.
Semi-supervised learning – in many practical situations, the cost of labeling is quite high, as it requires skilled human professionals. Thus, in the absence of labels, semi-guided algorithms are the best candidates for building a model. These methods take advantage of the idea that even if the membership of unlabelled data groups is unknown, the data carries important information about the group's parameters.
Reinforced learning – this method uses the observations collected during interaction with the environment to take actions that will maximize reward or minimize risk. A reinforcement learning algorithm (called an agent) continuously and iteratively learns from the environment. In the process, the agent learns from his experiences in the environment until it explores the full range of possible states.
That being said, AI bypasses the classical approach's error by allowing the system to identify patterns and learn implicit rules by analyzing thousands of examples (images, sound files, texts, etc.) according to certain concepts (as was the case with the cat example).
Every day, the amount of information we have to process increases exponentially. So does the pressure on our emotional state. Therefore, machine learning has become necessary for humanity to automate routine work, save time, and increase productivity.
How is Artificial Intelligence Used in Search Engine Algorithms?
Now when we have figured out a little about how AI algorithms work and why they're needed in general, let's move on to SEO and how it leverages AI technologies.
Advances in machine learning are driving the development of AI-based SEO. Although this space has been explored since 2003, the first major achievement happened ten years later, in 2013, with the launch of Word2vec, a "natural language processing (NLP) technique that uses a neural network model to learn word associations from a large corpus of text."
Two years later, in 2015, Google used the Word2vec database to build and launch RankBrain as part of the Hummingbird algorithm.
RankBrain is an AI-powered self-learning system that has enabled Google to accelerate keyword category validation in order to provide users with the most relevant content for their search query. RankBrain "knows" how to understand the meaning of a text, find connections between words, learn unfamiliar words and phrases, and adapt specifically to the country and language of the request.
All of this helped make organic search results even more relevant.
Google representatives point out that this algorithm is the third important factor in modern search ranking, along with content quality and links.
Well, the cherry on the cake was the Google BERT algorithm released in 2019.
BERT (Bidirectional Encoder Representations from Transformers) is also an NLP learning system based on a neural network. Unlike other models, BERT is designed for a deep understanding of natural speech.
In other words, BERT should allow the bots to understand what the words in the sentence mean, given every detail of the context. Google uses BERT to better understand user queries and provide them with genuinely relevant results.
Examples of Using AI in SEO
Content Creation
AI is already used extensively to create content. Some content and SEO professionals use OpenAI GPT-2 model for this purpose.
GPT-2
GPT-2 is a large transformer-based language model with 1.5 billion parameters, trained on a dataset of 8 million web pages with a simple objective of predicting the next word to match the context.
Image source: GitHub
They say that the texts written by this transformer are almost no different from the texts written by a person. I've decided to double-check it.
As a content marketer, one of my goals is to increase my employer's brand awareness and thought leadership and generate word of mouth through guest and ghost publications in top-tier and fringe media. For this objective, I've found a great UK-based media outlet to submit my guest publication.
However, every submitted article is read by real human editors. If they find no value in the content, they won't publish it.
I've used this transformer to create an article and submit it to the editor for approval. To my surprise, the editors accepted it and didn't understand that a bot wrote the text.
In general, you can safely apply the GPT-2 model to create articles and comments in different languages.
GPT-2 generated text example
How to work with GPT-2 model
Go to https://inferkit.com that hosts the working GPT-2 model. Find the source of the text you need. Copy a small (two to three sentences) piece of text, paste it into the form and click the "Complete Text" button. GPT-2 will create three to five paragraphs of text. If the result created with the help of artificial intelligence did not suit you, click on the "Complete Text" button again.
If the generated text meets your expectations, copy it. Then paste the last paragraph, written by GPT-2, into the transformer form and click the "Complete Text" button again. GPT-2 will continue writing your article.
GPT-3
OpenAI has recently released a third-generation open source language prediction model GPT-3, which allows computers to generate random sentences of approximately the same length and grammatical structure as the sample ones.
In his early experiments with GPT-3, a Github user Manuel Araoz found that the predicted GPT-3 proposals, when they were posted on the bitcointalk.org forum, attracted a lot of positive attention from fellow forum participants, including suggestions that the system must be smart (and / or sarcastic ) and that he found subtle patterns in their messages. He believes similar results can be obtained by republishing the GPT-3 results on other message boards, blogs, and social networks.
Every day in May he posted to bitcointalk.org one interesting technical post generated entirely by GPT-3 model. While users interacted with his posts, GPT-3 model created replies and even predicted to the next comments.
According to Araoz, whenever he posts to the forum as himself, people frequently mention that they think he must be a “bot” to be able to post so quickly, be so accurate, and / or say the same thing as someone else.
That experiment made him believe that GPT-3 was one of the major technological advancements he's seen so far.
How to use GPT-3 in SEO
If content marketing generates 50% or more of your business results, it might be worth expanding your skill set to become a more AI-savvy marketer.
You can use GPT-3 models for the following tasks:
- Create short-form content at scale (e.g., footer content on product catalogue pages, tooltips, etc);
- Create meta descriptions, ALT-tags for old content, and other missing SEO elements
- Create evergreen content – if your industry has evergreen topics that are written about consistently (e.g. definition questions, standard processes, etc.), GPT-3 can generate a short outline and provide a working draft to elaborate on and complete.
Routine Tasks Elimination
AI can be used to minimize routine processes by teaching a machine using pre-built models. In my practice, I have come across the following SEO tasks that were completely or partially automated with AI.
- Semantics clustering;
- Selection of performers;
- Classification of types of requests by determining the type of issue and a relevant landing page;
- Classification of site pages based on screenshots;
- Automation of backlinks acquisition,
- Detecting and fixing thin content such as doorway pages, low-quality affiliate pages, or simply pages with very little or no content;
- Content plan analysis.
Also, using unique algorithms, content marketers can create and analyze their content plans more efficiently.
Some time ago, I asked my former employer's tech team to build an ML algorithm that would allow our marketing team to "filter out" articles for publication in our owned, paid, and earned media. This allowed us to predict with precision which topic will best match Google ranking factors, which article will become evergreen, or which item has the best potential of getting Google's featured snippet.
Voice Search
One of the new skills to be found in any modern SEO professional's inventory is understanding how to optimize content for voice queries. Voice search is becoming increasingly popular thanks to the rise in popularity of AI virtual assistants such as Alexa, Siri, Cortana, and others.
In fact, 35% of internet users have already used their virtual assistants to make a purchase, and Gartner predicts that 30% of all web browsing sessions will be done off the screen during 2021. People use voice search to communicate with their favorite brands and search for products and businesses on the Internet.
Image source: dialogtech.com
If you want your brand to stay competitive or if you need to improve the performance of your campaigns, you need to keep up with this trend and optimize your content for voice search. To satisfy the algorithms and get high rankings, you should use the same tools and tactics used by search engines. This is why tools like Moz or Yoast can be very helpful when it comes to making content more accessible to search engines and voice search queries.
There are a great many tasks where you can apply AI; it all depends on what you want to do and how much data you need to process regularly. There is always a question of profitability.
Since AI, in principle, has not yet been created, we can only work with weak or narrowly targeted "analogs", such as gradient boosting over decision trees.
There are a lot of examples of using neural networks, namely:
- recommendation systems based on user behavior;
- algorithms for determining LSI words in search engines;
- chatbots performing linguistic analysis, etc.
SEO Results Manipulation With Deepfake Technology
Deepfakes are synthetic media in which a person in an existing image or video is replaced by someone else's. While content counterfeiting is not new, deepfakes use powerful machine learning and artificial intelligence techniques to manipulate or create visual and audio content with a high potential for deception.
SEO-savvy propagandists can spread disinformation and spoil search results in many ways, including:
Ambiguation – using AI bots to deliberately flood the Internet with the wrong addresses or phone numbers for a competitor's location;
Google Bombing (a.k.a. Googlewashing) – the practice of causing a website to rank highly on SERP for irrelevant, unrelated, or off-topic search terms by heavy linking;
302 hijacking - using AI bots to configure a temporary redirect from one site to another, which allows the redirect page to start ranking for the landing page's keywords.
For example, after the March nerve agent attack in the UK and the April chemical weapons attack in Syria, articles by RT and Sputnik, the Russian government's propaganda agencies, appeared on the first page of Google searches. Likewise, YouTube (owned by Google) has an algorithm that prioritizes the time users spend watching content as a key metric for determining what content appears first in search results.
This algorithmic preference leads to false, extremist, and unreliable information at the top, which means that this content is viewed more often and perceived by users as more reliable.
"The revenue from the SEO manipulation industry is estimated at billions of dollars." Brookings.Edu
AI-Powered SEO Tools
If you're an SEO specialist looking to tap into the new opportunities provided by AI, you should start familiarizing yourself with useful tools now.
Clearscope is an AI-based platform for content optimization. The tool is paid. For any keyword, the tool analyzes the top-performing organic content and gives you a breakdown of all relevant terms in order of importance using Watson AI.
Another SEO startup using AI technologies is frase.io. Like Clearscope, its main job is to gather information for your content and optimize it. The service is paid, but there is a free option for testing.
BigML is a machine learning tool with a free subscription plan. I use it for all sorts of research. All in all, I highly recommend it. You will need three things:
- a clear understanding of what you are doing and what you want to get,
- a dataset, and
- basic knowledge of machine learning (you can read the BigML blog for this).
Diib is an AI-based tool that "instantly syncs with Google Analytics and uses advanced algorithms and AI to give you an easy-to-use traffic growth plan." It only takes 60 seconds to scan your website in 60 seconds and generate guidance for improving your SEO, page load speed, security, and user experience in general. Their PRO subscription costs $29,99 and includes a full-fledged growth plan, social media analytics, support, and other advanced features. However, you can try it for free with limited functionality.
Unlike other tools that show dashboards and graphs, Diib talks to you in plain language and explains descriptively all steps you need to make to grow organic traffic and improve other crucial SEO KPIs.
Tips for Optimizing Your Site With AI in Mind
1) Use GPT-2 and GPT-3 models to create high-quality search engine optimized content (both short-form for social media and meta and long-form and evergreen for long-term strategic results).
2) Use Google's Cloud Vision to optimize images, detect emotion, understand the text, and more.
3) Pay attention to the quality of each page's content, the logic of your storytelling, and the context in which words are used. At the moment, checking for compliance with the context is the most difficult and time-consuming task because there are very few tools that use the national word corpora for this.
One of the best yet most expensive solutions for this is sketchengine. It uses the word corpora derived from Wikipedia texts. By the way, Google BERT was also trained on Wikipedia texts.
4) Move from optimizing pages for keywords (including in-text search queries and their density) to optimizing the essence of your content, namely:
- its completeness,
- richness of phrases,
- Latent Semantic Indexing (LSI),
- thematic keywords that set the topic,
- highlights in the search results (e.g., featured snippets), and so on.
Final Thoughts
The topic of using artificial intelligence in SEO is still quite controversial, and there are a lot of different opinions on this.
However, it makes no sense to deny that AI-based technologies impact the search optimization space. New search engine algorithms and tools are released that specialists can adapt to automate keyword research and content writing processes, streamline and improve backlinks profile, and user experience in general. On the other hand, AI technologies such as deepfake can significantly decrease the quality of your SEO and manipulate the search results by using unfair competitive practices and spreading disinformation about your brand.
Whether you're a white or gray-hat SEO expert (black-hat SEO guys are typically early technology adopters and first to try new things) – if you don't yet leverage AI – consider doing it as soon as possible to stay competitive, make informed decisions, and delight your target audiences with relevant, helpful, and high-quality search results.
Are you a startup working at the intersection of AI and marketing? Feel free to reach out to me on LinkedIn to pitch your story. I'll be happy to feature it in my next article.