NLP Tutorial: Topic Modeling in Python with BerTopic

August 24th 2021 1,751 reads

BerTopic is a topic modeling technique that uses transformers (BERT embeddings) and class-based TF-IDF to create dense clusters. It also allows you to easily interpret and visualize the topics generated. In this NLP tutorial, we will use Olympic Tokyo 2020 Tweets with a goal to create a model that can automatically categorize the tweets by their topics. The BerTopic algorithm contains 3 stages:Embed the textual data(documents) Embed the documents with BERT, or it can use any other embedding technique. The algorithm uses UMAP to reduce the dimensionality of embeddeddings and the HDBSCAN technique.

@davisdavid

Davis David

Data Scientist | AI Practitioner | Software Developer. Giving talks, teaching, writing.

NEWABOUT PAGE

Topic modeling is an unsupervised machine learning technique thaat automatically identifies different topics present in a document (textual data). Data has become a key asset/tool to run many businesses around the world. With topic modeling, you can collect unstructured datasets, analyzing the documents, and obtain the relevant and desired information that can assist you in making a better decision.

There are different techniques to perform topic modeling (such as LDA) but, in this NLP tutorial, you will learn how to use the BerTopic technique developed by Maarten Grootendorst.

Table of Contents:

1. What is BerTopic
2. How to Install BerTopic
3. Load Olympic Tokyo Tweets Data
4. Create BerTopic Model
5. Select Top Topics
6. Select One Topic
7. Topic Modeling Visualization
8. Topic Reduction
9. Make Prediction 
10. Save and Load Model

What is BerTopic?

BerTopic is a topic modeling technique that uses transformers (BERT embeddings) and class-based TF-IDF to create dense clusters. It also allows you to easily interpret and visualize the topics generated.

The BerTopic algorithm contains 3 stages:

1.Embed the textual data(documents)
In this step, the algorithm extracts document embeddings with BERT, or it can use any other embedding technique.

By default, it uses the following sentence transformers

• "paraphrase-MiniLM-L6-v2"- This is an English BERT-based model trained specifically for semantic similarity tasks. 
• "paraphrase-multilingual-MiniLM-L12-v2"- This is similar to the first, with one major difference is that the xlm models work for 50+ languages.

2.Cluster Documents
It uses UMAP to reduce the dimensionality of embeddings and the HDBSCAN technique to cluster reduced embeddings and create clusters of semantically similar documents.

3.Create a topic representation
The last step is to extract and reduce topics with class-based TF-IDF and then improve the coherence of words with Maximal Marginal Relevance.

How to Install BerTopic

You can install the package via pip:

pip install bertopic

If you are interested in the visualization options, you need to install them as follows.

pip install bertopic[visualization]

BerTopic supports different transformers and language backends that you can use to create a model. You can install one according to the options available below.

• pip install bertopic[flair]
• pip install bertopic[gensim]
• pip install bertopic[spacy]
• pip install bertopic[use]

The Libraries 

We will use the following libraries that will help us to load data and create a model from BerTopic.

#import packages

import pandas as pd 
import numpy as np
from bertopic import BERTopic

Step 1. Load Data

In this NLP tutorial, we will use Olympic Tokyo 2020 Tweets with a goal to create a model that can automatically categorize the tweets by their topics.

You can download the datasets here.

#load data 
import pandas as pd 
 
df = pd.read_csv("/content/drive/MyDrive/Colab Notebooks/data/tokyo_2020_tweets.csv", engine='python')
 
# select only 6000 tweets 
df = df[0:6000]

NB: We selected only 6,000 tweets for computational reasons.

Step 2. Create Model

To create a model using BERTopic, you need to load the tweets as a list and then pass it to the fit_transform method. This method will do the following: 

• Fit the model on the collection of tweets. 
• Generate topics.
•  Return the tweets with the topics.

# create model 
 
model = BERTopic(verbose=True)
 
#convert to list 
docs = df.text.to_list()
 
topics, probabilities = model.fit_transform(docs)

Step 3. Select Top Topics

After training the model, you can access the size of topics in descending order.

model.get_topic_freq().head(11)

Note:  Topic -1 is the largest and it refers to outliers tweets that do not assign to any topics generated. In this case, we will ignore Topic -1.

Step 4. Select One Topic

You can select a specific topic and get the top n words for that topic and their c-TF-IDF scores.

model.get_topic(6)

For this selected topic, common words are Sweden, goal,rolfo, swedes, goals, soccer. It is obvious this topic focuses on “soccer for Sweden team”.

Step 5:Topic Modeling Visualization

BerTopic allows you to visualize the topics that were generated in a way very similar to LDAvis. This will allow you to get more insights into the topic's quality. In this article, we will look at three methods to visualize the topics.

Visualize Topics 

The visualize_topics method can help you visualize topics generated with their sizes and corresponding words. The visualization is inspired by LDavis.

model.visualize_topics()

Visualize Terms

The visualize_barchart method will show the selected terms for a few topics by creating bar charts out of the c-TF-IDF scores. You can then compare topic representations to each other and gain more insights from the topic generated.

model.visualize_barchart()

In the above graph, you can see Top words in Topic 4 are proud, thank, cheer4india, cheering, and congrats.

Visualize Topic Similarity 

You can also visualize how similar certain topics are to each other. To visualize the heatmap, simply call.

model.visualize_heatmap()

In the above graph, you can see that topic 93 is similar to topic 102 with a similarity score of 0.933.

Topic Reduction

Sometimes you may end up with too many topics or too few topics generated, BerTopic gives you an option to control this behavior in different ways. 

(a) You can set the number of topics you want by setting the argument "nr_topics" with a number of topics you want. The BerTopic will find similar topics and merge them.

model = BERTopic(nr_topics=20) 

In the above code, the number of topics that will be generated is 20.

(b)Another option is to reduce the number of topics automatically. To use this option, you need to set "nr_topics" to "auto" before training the model.

model = BERTopic(nr_topics="auto")

(c) The last option is to reduce the number of topics after training the model. This is a great option if retraining the model will take many hours.

new_topics, new_probs = model.reduce_topics(docs, topics, probabilities, nr_topics=15)

In the above example, you reduce the number of topics to 15 after training the model.

Step 6:Make Prediction 

To predict a topic of a new document, you need to add a new instance(s) on the transform method.

topics, probs = model.transform(new_docs)

Step 7:Save Model

You can save a trained model by using the save method.

model.save("my_topics_model")

Step 8:Load Model

You can load the model by using the load method.

BerTopic_model = BERTopic.load("my_topics_model")

Final Thoughts on Topic Modeling in Python with BerTopic

In this NLP tutorial, you have learned

• How to create a BerTopic Model.
• Select topics generated.
• Visualize topics and words per topic to gain more insights.
• Different techniques to reduce the number of topics generated.
• How to make predictions.
• How to save and load BerTopic Model.

BerTopic has a lot of features to offer when creating the model. For example, if you have a dataset for a specific language(by default, it supports the English model) you can choose the language by setting the language parameter while configuring the model.

model = BERTopic(language="German")

Note: Select a language in which its embedding model exists.

If you have a mixture of languages in your documents, you can set

language="multilingual"

If you learned something new or enjoyed reading this article, please share it so that others can see it. Until then, see you in the next post!

You can also find me on Twitter @Davis_McDavid.

And you can read more articles like this here.

Want to keep up to date with all the latest in python? Subscribe to our newsletter in the footer below.

The Cosmos HackAtom is here! Prizes worth $1,000,000 up for grabs.

Join Hacker Noon