AI-driven Sentiment Analysis: Hacking Emotions to Boost Customer Service

“People will forget what you said, people will forget what you did, but people will never forget how you made them feel.”

- Maya Angelou, American poet, and activist

If this quote is accurate, then understanding customers’ sentiment is essential for business success. If a company fails to make a good impression, clients will leave. With research showing that gaining a new customer is six to seven times more expensive than keeping an existing one, understanding people’s sentiment seems like a “cheaper” path to success.

Are you interested in AI-powered sentiment analysis but not sure how to proceed? Do you doubt whether to opt for a ready-made solution or turn to AI software consultants to develop a custom tool? Hopefully, this article will resolve your doubts. If not, get in touch. We will be happy to help.

Introduction to AI sentiment analysis

What is sentiment analysis?

Sentiment analysis (or opinion mining) is the process of identifying emotions expressed in words using artificial intelligence and its subtypes. It follows a predetermined metric to understand how positive, neutral, or negative a piece of text sounds. AI can analyze millions of comments posted on social media, review sites, and online surveys. It can even obtain data from videos. Sentiment analysis enables companies to spot negative attitudes towards their products, empowering them to make a change and address those issues in real-time.

Here is an example of different sentiments identified in customer feedback:

Benefits of sentiment analysis

Understanding sentiment towards your organization is essential to retain customers. According to a study by PwC, one in three people cut ties with a brand after only one bad experience.

Source: Americanscientist.org

Sentiment analysis turns unstructured data into actionable information, which empowers businesses to achieve the following goals:

• Retain present customers and attract new ones
• Improve marketing campaigns by understanding your customers’ reactions and gauging sentiment towards competition’s marketing efforts
• Prioritize customer service issues. Using AI-based sentiment analysis, you can rearrange customer service tickets in the queue to address negative feedback on time
• Understand how clients perceive products. Companies uncover which aspects their clients appreciate the most and what needs improvement
• Track how product modifications impact customers’ attitudes. For example, you can see how customers respond to changes in a product’s interface or adding new features
• Anticipate customer churn. Real-time sentiment analysis enables companies to notice heated discussions on social media and interfere when an angry customer is ready to churn

How AI-based sentiment analysis works?

Earlier, companies used traditional methods, such as surveys and focus groups, to understand customers’ feelings about their products. Employing big data analytics empowers organizations to mine large data volumes, like social media data, to get a more precise picture of clients’ opinions. In its current state, sentiment analysis is a sub-field of natural language processing (NLP).

Sentiment analysis is built on one (or a combination) of these two techniques:

1. Rule-based sentiment analysis: this method uses a dictionary that contains words labeled by sentiment. Here is an example of such a dictionary:

The rules can use NLP techniques, such as tokenization and stemming, to identify words before looking them up in the dictionary. The final sentiment score is combined with additional rules to accommodate negations, dependencies, and other issues in rule-based methods.

Rule-based sentiment analysis is naïve in that it doesn’t consider how words are arranged in a sentence and can’t recognize sarcasm. You can always add new rules, but this might alter the existing ones. This approach requires frequent updates and maintenance. 

2. Machine learning-based sentiment analysis: here, we train an ML model to extract information using labeled datasets. After sufficient training, the algorithm will be able to infer sentiment from new texts. It doesn’t just follow predefined rules, but can also learn to detect sarcasm, synonyms, and other complex cases.

It is also possible to use a hybrid approach that combines rule-based and ML methods into one system. Some sources claim that this technique often produces more accurate results.

Sentiment analysis use cases from 5 industries

1. AI sentiment analysis in tourism and hospitality

The hospitality industry uses artificial intelligence-powered sentiment analysis mainly to process hotel reviews and understand what customers like the most and which aspects can be improved.

Lexalytics, an analytics company based in Massachusetts, analyzed a set of 50,000 customer reviews of ten Best Western locations on TripAdvisor to process their sentiment and experiences. This amounted to 34 pages of comments. Lexalytics cleaned this data by transferring it to a Microsoft Excel sheet and dividing it into four columns:

• A unique ID to identify each review
• Natural language text (the review itself)
• Review title
• Location (which one of the ten Best Western locations the customer reviews)

The team started by visualizing what topics people touched upon in their reviews.

Source: Lexalytics.com

Afterwards, Lexalytics was able to dive deeper into the areas of poor guest experience and offer Best Western recommendations of what they could enhance.

2. AI-driven sentiment analysis in retail

This sector can use AI-powered retail solutions to discover how consumers feel about their brand and identify trends that might affect their market in the future.

One example of extensive AI sentiment analysis in retail comes from Deloitte. The company analyzed over 1.7 million social media comments and other online engagements to understand customer sentiment towards the largest grocery retail chains in South Africa.

The company discovered that the sentiment towards customer service provided by the grocery chains was overly negative, with the main issue being turnaround time.

Source: Deloitte.com

Additionally, Deloitte’s research helped grocery chains understand the upcoming trends, such as how customers feel about innovative technology and sustainability. Some retailers were cautious when it came to incorporating modern technology, such as contactless payment based on near field communication (NFC), into their business. The same goes for offering non-traditional retail services, such as money transfer and accessing governmental sites. Deloitte’s sentiment analysis showed that 76% of comments mentioning such technology had a positive tone.

Furthermore, the report revealed an increasing environmental impact awareness trend among consumers. This is also something retail chains can capitalize on.

3. ML-based sentiment analysis in telecommunications

Telecommunications is another sector where a company’s success depends on its ability to handle customer complaints and requests.

A European mobile network operator wanted to understand how their clients feel when interacting with customer service representatives. The company converted their stored calls into text and used AI sentiment analysis software to calculate sentiment scores and identify instances where customers sustained a negative sentiment below a predetermined threshold. Afterwards, the application would send the struggling customers text messages with apologies and offer them discounts.

Speaking of sentiment analysis in other languages, researchers from Saudi Arabia and the United Kingdom used deep learning sentiment analysis algorithms to determine the level of customer satisfaction with Saudi telecom providers. They analyzed around 20,000 tweets posted in the Arabic language to find who offers the best customer service among the popular telecom companies.

4. AI-driven sentiment analysis in healthcare

Sentiment analysis is gaining popularity among AI health solutions. Medical facilities can use this technique to understand their patients’ feelings about medical appointments, procedures, and costs.

For instance, a research team wanted to gauge patients’ sentiment towards public hospitals in Malaysia and compare it to hospitals’ accreditation.

Researchers used machine learning sentiment analysis algorithms to examine 1,852 Facebook posts and retrieve opinions across five dimensions: hospital’s staff empathy, assurance, responsiveness, reliability, and tangible (equipment used by the staff). The results unveiled a pleasant picture yet showed room for improvement.

Source: Mdpi.com

5. AI-powered sentiment analysis in banking

With the help of AI-powered sentiment analysis, banks can get ideas on improving customer acquisition and better serving existing customers. Additionally, banks can get inspired by analyzing sentiment towards the competition. If a competitor launched a promotional campaign that received positive sentiment, it is worth crafting something similar.

A South African bank headquartered in Johannesburg wanted to gauge customer sentiment to avoid losing clients to the competition. The bank partnered with Repustate, a text analytics company based in Toronto. Together, they extracted and analyzed two million relevant posts from Facebook and Twitter over 90 days.

The results revealed that people mostly complain about poor customer service at particular branches during lunchtime. Given this information, the bank ensured its branches have more functioning tellers during peak hours. Thanks to its timely efforts, the bank witnessed a reduction in customer attrition and even welcomed new customers.

How to choose the right option for sentiment analysis?

Ready to incorporate AI-powered sentiment analysis into your business? Our machine learning expert, Sergey Leyko, highlights three options available for consideration:

• Cloud-based ready-made solutions. Cloud vendors offer tools that they built and trained on datasets of their choice. You can submit your sentiment extraction requests through an API and receive a response, but the vendor will not explain how the tool works under the hood
• Auto ML solutions. An AI software vendor has an algorithm that they train using your dataset. You don’t participate in this algorithm development
• Custom AI-enabled sentiment analysis. You present your requirements to the vendor. They build a model corresponding to your business demands and train it with your dataset

The table below compares the three options based on different aspects arranged in a prioritized list.

As you can see, there are many benefits to using custom AI-powered sentiment analysis. You can build a unique and tailored system to satisfy your complex inquiries and give you a competitive edge over businesses that use standard cloud-based solutions. A custom tool with on-premise processing will offer better data protection and allow you to update the system whenever needed. It will require a higher initial investment, but you will spend less along the way, balancing your expenses and profit.

Who needs to consider custom development options?

We advise businesses with the following characteristics to opt for custom machine learning sentiment analysis solutions:

• When data privacy is of the essence, like in the healthcare sector
• When you would have to submit too many queries and cloud-based solutions would be too expensive
• When you want to process rather complex cases with a custom scoring system, as you will see below

Before you start implementing AI-driven sentiment analysis

1. Formulate your strategy:

Define how you will use the results of the analysis. What business problems do you want to solve, and how do you expect this tool to helpDevelop metrics to measure the solution’s successThink of which languages you wish to include in your analysis

2. Prepare your training dataset (only for auto ML and custom AI-driven sentiment analysis solutions):

Gather as much data as you canClean the data. If you use any special symbols, eliminate themAnnotate your training set, if possible

3. Identify the desired capabilities for your algorithm:

• The level of detail

Is it enough for you to know a general sentiment (positive, neutral, negative) or do you want to dig deeper and understand what exactly your customers enjoy or despise in your product? For example, does it suffice to see that someone wasn’t happy with their stay at a hotel? Or do you want to know what exactly went wrong, like “the receptionist was rude” and “there was no coffee at breakfast?”

• Different ways of saying the same thing

People naturally use different words to refer to the same thing. Coming back to the hotel example, to express the fact that the hotel room was dirty, one customer can say “the room was filthy,” while another would use different terminology, saying, “the room looked like a dump.” So, there are not only synonyms of a particular word but fully different ways of expressing the same sentiment.

• Typos

Many people make typos in their writing. How do you want your algorithm to handle that?

• Sarcasm

Sarcastic sentences express negative sentiment using positive words. In the example below, a positive tone is used, but the sentiment towards the laptop bag is clearly negative. “This is the best laptop bag ever. It is so good that within two months of use, it is worthy of being used as a grocery bag.”

• The way you assign scores

How do you want to calculate the final sentiment score? Take the following restaurant review as an example,

“Tuna salad was fresh and delicious, but the dining room was dark and tiny.”

The sentiment towards the salad is positive, but it is negative towards the dining space. What is the final score here? Is it neutral (0), as negative cancels out the positive? Or do you want your algorithm to break this sentence down into two categories, food and space, and score it (+1) for food and (-1) for space separately?

On a final note

Customer sentiment analysis has many benefits. It can help you gauge what your customers are feeling towards your product, thereby allowing you to improve your offering. You can also analyze the sentiment towards your competition to repeat their success and avoid their mistakes. However, incorporating AI-based sentiment analysis into your business is a challenge.

We at ITRex can assist you in various ways. If you opt for cloud-based sentiment analysis, we can help you adapt your system to the cloud vendor’s API. If you choose the auto ML option, we can assist you with dataset preparation and training, and with the vendor’s API integration. We will be of at most help if you decide to build a custom AI sentiment analysis tool.

Our team will allocate ML engineers and work with you on data collection, cleaning, and annotation (if needed). We will build and train a model based on your own datasets and corresponding to your specific needs. Furthermore, we will develop an API and integrate it into your back end seamlessly. Finally, you can count on our support in case of malfunctioning, the need to scale, or incorporating updates.

Want to understand what customers feel towards you and your competitors? Drop ITRex developers a line! Their AI experts will help you build and train an algorithm corresponding to your needs.

Also published here.