ChatGPT is a Huge Win for Conversational AI Companies (and Their Customers) — Here's Why

Launched by OpenAI on November 30th, 2022, ChatGPT has already amassed 1 million users as the fastest-growing tech platform of all time, setting the internet on fire with next-level generative AI that’s captivating the imaginations of engineers, linguists, marketers, students, and more.

At its core, ChatGPT is the world’s most advanced general-purpose chatbot, spawned from the large language model (LLM) GPT-3.5. It represents an iteration of InstructGPT, which went live in January 2022 and made similar yet smaller waves in the conversational AI space. GPT-3.5 is a variation of the GPT-3 model, trained on a wide concoction of selected excerpts and codes, to the tune of 175 billion parameters, in late 2021. The results, so far, have been beyond impressive.

ChatGPT can harness the power of Shakespeare to write sonnets, formulate college-level essays from complex text, provide creative comedy and entertainment including obscure rap lyrics, and most notably, field highly sophisticated general knowledge questions – think, Google Assistant meets an automated Quora.

It’s been touted as the AI wrecking ball to writer’s block, and the new search engine extraordinaire. There’s no doubt that ChatGPT’s eye-popping capacity to generate content, and the context retention that unfolds throughout conversations, is unrivaled. But, while the consumer-facing use cases are clear, there remains much to be desired with OpenAI’s latest marvel as it relates to enterprises and B2B, or B2B2C, real-world implementation.

Is ChatGPT Right for Enterprises?

ChatGPT (and/or any other LLM-based chatbot) is one major ingredient in the recipe, not the whole dish. While barriers have certainly been broken in natural language understanding (NLU), ChatGPT’s conversational prowess can only go so far. Statistically accurate isn’t enough – close to correct, for sensitive verticals such as healthcare and government, is a non-starter for enterprise implementation.

The other half of the recipe, for business use cases, will be matching real-time access to specialized organizational knowledge (their proprietary data) with dynamic business logic (their ever-changing internal and external processes), to ensure precision while navigating their users to achieve their goals. Without an additional key ingredient in the mix, using ChatGPT or any other LLM-based chatbot is just not an option– so in the meantime, expect organizations to order different enterprise solutions on the menu.

Enter: existing conversational AI companies.

Michelin Star chefs don’t grow their own tomatoes, but they sure as hell know how to find the best around – similar to how most conversational AI solutions don’t reinvent the wheel with STT (Speech-to-Text) and TTS (Text-to-Speech), instead opting to source the strongest already in existence from Google or Microsoft, so too will conversational AI companies embed GPT-3.5, now the top large language model in the world, within their conversational stack.

Those that will do this seamlessly—while prioritizing security and precision for their customers—will emerge as winners in the new age of generative and conversational AI. This is where the real magic happens for enterprises.

The 5 Key Ingredients for Enterprise Conversational AI

Let’s explore what co-existence between current conversational AI solutions and ChatGPT looks like. First, we’ll cover what business-grade conversational AI requires for a successful deployment. In order to be viable for enterprises, and other “information-heavy” organizations, conversational AI needs to check off 5 key boxes:

1. Real-Time Custom Knowledge — In order to make data-driven decisions, including scheduling an appointment with doctors, or finding technical product information, users demand instantly updated and accurate business and product information, usually captured via APIs. Those datasets and business rules are inherently unique to each individual organization, and beyond understanding “common-sense” knowledge as ChatGPT does so effectively, their dynamic ontologies would ideally need to be absorbed by a knowledge graph. For ChatGPT, access to that domain-specific and even organization-specific information is non-existent – and the information it could give, as noted earlier, is based off of a large language model that was trained on parameters from 2021, often rendering the data irrelevant at best. Auto-updating an organization’s knowledge base, a skill found in only a select few enterprise conversational AI companies, is required for minute-to-minute delivery of digital services.

   
   

   For those organizations who have prioritized customer service use cases and personalization within the scope of their deployments, that adds another layer of missing knowledge – general data needs to be easily integrated with both ongoing conversational context and individual customer preferences logged within, let’s say, a CRM like Salesforce. When using solely LLM’s generic and outdated information, that’s a well-documented obstacle, as AI assistants in action won’t be able to learn or be retrained as new data points and evolving scenarios are encountered.

   
   

2. Explainability — Back in 2021, we covered GPT-3, and as with most neural networks, it represented a black box with which it would be difficult to understand the reasoning behind the actual conversational outputs. GPT-3.5—on which ChatGPT is based—bears the same lack of visibility. Sure, humans building ChatGPT can control the inputs (the data that is ingested) and witness the outputs that are produced, but unfortunately, we can't understand how different variables and parameters are being combined to create those outputs.

ChatGPT, and the LLM it’s based on, is rightfully impressive due to its ability to answer a slew of complex questions correctly, but it also generates a ton of inaccurate responses due to outdated data and the ability to be gamed into providing biased answers. When the assistant fails, there is no accountability – there’s no way of debugging the issue at hand or tracing/pinpointing the source of the inaccurate output. In layman's terms, ChatGPT doesn’t know what it doesn’t know, and, additionally, can’t dispute presumptions proposed by users.

One major reason businesses have begun exploring knowledge graph technologies is to avoid that lack of data transparency and explainability. Any user-facing interface which cannot be iterated and revised is unsustainable or scalable in a business environment, especially when those industries are considered highly sensitive.

This is another aspect in which existing conversational AI solutions are superior to LLMs. Even bottom-shelf DIY chatbots allow their users to alter and improve their conversational flows as needed. In the case of more sophisticated conversational AI interfaces, users not only have a clear snapshot of the error but can track down, diagnose, and remedy the issue instantly.

3. Security and Predictability – Especially in data-sensitive industries, such as healthcare, government, and banking, generic and generative responses pose liabilities and risks unbeknownst to us at present. Ingesting highly sensitive data, like EMRs (Electronic Medical Records) in healthcare, could produce outcomes that jeopardize customers’ trust in the enterprise, and throw the validity of the entire solution into question.

   
   

   Take the simple math example below. 2 is a natural number and integer, right? Not if you bias ChatGPT to think otherwise:

This example shows that:

1. ChatGPT can be manipulated by user prompts (or by biases in the training data the LLM was trained on).

2. ChatGPT can make up completely false answers (regardless of whether the user tried to manipulate it or not) which significantly reduces the trust in it.

3. There’s no way for an organization using ChatGPT to backtrace and understand where the answer came from and adjust it accordingly. (Did I mention it’s a black box?)

Now take much more complex and sensitive use cases, such  as symptom checking or querying the patient portal, which just can’t be left to chance. Currently, security assurances and use cases involving personal identifiable information (PII) that are subject to HIPAA compliance are years away for ChatGPT as a stand-alone solution, unless it’s safeguarded by protocols pre-built into an existing conversational AI platform.

4. Omnichannel Deployment – Chat is one piece of the puzzle, but enterprises have since moved to prioritizing omnichannel platforms, or as Gartner calls it, “establishing the Language-Enabled Enterprise”. Currently, ChatGPT is just that – chat, a single-channel assistant. When it comes to other channels, for example, call centers and smart speakers, tailoring unpredictable, long-winded responses to be repurposed for voice AI is a current impossibility for ChatGPT. OpenAI themselves claim that GPT-3, and by extension GPT-3.5, is optimized for long utterances that aren’t primed for truncation. Speaking from 10+ millions of conversations worth of experience, that’s a death sentence for conversion metrics and efficiency – ChatGPT is simply too inefficiently human-centric and chit-chatty, an unideal choice for goal-oriented tasks.

While originally expanding from chat via the website to voice via call centers, we had to undergo major alterations – optimizing length, adding context, and ensuring that we’d maximally reduced time to objective, or, time to value. Here’s an example of what happens when you don’t adjust:

While that may not seem long textually, the snippet above is actually a 45-second monologue when transferred to pure AI voice generation. That’s poor UX via call centers, and far from the ping-pong-like, human-centric conversational AI that the industry is meant to be migrating towards.

5. Complex Actions — As a text interface, ChatGPT only returns text outputs. In order to combine multiple, intricate business logics, the conversational AI assistant needs to go beyond responsive behavior (reaction) and evolve into actionable behavior – including proactive complex actions (interaction).

For example, when a patient asks to book a doctor’s appointment, a subpar experience would be to exclusively convey that the appointment exists, without next steps. What consumers of top-performing enterprises truly need, and now expect, are proactive actions to be taken to complete their objectives end-to-end, i.e., go forth and schedule that doctor’s appointment, which may include updating multiple databases or APIs.

Integrations, especially those that are vertical-specific, become increasingly important when complex actions are taken into consideration – without them, ChatGPT is unable to carry out true automation at an enterprise level.

The Ultimate Benefits of ChatGPT for Enterprise Conversational AI

1. Context Elasticity – The ability of ChatGPT to keep context, particularly in long sequences of utterances, and hold a human-esque conversation throughout, is an extraordinary accomplishment.

   
   

   Typical chabots and voice assistants are known to break due to the rigidity of conversational flows and predefined intents; with ChatGPT, there’s no context lost throughout the conversation, meaning customers, who are of course naturally prone to human error, can switch directions, change inputs, rephrase, and more, without losing a beat. Incorporating context, and referencing previous parts of the conversation, will become a boon not just for enhanced customer experience, but for upskilled assistance for employee use cases as well.

2. Data Supremacy – ChatGPT represents a major step forward for non-symbolists (see: Yann Lecun) who preach that more data is essential for better models. GPT-3.5 is one such model, which ingests billions of parameters of data and lets the AI create its own rules based on self-drawn conclusions.

   
   

   For example, see enough photos of the ocean and AI can confidently deduce that it is blue. The sheer volume of data allows enterprises to forgo the slow and expensive process of training their own models via machine learning (ML), and goes beyond the boundaries of their own training data limitations. Symbolists (see: Gary Marcus) on the other hand, believe the natural language and conversational AI space has reached some kind of limit dubbed “dumb intelligence”, and in order to advance the space we’d need to do something drastically differnet, such as symbolic AI.

The Path Forward

Combining elements of ChatGPT to improve the conversational experience, while opting for a more controlled, and security-heavy engine run on enterprise data, will be the ultimate path forward as we enter this next wave of conversational and generative AI. When up-to-date, specialized knowledge can seamlessly combine with the world’s most powerful large language model, the possibilities of conversational automation are truly endless.­