GPT models weren’t widely popular until November 2022, once OpenAI built a successor of their language models called ChatGPT. As of March 2023, ChatGPT is now officially released with an API and SDK to integrate the model into your products. While there are a plethora of applications on the surface, ChatGPT offers a wide range of improvements for different types of technical and business processes. So here we’ll put to the test the most viable, and see where it gets us.

I’ll share with you my vision on how we can benefit from GPT models on different levels of automation, while providing you with a concise answer of the question, “What do I need to implement a GPT model in my product?”.

Please note: all the tests were made during the pre-release stage.

What is ChatGPT and What are GPT Models in General?

If you already know what ChatGPT and other GPT models are, you can skip this part. Because here, I’ll define ChatGPT from different angles, along with how to exploit its capabilities to dig for data and provide us with necessary information. The easiest way to do this is by simply asking the model, and here is how ChatGPT identifies itself:

ChatGPT is the latest GPT model that was developed on the basis of GPT 3.5. It has the ability to interact with the user in the form of a dialogue and provide answers to almost any question based on events and information up until 2021. GPT models are highly capable of solving a range of Natural Language Processing (NLP) tasks with the possibility of customization such as:

• Keywords extraction
• Text translation
• Question answering
• Text generation
• Text summary
• Spelling correction
• Text style transferring
• Named entity recognition

How to Use ChatGPT in Your App or Website

As of March 2023, ChatGPT is available via API and some companies like Snapchat are already using it. So it makes sense to discuss how ChatGPT can be integrated in your application for a technical standpoint. There is a certain fork of options here:

1. THE TASK IS VIABLE BY MEANS OF THE EXISTING MODEL.

Here, the model operates as a smart catalog, helping the user to browse through content and make choices more quickly using the response of GPT. However, customizations and personalization are not available.

2. FINE-TUNING THE MODEL WITH YOUR OWN DATA.

Such models can be useful for creating a personalized chatbot recommender system based on user actions/desires and other use cases. Using your own data, the GPT model can be trained to introduce the element of personalization. The amount of data needed to fine tune a GPT model depends on various factors, such as the size and complexity of the data set, and the desired performance of the model. Typically, at least a few hundred examples are required to effectively fine-tune a GPT model. However, in complex domains, it may take tens of thousands or even millions of examples to achieve optimal model performance.

To understand whether you’ll need to fine-tune the model, I would recommend you running through these 5 steps:

1. Business understanding that involves learning the business case, what task is delegated to the model, what data it should use, and the personalization features that are expected.
2. Data gathering. This step can be skipped if personalized information can be skipped at the early stages. But if you plan to use the GPT model in a highly specific domain, by default, the model will fail to operate correctly. Which means, this stage is necessary. For example, it is necessary to recommend clothes for the user based on what is available in the store. Then it is necessary to collect information about the products of the store and provide basic instructions for recommendations (for example, jeans with the article YYYY should be worn over a T-shirt with such XXXX).
3. Data annotation/Data labeling. This stage will include creating a prompt for each sample from the data gathering stage. For example: shorts with YYYY articles must be worn over a T-shirt with XXXX article = what should I wear under a T-shirt with XXX article? And the expected answer: jeans with YYYY article.
4. Data Preparation. This implies converting the data into a suitable format, e.g. “{“prompt”: “<what should I wear under a T-shirt with article XXX?>”, “completion”: “<Shorts with article YYYY>”}”.
5. Modeling. Finally, applying those changes and implementing the model in production.

Keep in mind that the GPT model is not a complete product that covers all the domain areas out of the box. You’ll also need the application for it and management tools on the backend. Then, the output of a GPT model has to be processed with NLP methods in order to extract meaningful information. The model returns an answer as text in multiple sentences, like a list. This output should be presented in the form of one word or several, which will be used as a filter or keywords in the following parts of the pipeline. So this means the GPT model will also require other models to operate.

Knowing that, let’s discuss GPT models as a complementary tool suitable for different business and technical needs.

ChatGPT Business Use Cases

ChatGPT is much more capable than its predecessors since it was trained with the help of the reinforcement learning with human feedback (RLHF) technique. This provided a distinguishing level of performance compared to the no less popular GPT 3 model. Because of this, ChatGPT is now adopted to a range of business and technical tasks using its NLP capabilities. Let us ask what ChatGPT thinks of its business use cases:

Here we can see the model providing an answer listing some of the most common use cases. Since the model relies on data limited to 2021, it’s not even a full list and there are tons of low level tasks we can delegate to ChatGPT. We can also try to narrow down our focus, and extract some value in a single domain area:

Based on these examples, we can make a bunch of conclusions concerning three of the most promising business use cases:

1. CUSTOMER SUPPORT

ChatGPT is a highly capable conversational bot that can be customized with your own data. As it’s able to understand the question correctly even with broken grammatical structure or missing syntactical parts, it can be considered a ready made solution for customer support systems, and namely chat bots.

2. LEAD GENERATION

Following up on the previous point, the same capabilities can be used in lead generation activities since in most cases, the ChatGPT level of speech doesn’t differ too much from what people can write.

3. DATA ANALYTICS

At some point, GPT models may become an alternative to modern business intelligence and data analytic solutions. Since they can operate on huge databases of historical data and provide human workers with valuable insights.

However, when it comes to real-time data analysis that requires constant updates and the most recent information available, GPT models appear to be clumsy. The process of training with the RLHF technique is complex, and besides that, it will require a machine learning pipeline with automated retraining and redeployment of the models. Which is, kind of a rocket science… for now.

So applications that rely heavily on real-time data updates are not really suitable for GPT models yet. What else can we do with it?

1. Copywriting with ChatGPT

One of the most popular statements about language models and neural networks is that they can definitely take the job of copywriters or at least boost the speed of writing to a large extent. Let us see whether it’s true for today. We’ll use two types of requests to check how the model will handle article writing on the software engineering topic. The first type of request simply states the general idea of the text, while the second one will specify things I want the model to describe and provide an expert opinion or arguments for.

Request: “Write me an article about different PHP frameworks.”

ChatGPT output:

Request: “Write me an article about the difference between Laravel and Symfony frameworks, and what PHP framework should I choose for an Ecommerce application.”

ChatGPT output:

In both cases, the model handles the topic pretty well concerning the information standpoint. Grammatically and stylistically texts are also fine. However, when it comes to the level of creativity, ease of reading, reasoning, and justification of a certain point, it looks very standard and unconvincing.

While ChatGPT certainly can generate some blueprints to simplify your marketing efforts, it imposes one problem for human proofreaders. As the model parses the data by itself, it appears difficult to find the source of information and prove the model’s claims. So to be used for marketing purposes, ChatGPT requires fine tuning and additional features like AI explainability to be effective and useful.

2. General Programming with ChatGPT

If GPT models can speak a human language, nothing stops them from using programming languages to write applications. Our PHP/DevOps group leader, Anton Logvinenko, put this to the test to see what results can be achieved by exploiting the model to write code in PHP for various functions.

Request: “Write a PHP code on Laravel Framework which will display the list of users”.

ChatGPT response:

In most cases, this code will definitely work. In this specific case, it lacks the database structure, as well as which fields does each user have, like name and surname. So you should keep in mind that currently ChatGPT doesn’t provide you with a finished code that can be used off the shelf.

The use case I can see for this is to support junior-level developers in learning PHP or its frameworks and speed up the process a bit. While the code itself is written in a syntactically correct manner, there are a lot of things to improve.

Anton Logvinenko

PHP/DevOps Group Leader

Request: “Write a PHP code which implements REST API with CRUD endpoints for article resource”.

ChatGPT response:

Again, looking at what the models suggest, the code looks like a beginner level effort in PHP, but it will most likely work if you set up the database, configure access and so on. Although this code is written on the level of an intern PHP developer, it doesn’t use any of the frameworks available, and it is not optimized nor can you easily scale it to the whole application. Additionally, there are potential security threats in the form of SQL injections which ChatGPT leaves places for, which is not good in any case.

Anton Logvinenko

PHP/DevOps Group Leader

ChatGPT response:

In the first attempt of this example, I wanted ChatGPT to generate the tree of items based on the provided example. However, the output turned out to be a garbage code, and I figured out that I didn’t specify that it’s a tree in the prompt. So at the second attempt I’ve added an explanation of what I need as the output.

Anton Logvinenko

PHP/DevOps Group Leader

ChatGPT response:

In the second attempt, ChatGPT completed the task and the tree looks fine except for the code quality. What can we conclude from this little experiment? Obviously, ChatGPT isn’t able to replace software engineers yet because the approach and the quality of code writing is still very shallow. But at the end of the day, this thing will definitely see improvements over time. For now, it’s too early to speak of ChatGPT as an automated code writer.

Anton Logvinenko PHP/DevOps Group Leader

If you are considering options to partially automate coding workflows and support your developers, we would recommend using Copilot. Copilot uses OpenAI Codex to suggest code snippets and functions in real-time, and it proved to be much more usable by middle-level/senior software engineers with a much better quality code. Compared with just Googling what’s needed to finish the code, Copilot enables much faster workflows, so comes as a better alternative to what ChatGPT can suggest at the moment.

3. Building AI pipelines with the help of GPT Models

Given that we know GPT models can complete the majority of NLP tasks, we can think of it as an alternative to training your own language models. Following this tradition, I’ll provide the basics by questioning ChatGPT why it can be a better option than training a custom NLP model:

It is important to realize the limitations of this approach. Since GPT models do not have access to real-time data, they can only help you kickstart some service without initial data gathering. Over time, you’ll be able to collect live data from the model’s interaction with the users, but this doesn’t serve as a hotfix for all the specific NLP tasks.

Instead, we can think of other application areas in data science where GPT models can be used.

Processing Table Data with ChatGPT

In order to build a classification, regression, or forecasting model, we need to run through exploratory data analysis (EDA) steps. EDA allows us to find the relationships between different features in data, and denote the insights and outliers to understand the reason for this or that anomalous behavior. So the majority of data science projects will require performing 4 steps:

With the help of the GPT model, the above pipeline can be stripped down to a single well-formulated request to the GPT model. For instance, let’s use this request about Amazon sales numbers by quarters, and ask ChatGPT to extract insights from this information.

And here, we can see what insights and conclusions ChatGPT came up with, based on the previously presented data:

ChatGPT response:

“Insights:

• Amazon’s net sales increased by 38% YoY in Q4
• In Q1, net sales increased by 26% YoY
• in Q2 it increased by 40% YoY
• Q2 it increased by 40% YoY, and in Q3 it increased by 37% YoY

Conclusions (reasons):

• It’s important to note that the COVID-19 pandemic had a significant impact on Amazon’s sales, as more people turned to online shopping while physical stores were closed. Additionally, Amazon’s AWS and Advertising segments also had significant growth in 2020.”

The output of the model can vary in quality, so it does require supervision or finetuning of the initial prompt. However, such capabilities can be easily applied to make the EDA stage easier and potentially automate data science pipeline tasks.

5. ChatGPT for recommendation systems

Recommendation systems or recommender engines have become as common as search engines on various platforms and within various applications. Recommendations perform one very important task: understanding the preferences of each customer, to suggest relevant content and keep the customer on the platform as long as possible.

But, despite competitive advantages recommender systems can provide, the fact that we can’t build one without gathering data, experimenting and training a working model appears to be a major reason why not everyone can afford recommendations. To start recommending, we generally need to collect data concerning our content first, and then methodically track what content categories our customers gravitate towards.

In this situation, we can use ChatGPT as an alternative to recommendation engines, because it already has tons of data inside of it. While it’s not fine tuned for your content and it doesn’t know your users’ search history, it can provide fairly specific recommendations once we use specific enough prompts.

For instance, we used this type of request: “I want to have a journey. I have a 2-year-old kid and like to spend time near the sea. Where should I go?”. Here is the output:

ChatGPT response: “Taking a trip with a 2-year-old and spending time near the sea can be a great idea. Here are a few suggestions for destinations that might be a good fit for you and your family:

• The beaches of California: The beaches of California offer a great combination of sun, sand, and sea. Some popular destinations include Santa Monica, Huntington Beach, and Newport Beach.
• The beaches of Florida: Florida is home to many beautiful beaches, including Miami Beach, Siesta Key, and Clearwater Beach.
• The beaches of Hawaii: Hawaii’s beaches are famous for their clear waters and soft, white sand. Some popular destinations include Waikiki Beach, Kaanapali Beach, and Poipu Beach.
• The beaches of the Caribbean: The Caribbean is home to many beautiful beaches, including Seven Mile Beach in Jamaica, Eagle Beach in Aruba, and Grace Bay in Turks and Caicos.
• The beaches of the Mediterranean: Mediterranean countries like Spain, Greece, and Italy are home to many beautiful beaches. Some popular destinations include Playa de Muro in Mallorca, Mykonos in Greece, and Positano in Italy.

It’s worth noting that these destinations also offer a variety of other activities and attractions for both adults and children, such as swimming, sunbathing, water sports, playgrounds and theme parks. It’s also important to consider other factors such as weather, safety, and accessibility before making a decision.”

If we specify our request by adding details, here’s how the answer may look:

Request: “I want to have a journey. I have a 2-year-old kid and like to spend time near the sea, I have 3000$. Where should I go?”.

ChatGPT response:

“Taking a trip with a 2-year-old and spending time near the sea can be a great idea. It’s important to consider your budget when planning your trip. Here are a few suggestions for destinations that might be a good fit for you and your family within your budget:

• The beaches of Mexico: Mexico offers a great combination of sun, sand, and sea. Some popular destinations include Cancun, Playa del Carmen, and Puerto Vallarta.
• The beaches of Central America: Countries like Costa Rica, Panama, and Nicaragua have beautiful beaches, great weather and are budget-friendly.
• The beaches of Southeast Asia: Countries like Thailand, Vietnam, and the Philippines have beautiful beaches, great weather and are budget-friendly.
• The beaches of the Caribbean: Some Caribbean destinations such as the Dominican Republic, Jamaica, and Cuba are budget-friendly and have beautiful beaches.

It’s worth noting that these destinations also offer a variety of other activities and attractions for both adults and children, such as swimming, sunbathing, water sports, playgrounds and theme parks. Also, when traveling with a 2-year-old, it’s important to consider accessibility, safety, and the availability of good quality accommodation and services, such as good hospitals and childcare facilities.”

As you can see the output isn’t framed around parameters we provided as a request, as the model tries to go past the initial prompt. And even those raw results can perform better than trying to find content by applying dozens of filters, because it requires a human to use the right words to get satisfying results. That’s basically the major advantage of NLP-based recommendations over standard approaches.

Using ChatGPT for Computer Vision Tasks

Since GPT models are trained on text data, it’s obvious we can’t replace computer vision models this way. However, ChatGPT can simplify the implementation of certain parts of the computer vision pipeline such as:

• Color detection for known images
• Image similarity, e.g. logos similar to MCDonalds by color or typography
• Object detection, for instance identifying the object depicted on photos, graphic materials, brand symbols, etc.
• Optical character recognition or OCR

The information provided by ChatGPT can be useful when analyzing and preparing data for a computer vision model in different areas. For instance, in OCR tasks, the model can be used to drag text from a picture, like in ID card recognition. This is a separate step in preparing data for training.

Since ID cards have their own format and different fields in each country, the task of recognizing ID cards can be very resource intensive. With GPT models, we can reduce the time needed for extracting textual data from the images, since ChatGPT can take some part of a pipeline on its own.

The Importance of Prompt Engineering and the Future of GPT Models

Prompt engineering is basically the discipline of creating working input text that’s passed to the GPT model like a technical task. The output will always depend on how you explain what you need, and it involves understanding what’s the best way to point to each aspect of your request so that the model doesn’t produce some Frankenstein for you. There are several basic principles of rapid development of GPT models:

Be specific: the more specific the prompt, the more relevant the generated text will be to the task at hand.

Define the context: the more context the model has, the clearer the task for the GPT model is. Set the prompt in the same language and in the same terminology (using the features of the domain area) in which you want to receive an answer.

Find a compromise between prompt length and specificity: longer and more specific prompts can lead to more accurate and relevant answers, but also complicate model management. In some cases, complex and long prompts can trigger the model to break and write an unfinished response, which we have experienced several times during our testing.

All the following examples were made exploiting ChatGPT on a pre-release stage, and it shows much better results than its predecessors. However, GPT 4 model has been announced by OpenAI in a first quarter of 2023, and it uses 500x more data samples then GPT 3. So it also makes sense to keep track of the release dates and the situation is very dynamic and you might prefer future models by OpenAI for your product.

The main advantage of GPT models is the ability to find ways to solve a problem without an initial large volume of data. Obviously, such models cannot replace the desired functionality completely. However, in the first iterations of the product, the use of such models helps to start development, providing users with a service while collecting custom data.