1,913 reads

Using Function Calling to Integrate Your GPT Chatbot With Anything

by Tomas FernandezAugust 9th, 2023

Too Long; Didn't Read

Exploring OpenAI's GPT new function calling feature, allowing chatbots to interact with external tools and APIs. Unlock the potential of AI-powered applications with this feature.

People Mentioned

featured image - Using Function Calling to Integrate Your GPT Chatbot With Anything

Imagine creating an AI assistant to which you can say something like, "Book me the earliest reservation for the nearest Thai restaurant and update my calendar." Language models continue to push boundaries and evolve. OpenAI, the company behind ChatGPT, recently introduced a powerful new feature called function calling in their GPT models. Function calling simplifies the creation of chatbots that communicate with external tools and APIs, opening up a new realm of possibilities for AI-powered applications.

In this article, we will delve into the concept of function calling, its implications, and its transformative impact on how we interact with AI systems by creating NewsGPT, a chatbot that brings you breaking news worldwide.

Example run of NewsGPT. A command line interface shows the user asking, "What's the latest news in the US?" and receiving a list of five articles with links to the journals or sources.

What is Function Calling?

Function calling is a new feature in OpenAI's GPT-4-0613 and GPT-3.5 Turbo-0613 models. These AI models are trained to detect the need for function calling based on the user's prompt and respond with a structured call request instead of regular text.

Function calling allows chatbots to interact with other systems, enabling the GPT models to respond to questions they otherwise could not, such as those requiring real-time information or data not included in their training set. In other words, function calling provides another way to teach AI models how to interact with the external world.

What is the Purpose of Function Calling?

Before function calling, there were only two ways of augmenting the capabilities of a GPT language model:

Fine-tuning: further training the language model by providing example responses. Fine-tuning is a powerful technique, but it requires significant work (and cost) to prepare the training data. In addition, only a few older models can be fine-tuned until OpenAI enables this feature in GPT-3.5 and GPT-4 models.
Embeddings: enriching the prompt with context data can extend the bot's knowledge and create more accurate responses. The downside is that this context can take up a lot of tokens, increasing the cost and leaving fewer tokens free for building complex responses.

Function calling adds a third way of extending the GPT capabilities by allowing it to ask us to run functions on its behalf. The model can then take the function's result and build a human-readable response that fits seamlessly into the current conversation.

How to Use Function Calling

The introduction of function calling changes how we interact with the GPT API. Before these functions, the interaction was simple:

Send a prompt to the API.
Receive a response.
Repeat.

A diagram showing the basic interaction with the GPT API before function calling.

With function calling, the sequence becomes more involved:

Send the user prompt along with a list of callable functions.
The GPT model responds with either a regular text response or a function call request.
If the model requests a function call, your chatbot's job is to execute it and return the results to the API.
Using the supplied data, the model then forms a coherent text response. However, in some cases, the API may request a new function call.

A diagram showing the more complex interaction with the GPT API using function calling.

Function Calling with the Chat Completions API

To allow the model to call functions, we must use the Chat Completions API. The API takes a POST request with a JSON payload containing a list of messages to process. A typical prompt sent to the API looks like the following:

{
 "model": "gpt-3.5-turbo",
 "messages": [
 {
 "role": "user",
 "content": "How many planets does the solar system have?"
 }
 ]
}

The role: user informs the API that the content is user-generated. The GPT API might reply with something along these lines:

{
   "id":"chatcmpl-7WVo3fYwerpAptzeqU46JamOvgBzh",
   "object":"chat.completion",
   "created":1687983115,
   "model":"gpt-3.5-turbo-0613",
   "choices":[
      {
         "index":0,
         "message":{
            "role":"assistant",
            "content":"There are eight planets in the solar system. They are:\n\n1. Mercury\n2. Venus\n3. Earth\n4. Mars\n5. Jupiter\n6. Saturn\n7. Uranus\n8. Neptune"
         },
         "finish_reason":"stop"
      }
   ],
   "usage":{
      "prompt_tokens":15,
      "completion_tokens":44,
      "total_tokens":59
   }
}

The role: assistant corresponds to messages generated by the GPT model. To keep the conversation flow, we must supply the entire message history back to the API on each request. For example, if we want to delve deeper into our previous question, the corresponding JSON payload would be:

{
   "model":"gpt-3.5-turbo",
   "messages":[
      {
         "role":"user",
         "content":"How many planets does the solar system have?"
      },
      {
         "role":"assistant",
         "content":"There are eight planets in the solar system. They are:\n\n1. Mercury\n2. Venus\n3. Earth\n4. Mars\n5. Jupiter\n6. Saturn\n7. Uranus\n8. Neptune"
      },
      {
         "role":"user",
         "content":"Tell me more about the second planet."
      }
   ]
}

To let the language model know it can call functions, we need to add a list of them to the payload. For example:

{
   "model":"gpt-3.5-turbo-0613",
   "messages":[
      {
         "role":"user",
         "content":"How is the weather in NYC?"
      }
   ],
   "functions":[
      {
         "name":"get_current_weather",
         "description":"Get the current weather in a given location",
         "parameters":{
            "type":"object",
            "properties":{
               "location":{
                  "type":"string",
                  "description":"The city and state, e.g. San Francisco, CA"
               },
               "unit":{
                  "type":"string",
                  "enum":[
                     "celsius",
                     "fahrenheit"
                  ]
               }
            },
            "required":[
               "location"
            ]
         }
      }
   ]
}

You may have noticed that we switched the model to "gpt-3.5-turbo-0613" because it supports function calling. If the model decides to call the function, we will receive a response of type role: assistant with a function_call property defined like this:

{
   "id":"chatcmpl-7WWG94C1DCFlAk5xmUwrZ9OOhFnOq",
   "object":"chat.completion",
   "created":1687984857,
   "model":"gpt-3.5-turbo-0613",
   "choices":[
      {
         "index":0,
         "message":{
            "role":"assistant",
            "content":null,
            "function_call":{
               "name":"get_current_weather",
               "arguments":"{\n \"location\": \"New York, NY\"\n}"
            }
         },
         "finish_reason":"function_call"
      }
   ],
   "usage":{
      "prompt_tokens":81,
      "completion_tokens":19,
      "total_tokens":100
   }
}

Our task is to execute get_current_weather with the provided arguments. OpenAI does not execute the function. Instead, it's the job of our chatbot to run it and parse the returned data.

Once we retrieve the weather data, we send it back to the model using a new type of role called function. For example:

{
   "model":"gpt-3.5-turbo-0613",
   "messages":[
      {
         "role":"user",
         "content":"How is the weather in NYC?"
      },
      {
         "role":"assistant",
         "content":null,
         "function_call":{
            "name":"get_current_weather",
            "arguments":"{\n \"location\": \"New York, NY\"\n}"
         }
      },
      {
         "role":"function",
         "name":"get_current_weather",
         "content":"Temperature: 57F, Condition: Raining"
      }
   ],
   "functions":[
      {
         "name":"get_current_weather",
         "description":"Get the current weather in a given location",
         "parameters":{
            "type":"object",
            "properties":{
               "location":{
                  "type":"string",
                  "description":"The city and state, e.g. San Francisco, CA"
               },
               "unit":{
                  "type":"string",
                  "enum":[
                     "celsius",
                     "fahrenheit"
                  ]
               }
            },
            "required":[
               "location"
            ]
         }
      }
   ]
}

Note that we passed the entire message history to the API, including our original prompt, the function call from the model, and the result of executing the weather function in our code. This enables the language model to understand the context in which the function was called.

Finally, the model may reply with a properly formatted answer, responding to our initial question:

{
   "id":"chatcmpl-7WWQUccvLUfjhbIcuvFrj2MDJVEiN",
   "object":"chat.completion",
   "created":1687985498,
   "model":"gpt-3.5-turbo-0613",
   "choices":[
      {
         "index":0,
         "message":{
            "role":"assistant",
            "content":"The weather in New York City is currently raining with a temperature of 57 degrees Fahrenheit."
         },
         "finish_reason":"stop"
      }
   ],
   "usage":{
      "prompt_tokens":119,
      "completion_tokens":19,
      "total_tokens":138
   }
}

Building NewsGPT

To learn more about function calling, we will build NewsGPT, a Python chatbot capable of accessing breaking news in real-time. The bot will use function calling to determine what kind of data to fetch from NewsAPI.org.

To build the bot, you will need the following:

An OpenAI API key. This requires a credit card because API requests have a cost. However, new accounts receive a $5 credit for the first three months.
A NewsAPI API key. Register at NewsAPI.org and get a starter key for free.
Python 3.

Setting Up the Project

Install the required dependencies:

$ pip install openai tiktoken

The project consists of only one file; let's name it newsgpt.py. I will begin by adding all the necessary imports:

import openai
import tiktoken
import json
import os
import requests

Next, I will define a few constants:

The GPT model to use. I will use gpt-3.5-turbo-16k as it has a 16k token limit, allowing me to process longer conversations with more context.
The system prompt that instructs the model on its basic purpose.
The encoding used to count tokens in strings and messages; required to ensure we do not exceed the language model limits.
The maximum number of functions to call in a chain (more on this later).

llm_model = "gpt-3.5-turbo-16k"
llm_max_tokens = 15500
llm_system_prompt = "You are an assistant that provides news and headlines to user requests. Always try to get the lastest breaking stories using the available function calls."
encoding_model_messages = "gpt-3.5-turbo-0613"
encoding_model_strings = "cl100k_base"
function_call_limit = 3

All OpenAI models have a token limit. If this limit is exceeded, the API will throw an error instead of responding to our request. So, we need a function to count the number of tokens. I will use this function from the official example documentation:

def num_tokens_from_messages(messages):
 """Returns the number of tokens used by a list of messages."""
 try:
 encoding = tiktoken.encoding_for_model(encoding_model_messages)
 except KeyError:
 encoding = tiktoken.get_encoding(encoding_model_strings)

 num_tokens = 0
 for message in messages:
 num_tokens += 4
 for key, value in message.items():
 num_tokens += len(encoding.encode(str(value)))
 if key == "name":
 num_tokens += -1
 num_tokens += 2
 return num_tokens

Defining a Function to Call

Now, I will define a function to query the NewsAPI.org API to get the breaking news:


def get_top_headlines(query: str = None, country: str = None, category: str = None):
 """Retrieve top headlines from newsapi.org (API key required)"""

    base_url = "https://newsapi.org/v2/top-headlines"
    headers = {
    "x-api-key": os.environ['NEWS_API_KEY']
    }
    params = { "category": "general" }
    if query is not None:
    params['q'] = query
    if country is not None:
    params['country'] = country
    if category is not None:
    params['category'] = category

    # Fetch from newsapi.org - reference: https://newsapi.org/docs/endpoints/top-headlines
    response = requests.get(base_url, params=params, headers=headers)
    data = response.json()

    if data['status'] == 'ok':
    print(f"Processing {data['totalResults']} articles from newsapi.org")
    return json.dumps(data['articles'])
    else:
    print("Request failed with message:", data['message'])
    return 'No articles found'

To inform GPT about this function, we need to describe using a specific JSON structure. The format is described in the official documentation as follows:

signature_get_top_headlines = {
   "name":"get_top_headlines",
   "description":"Get top news headlines by country and/or category",
   "parameters":{
      "type":"object",
      "properties":{
         "query":{
            "type":"string",
            "description":"Freeform keywords or a phrase to search for."
         },
         "country":{
            "type":"string",
            "description":"The 2-letter ISO 3166-1 code of the country you want to get headlines for"
         },
         "category":{
            "type":"string",
            "description":"The category you want to get headlines for",
            "enum":[
               "business",
               "entertainment",
               "general",
               "health",
               "science",
               "sports",
               "technology"
            ]
         }
      },
      "required":[
         
      ]
   }
}

Using the Chat Completions API with Function Calling

Next, I will define the complete function, which performs several tasks:

Adds a system prompt at the end of the messages. This system message helps define the role that the GPT model will fulfill.
Removes old messages if the total token count exceeds the model's limit.
Sends the request to the GPT API.
Removes the system message from the end of the list.

def complete(messages, function_call: str = "auto"):
 """Fetch completion from OpenAI's GPT"""

 messages.append({"role": "system", "content": llm_system_prompt})

 # delete older completions to keep conversation under token limit
 while num_tokens_from_messages(messages) >= llm_max_tokens:
 messages.pop(0)

 print('Working...')
 res = openai.ChatCompletion.create(
 model=llm_model,
 messages=messages,
 functions=[signature_get_top_headlines],
 function_call=function_call
 )
 
 # remove system message and append response from the LLM
 messages.pop(-1)
 response = res["choices"][0]["message"]
 messages.append(response)

 # call functions requested by the model
 if response.get("function_call"):
 function_name = response["function_call"]["name"]
 if function_name == "get_top_headlines":
 args = json.loads(response["function_call"]["arguments"])
 headlines = get_top_headlines(
 query=args.get("query"),
 country=args.get("country"),
 category=args.get("category") 
 )
 messages.append({ "role": "function", "name": "get_top_headline", "content": headlines})

To complete the bot, we will create the main loop that continuously prompts the user for input and provides the answers:

print("\nHi, I'm a NewsGPT, a breaking news AI assistant. I can give you news for most countries over a wide range of categories.")
print("Here are some example prompts:\n - Tell me about the recent science discoveries\n - What is the latest news in the US?\n - What has Elon Musk been up to recently?")

messages = []
while True:
    prompt = input("\nWhat would you like to know? => ")
    messages.append({"role": "user", "content": prompt})
    complete(messages)

    # the LLM can chain function calls, this implements a limit
    call_count = 0
    while messages[-1]['role'] == "function":
    call_count = call_count + 1
    if call_count < function_call_limit:
    complete(messages)
    else:
    complete(messages, function_call="none")

    # print last message
    print("\n\n==Response==\n")
    print(messages[-1]["content"].strip())
    print("\n==End of response==")

Next, we need to define environment variables for the OpenAI and NewsAPI API keys:

export OPENAI_API_KEY=YOUR_API_KEY
export NEWS_API_KEY=YOUR_API_KEY

Finally, load the file and run the chatbot.

$ source .env
$ python newsgpt.py

Testing NewsGPT

We can inquire about a specific topic or person. For example:

=> What is Elon Musk up these days?
Working...
Processing 1 articles from newsapi.org
Working...


==Response==

Elon Musk has been in the news recently regarding a feud with Mark Zuckerberg, the CEO of Meta. According to an article from The Washington Post, Mark Zuckerberg is trying to make himself more relevant to the tech elite, and there seems to be a rivalry between him and Elon Musk. The article discusses their ongoing fight and how Zuckerberg is trying to compete with Musk's online antagonism and offline antics. You can read more about it [here](https://www.washingtonpost.com/technology/2023/06/24/elon-musk-mark-zuckerberg-fight-meta-twitter-rival/).

==End of response==

The bot is also capable of recognizing categories, allowing us to request health-related news:

Are there important developments in health?
Working...
Processing 1000 articles from newsapi.org
Working...


==Response==

Based on the latest headlines in the health category, here are some important developments:

1. [A Case of Herpes Zoster Ophthalmicus in a Recently Transplanted Renal Patient](https://www.cureus.com/articles/164429-a-case-of-herpes-zoster-ophthalmicus-in-a-recently-transplanted-renal-patient?score_article=true) - This case study presents the clinical course of a 51-year-old male who underwent a renal transplant and developed Herpes Zoster Ophthalmicus.

2. [Cuáles son los principales consejos a seguir para mejorar el estado de ánimo a base de una buena alimentación](https://news.google.com/rss/articles/CBMilQFodHRwczovL3d3dy5jYW5hbDI2LmNvbS9nZW5lcmFsL2N1YWxlcy1zb24tbG9zLXByaW5jaXBhbGVzLWNvbnNlam9zLWEtc2VndWlyLXBhcmEtbWVqb3Jhci1lbC1lc3RhZG8tZGUtYW5pbW8tYS1iYXNlLWRlLXVuYS1idWVuYS1hbGltZW50YWNpb24tLTM0NTMwM9IBAA?oc=5) (in Spanish) - This article provides tips on improving mood through a good diet.

3. [Как да стопите коремните мазнини за лятото](https://news.google.com/rss/articles/CBMiXmh0dHBzOi8vbS5hei1qZW5hdGEuYmcvYS81LXpkcmF2ZS1pLWtyYXNvdGEvNjM1Mzkta2FrLWRhLXN0b3BpdGUta29yZW1uaXRlLW1hem5pbmktemEtbGlhdG90by_SAWJodHRwczovL20uYXotamVuYXRhLmJnL2EvNS16ZHJhdmUtaS1rcmFzb3RhLzYzNTM5LWthay1kYS1zdG9waXRlLWtvcmVtbml0ZS1tYXpuaW5pLXphLWxpYXRvdG8vYW1wLw?oc=5) (in Bulgarian) - This article provides tips on losing belly fat for the summer.

4. [Recap: From RCC Diagnosis to Treatment, Toxicity Management, and Beyond](https://news.google.com/rss/articles/CBMiZ2h0dHBzOi8vd3d3LmNhbmNlcm5ldHdvcmsuY29tL3ZpZXcvcmVjYXAtZnJvbS1yY2MtZGlhZ25vc2lzLXRvLXRyZWF0bWVudC10b3hpY2l0eS1tYW5hZ2VtZW50LWFuZC1iZXlvbmTSAQA?oc=5) - This article discusses the diagnosis, treatment, and management of renal cell carcinoma.

5. [Review Highlights Progress, Challenges With CRS in Cancer Immunotherapies](https://news.google.com/rss/articles/CBMiYmh0dHBzOi8vd3d3LmFqbWMuY29tL3ZpZXcvcmV2aWV3LWhpZ2hsaWdodHMtcHJvZ3Jlc3MtY2hhbGxlbmdlcy13aXRoLWNycy1pbi1jYW5jZXItaW1tdW5vdGhlcmFwaWVz0gEA?oc=5) - This review highlights the progress and challenges in cancer immunotherapies targeting CRS (cytokine release syndrome).

These are just a few of the recent developments in the field of health. For more detailed information, you can read the full articles by clicking on the provided links.

==End of response==

It can countries as well, enabling us to query the bot for news about a particular region:

What is the latest news from France?
Working...
Processing 34 articles from newsapi.org
Working...


==Response==

Here are some of the latest news headlines in France:

1. "Diplôme national du brevet session 2023" - The Ministry of National Education announces the national diploma for the 2023 session. [Read more](https://news.google.com/rss/articles/CBMiTGh0dHBzOi8vd3d3LmVkdWNhdGlvbi5nb3V2LmZyL2RpcGxvbWUtbmF0aW9uYWwtZHUtYnJldmV0LXNlc3Npb24tMjAyMy0zNzg1NjDSAQA?oc=5)

2. "Cyclisme: la Nordiste Victoire Berteau sacrée championne de France après sa victoire sur les routes de Cassel" - Victoire Berteau from Nord wins the championship in cycling in France. [Read more](https://news.google.com/rss/articles/CBMiiQFodHRwczovL3d3dy5mcmFuY2V0dmluZm8uZnIvc3BvcnRzL2N5Y2xpc21lL2N5Y2xpc21lLXZpY3RvaXJlLWJlcnRlYXUtc2FjcmVlLWNoYW1waW9ubmUtZGUtZnJhbmNlLWFwcmVzLXNhLXZpY3RvaXJlLWEtY2Fzc2VsXzU5MDg4NDcuaHRtbNIBAA?oc=5)

3. "Guerre en Ukraine: comment les capitales étrangères réagissent-elles à la rébellion de la milice Wagner en Ru" - Foreign capitals' reactions to the rebellion of the Wagner militia in Ukraine. [Read more](https://news.google.com/rss/articles/CBMiwAFodHRwczovL3d3dy5mcmFuY2V0dmluZm8uZnIvbW9uZGUvZXVyb3BlL21hbmlmZXN0YXRpb25zLWVuLXVrcmFpbmUvZ3VlcnJlLWVuLXVrcmFpbmUtY29tbWVudC1sZXMtY2FwaXRhbGVzLWV0cmFuZ2VyZXMtcmVhZ2lzc2VudC1lbGxlcy1hLXJlYmVsbGlvbi1kZS1sYS1taWxpY2Utd2FnbmVyLWVuLXJ1c3NpZV81OTA4NzY2Lmh0bWzSAQA?oc=5)

4. "Marche des fiertés LGBT+: six jeunes mineurs interpellés pour homophobie" - Six minors arrested for homophobia during the LGBT+ Pride March. [Read more](https://news.google.com/rss/articles/CBMifmh0dHBzOi8vd3d3LnJ0bC5mci9hY3R1L2p1c3RpY2UtZmFpdHMtZGl2ZXJzL21hcmNoZS1kZXMtZmllcnRlcy1sZ2J0LXNpeC1qZXVuZXMtbWluZXVycy1pbnRlcnBlbGxlcy1wb3VyLWhvbW9waG9iaWUtNzkwMDI3Nzg4M9IBAA?oc=5)

5. "ATP 500 Queen's - De Minaur a dominé Rune avec autorité: le film de la demi-finale" - Alex de Minaur dominates Rune in the ATP 500 Queen's semifinals. [Read more](https://news.google.com/rss/articles/CBMimwFodHRwczovL3d3dy5ldXJvc3BvcnQuZnIvdGVubmlzL2F0cC1sb25kcmVzLzIwMjMvYXRwLTUwMC1xdWVlbi1zLXN1aXZlei1sYS1kZW1pLWZpbmFsZS1lbnRyZS1hbGV4LWRlLW1pbmF1ci1ldC1ob2xnZXItcnVuZS1lbi1kaXJlY3Rfc3RvOTY3MTM4My9zdG9yeS5zaHRtbNIBAA?oc=5)

These are just a few of the latest news headlines in France. Let me know if you want more information about any specific news article.

==End of response==

Ideas for Improvement

This simple bot is quite capable, even with a single function call. Now, imagine the possibilities if we integrate more features. Here are a few ideas to augment NewsGPT:

Retrieve the original articles to get summaries and analyze the news. We would need to navigate paywalls, perform web scraping, or check if RSS feeds or APIs provide content.
Add more endpoints. NewsAPI offers endpoints for searching news by date, categories, and filtering through sources.
Incorporate extra integrations, such as obtaining real-time data from sources like weather or finance.

Conclusion

Function calling is a powerful feature in OpenAI's GPT models, enabling them to interact with external tools and APIs in a more deterministic and structured manner. This feature lays the groundwork for more dynamic and responsive AI applications capable of providing current information and executing tasks beyond what was previously possible.

Happy building!

Also published here.