How to Use Function Calling with OpenAI GPT Models

Intro

Following our previous exploration in "In-Depth Analysis: OpenAI API 1106 Updates - Parallel Function Calling," this post delves into the practical application of Function Calling in your projects using OpenAI's GPT models.

Why

Function Calling extends the native capabilities and knowledge boundaries of OpenAI GPT models. With data cutoffs inherent in models like gpt-3.5-turbo-1106(Sep. 2021) and gpt-4-1106-preview(Apr. 2023), accessing up-to-date or real-time information poses a challenge. Native GPT models, limited to text-based reasoning, struggle with tasks like image generation, reading PDFs, online data retrieval, external tool integration, and system interactions. Function Calling addresses these limitations, enabling developers to invoke external functions with the GPT models.

What

Function Calling allows GPT models to, through function descriptions in request as well as user queries, infer the need for external function execution. Note that the GPT model itself does not execute functions; it structures function calls and necessary parameters, which are then executed by the server. This process typically involves at least two requests to OpenAI chat completion endpoint.

How

The server-side process of function calling involves:

1. Incorporating function definitions and parameters in the chat completion request to the GPT model.

2. The GPT model, upon receiving the request, decides if a function call is necessary and determines the parameters.

3. The server executes the specified function with the provided arguments and retrieves the result.

4. A second chat completion request is sent with the function result, allowing the GPT model to generate a final output based on all gathered information.

Example

Using the "Get Weather" example from Prompter, we illustrate this process:

1. A chat completion request is sent, listing available tools (only type=function currently) and specifying parameters based on the JSON Schema reference.

   Tools_choice tells how the model responds to function calls.

   • Auto: either respond to user directly or call a function.

   • None: respond to user directly without calling any function.

   • Specified function name: instruct the model to call the specific function.

   In the 'Functions' tab within Prompter, you only need to fill in the 'parameters' section of the schema.

   {
       "messages": [
           {
               "role": "system",
               "content": "You are a weather forecaster."
           },
           {
               "role": "user",
               "content": "What's the weather in San Jose tomorrow?"
           }
       ],
       "model": "gpt-3.5-turbo-1106",
       "temperature": 1,
       "top_p": 1,
       "n": 1,
       "frequency_penalty": 0,
       "presence_penalty": 0,
       "tool_choice": "auto",
       "tools": [
           {
               "type": "function",
               "function": {
                   "name": "get_weather",
                   "description": "Determine weather in my location.",
                   "parameters": {
                       "type": "object",
                       "properties": {
                           "location": {
                               "type": "string",
                               "description": "The city and state e.g. San Francisco, CA"
                           },
                           "unit": {
                               "type": "string",
                               "enum": [
                                   "c",
                                   "f"
                               ]
                           }
                       },
                       "required": [
                           "location"
                       ]
                   }
               }
           }
       ]
   }

2. The GPT model's response includes tool_calls if function calling is needed, specifying the function name and arguments.

   Below example instruct the server to call a function named 'get_weather' with arguments like 'location=San Jose, CA'. Pay attention to the ID (e.g., 'call_1DNpUWV55n4Gccq28CPRJYmo'), as it is essential for reuse in the second chat completion request in step 4.

   [
       {
           "index": 0,
           "message": {
               "role": "assistant",
               "content": null,
               "tool_calls": [
                   {
                       "id": "call_1DNpUWV55n4Gccq28CPRJYmo",
                       "type": "function",
                       "function": {
                           "name": "get_weather",
                           "arguments": "{\"location\":\"San Jose, CA\"}"
                       }
                   }
               ]
           },
           "logprobs": null,
           "finish_reason": "tool_calls"
       }
   ]

3. The server parses the response, validates the function and arguments, and executes the function to obtain the result.

   In Prompter, you can generate mock responses to simulate the function's output.

   

4. The initial messages, the GPT model's function call, and the function output result are combined in a second chat completion request.

   The process involves concatenating the messages(role=system & role=user) from step 1 with the GPT model's function call (role=assistant) from step 2, and finally appending the function's output result (role=tool) from step 3. This tool_calls' ID should correspond to the one from step 2.

   {
       "messages": [
           {
               "role": "system",
               "content": "You are a weather forecaster."
           },
           {
               "role": "user",
               "content": "What's the weather in San Jose tomorrow?"
           },
           {
               "role": "assistant",
               "content": null,
               "tool_calls": [
                   {
                       "id": "call_1DNpUWV55n4Gccq28CPRJYmo",
                       "type": "function",
                       "function": {
                           "name": "get_weather",
                           "arguments": "{\"location\":\"San Jose, CA\"}"
                       }
                   }
               ]
           },
           {
               "role": "tool",
               "content": "{\"temperature\":\"22\",\"unit\":\"celsius\",\"description\":\"Sunny\"}",
               "tool_call_id": "call_1DNpUWV55n4Gccq28CPRJYmo"
           }
       ],
       "model": "gpt-3.5-turbo-1106",
       "temperature": 1,
       "top_p": 1,
       "n": 1,
       "frequency_penalty": 0,
       "presence_penalty": 0,
       "tool_choice": "none",
       "tools": [
           {
               "type": "function",
               "function": {
                   "name": "get_weather",
                   "description": "Determine weather in my location.",
                   "parameters": {
                       "type": "object",
                       "properties": {
                           "location": {
                               "type": "string",
                               "description": "The city and state e.g. San Francisco, CA"
                           },
                           "unit": {
                               "type": "string",
                               "enum": [
                                   "c",
                                   "f"
                               ]
                           }
                       },
                       "required": [
                           "location"
                       ]
                   }
               }
           }
       ]
   }

5. The GPT model then generates the final output based on all provided information.

   [
       {
           "index": 0,
           "message": {
               "role": "assistant",
               "content": "The weather in San Jose tomorrow will be sunny with a temperature of 22°C."
           },
           "logprobs": null,
           "finish_reason": "stop"
       }
   ]

   

Worth noting

• The GPT model itself does not execute functions.

• In the latest API documentation, parameters like functions/function_call have been replaced with tools/tool_choice.

• In the second request, avoid specifying a function name in tool_choice to prevent potential recursive calls.

• Function-related inputs count towards token billing, so minimize descriptions for cost efficiency.

• Implement a secondary confirmation process for sensitive actions (e.g., sending emails on someone's behalf).

• Not all GPT models support Function Calling; some, like gpt-3.5-turbo-0301 and gpt-4-0314, do not.

• Currently, models like gpt-4-1106-preview and gpt-3.5-turbo-1106 support Parallel Function Calling.

Summary

This post offers a detailed guide to implementing Function Calling with OpenAI GPT models. From understanding the rationale behind this feature to executing practical steps and examples, it equips developers with the knowledge to enhance their applications beyond the traditional capabilities of GPT models, addressing real-time data needs and expanding functional boundaries.