Knowledge-Sharing
In-Depth Analysis: OpenAI API 1106 Updates - Parallel Function Calling
In-Depth Analysis: OpenAI API 1106 Updates - Parallel Function Calling
Dec 4, 2023
Introduction to the Latest Updates
Just a month ago, on November 6th, 2023, OpenAI unveiled its latest set of exhilarating updates, introducing significant changes to the Create chat completion endpoint. These updates have not only revamped the legacy function calling but have also introduced new features like seed parameter to enable reproducible outputs, and JSON mode for response format.
This article dives deep into the specifics of these function calling changes, focusing particularly on the transition from function_call/functions to tool_choice/tools. This shift marks a significant stride towards more comprehensive and versatile API functionality.
Reasons Behind the Change
The major reasons for such change are:
Broader Tool Integration:
toolsbroadens potential integrations, paving the way for various tools like DALL-E, web browsers, and code interpreters. This opens doors to a versatile, future-ready API.Parallel Function Calling: The highlight of this update is parallel function calling, allowing simultaneous execution of multiple function calls, thereby boosting efficiency and cutting down response times.
Legacy vs New Approach
Previously, function_call determined which function the model would call, offering options like none, auto, and specifying a function. functions provided a list of callable functions.
Example - Fetching weather information:
{
"stream": false,
"function_call": "auto",
"frequency_penalty": 0,
"messages": [
{
"content": "You are a helpful assistant.",
"role": "system"
},
{
"content": "what\'s the weather in San Jose?",
"role": "user"
}
],
"model": "gpt-3.5-turbo",
"seed": 12345,
"temperature": 1,
"presence_penalty": 0,
"top_p": 1,
"n": 1,
"functions": [
{
"name": "get_current_weather",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "The city and state, e.g. San Francisco"
}
},
"required": [
"city"
]
},
"description": "Get the current weather"
}
]
}Response from OpenAI included a singular function_call
{
"id": "chatcmpl-xxxxxxxxxxxx",
"object": "chat.completion",
"created": 1701336426,
"model": "gpt-3.5-turbo-0613",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": null,
"function_call": {
"name": "get_current_weather",
"arguments": "{\n \"city\": \"San Jose\"\n}"
}
},
"finish_reason": "function_call"
}
],
"usage": {
"prompt_tokens": 71,
"completion_tokens": 17,
"total_tokens": 88
},
"system_fingerprint": null
}The backend program then could call get_current_weather along with the provided arguments to get the proper function result.
The new approach introduces two fields:
tools: An adaptable array listing various tools, currently supporting functions.
tool_choice: Mirroring
function_call, this governs the tool usage by the model.
Example - Morning briefing (weather, news, schedule):
{
"messages":
[
{
"role": "system",
"content": "You are a helpful assistant."
},
{
"role": "user",
"content": "Provide a morning briefing including weather in San Jose, news, and personal schedule."
}
],
"model": "gpt-3.5-turbo-1106",
"temperature": 1,
"top_p": 1,
"n": 1,
"frequency_penalty": 0,
"presence_penalty": 0,
"tool_choice": "auto",
"tools":
[
{
"type": "function",
"function":
{
"name": "get_current_weather",
"description": "Get the current weather in a given city.",
"parameters":
{
"type": "object",
"required":
[
"city"
],
"properties":
{
"city":
{
"type": "string",
"description": "The city e.g. San Francisco"
}
}
}
}
},
{
"type": "function",
"function":
{
"name": "get_top_news",
"description": "Fetch today's top news headlines from a specified news source.",
"parameters":
{
"type": "object",
"required":
[
"source"
],
"properties":
{
"source":
{
"type": "string",
"description": "News source e.g. BBC, CNN"
}
}
}
}
},
{
"type": "function",
"function":
{
"name": "get_calendar_schedule",
"description": "Retrieve today's personal calendar schedule.",
"parameters":
{
"type": "object",
"required":
[
"user_id"
],
"properties":
{
"user_id":
{
"type": "string",
"description": "Unique identifier for the user"
}
}
}
}
}
]
}OpenAI's response now comprises a list of tool_calls.
{
"role": "assistant",
"content": null,
"tool_calls": [
{
"id": "call_UZqrVFqwscEKjn4u1m8Fp1rl",
"type": "function",
"function": {
"name": "get_current_weather",
"arguments": "{\"city\": \"San Jose\"}"
}
},
{
"id": "call_JsyOgiluzCXNRF3ZzK8f1FrU",
"type": "function",
"function": {
"name": "get_top_news",
"arguments": "{\"source\": \"CNN\"}"
}
},
{
"id": "call_MckHUtWfc97t6A9ym0VyBTx9",
"type": "function",
"function": {
"name": "get_calendar_schedule",
"arguments": "{\"user_id\": \"12345\"}"
}
}
]
}Parallel Function Calling in Action
With this response structure, backend programs can simultaneously call multiple functions and relay the results to OpenAI for the final output as below.
This showcases the efficiency of handling multiple tasks in one go. Please note that, only the latest models (GPT-4-1106-Preview and GPT-3.5-Turbo-1106) support parallel function calling. Older models still process a single function call per response.
Wrapping Up
The great news is that Prompter now supports parallel function calling, enhancing user experience with more complex and varied interactions as demonstrated in the screenshots. We invite you to explore and enjoy these new capabilities to their fullest.
Introduction to the Latest Updates
Just a month ago, on November 6th, 2023, OpenAI unveiled its latest set of exhilarating updates, introducing significant changes to the Create chat completion endpoint. These updates have not only revamped the legacy function calling but have also introduced new features like seed parameter to enable reproducible outputs, and JSON mode for response format.
This article dives deep into the specifics of these function calling changes, focusing particularly on the transition from function_call/functions to tool_choice/tools. This shift marks a significant stride towards more comprehensive and versatile API functionality.
Reasons Behind the Change
The major reasons for such change are:
Broader Tool Integration:
toolsbroadens potential integrations, paving the way for various tools like DALL-E, web browsers, and code interpreters. This opens doors to a versatile, future-ready API.Parallel Function Calling: The highlight of this update is parallel function calling, allowing simultaneous execution of multiple function calls, thereby boosting efficiency and cutting down response times.
Legacy vs New Approach
Previously, function_call determined which function the model would call, offering options like none, auto, and specifying a function. functions provided a list of callable functions.
Example - Fetching weather information:
{
"stream": false,
"function_call": "auto",
"frequency_penalty": 0,
"messages": [
{
"content": "You are a helpful assistant.",
"role": "system"
},
{
"content": "what\'s the weather in San Jose?",
"role": "user"
}
],
"model": "gpt-3.5-turbo",
"seed": 12345,
"temperature": 1,
"presence_penalty": 0,
"top_p": 1,
"n": 1,
"functions": [
{
"name": "get_current_weather",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "The city and state, e.g. San Francisco"
}
},
"required": [
"city"
]
},
"description": "Get the current weather"
}
]
}Response from OpenAI included a singular function_call
{
"id": "chatcmpl-xxxxxxxxxxxx",
"object": "chat.completion",
"created": 1701336426,
"model": "gpt-3.5-turbo-0613",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": null,
"function_call": {
"name": "get_current_weather",
"arguments": "{\n \"city\": \"San Jose\"\n}"
}
},
"finish_reason": "function_call"
}
],
"usage": {
"prompt_tokens": 71,
"completion_tokens": 17,
"total_tokens": 88
},
"system_fingerprint": null
}The backend program then could call get_current_weather along with the provided arguments to get the proper function result.
The new approach introduces two fields:
tools: An adaptable array listing various tools, currently supporting functions.
tool_choice: Mirroring
function_call, this governs the tool usage by the model.
Example - Morning briefing (weather, news, schedule):
{
"messages":
[
{
"role": "system",
"content": "You are a helpful assistant."
},
{
"role": "user",
"content": "Provide a morning briefing including weather in San Jose, news, and personal schedule."
}
],
"model": "gpt-3.5-turbo-1106",
"temperature": 1,
"top_p": 1,
"n": 1,
"frequency_penalty": 0,
"presence_penalty": 0,
"tool_choice": "auto",
"tools":
[
{
"type": "function",
"function":
{
"name": "get_current_weather",
"description": "Get the current weather in a given city.",
"parameters":
{
"type": "object",
"required":
[
"city"
],
"properties":
{
"city":
{
"type": "string",
"description": "The city e.g. San Francisco"
}
}
}
}
},
{
"type": "function",
"function":
{
"name": "get_top_news",
"description": "Fetch today's top news headlines from a specified news source.",
"parameters":
{
"type": "object",
"required":
[
"source"
],
"properties":
{
"source":
{
"type": "string",
"description": "News source e.g. BBC, CNN"
}
}
}
}
},
{
"type": "function",
"function":
{
"name": "get_calendar_schedule",
"description": "Retrieve today's personal calendar schedule.",
"parameters":
{
"type": "object",
"required":
[
"user_id"
],
"properties":
{
"user_id":
{
"type": "string",
"description": "Unique identifier for the user"
}
}
}
}
}
]
}OpenAI's response now comprises a list of tool_calls.
{
"role": "assistant",
"content": null,
"tool_calls": [
{
"id": "call_UZqrVFqwscEKjn4u1m8Fp1rl",
"type": "function",
"function": {
"name": "get_current_weather",
"arguments": "{\"city\": \"San Jose\"}"
}
},
{
"id": "call_JsyOgiluzCXNRF3ZzK8f1FrU",
"type": "function",
"function": {
"name": "get_top_news",
"arguments": "{\"source\": \"CNN\"}"
}
},
{
"id": "call_MckHUtWfc97t6A9ym0VyBTx9",
"type": "function",
"function": {
"name": "get_calendar_schedule",
"arguments": "{\"user_id\": \"12345\"}"
}
}
]
}Parallel Function Calling in Action
With this response structure, backend programs can simultaneously call multiple functions and relay the results to OpenAI for the final output as below.
This showcases the efficiency of handling multiple tasks in one go. Please note that, only the latest models (GPT-4-1106-Preview and GPT-3.5-Turbo-1106) support parallel function calling. Older models still process a single function call per response.
Wrapping Up
The great news is that Prompter now supports parallel function calling, enhancing user experience with more complex and varied interactions as demonstrated in the screenshots. We invite you to explore and enjoy these new capabilities to their fullest.
Introduction to the Latest Updates
Just a month ago, on November 6th, 2023, OpenAI unveiled its latest set of exhilarating updates, introducing significant changes to the Create chat completion endpoint. These updates have not only revamped the legacy function calling but have also introduced new features like seed parameter to enable reproducible outputs, and JSON mode for response format.
This article dives deep into the specifics of these function calling changes, focusing particularly on the transition from function_call/functions to tool_choice/tools. This shift marks a significant stride towards more comprehensive and versatile API functionality.
Reasons Behind the Change
The major reasons for such change are:
Broader Tool Integration:
toolsbroadens potential integrations, paving the way for various tools like DALL-E, web browsers, and code interpreters. This opens doors to a versatile, future-ready API.Parallel Function Calling: The highlight of this update is parallel function calling, allowing simultaneous execution of multiple function calls, thereby boosting efficiency and cutting down response times.
Legacy vs New Approach
Previously, function_call determined which function the model would call, offering options like none, auto, and specifying a function. functions provided a list of callable functions.
Example - Fetching weather information:
{
"stream": false,
"function_call": "auto",
"frequency_penalty": 0,
"messages": [
{
"content": "You are a helpful assistant.",
"role": "system"
},
{
"content": "what\'s the weather in San Jose?",
"role": "user"
}
],
"model": "gpt-3.5-turbo",
"seed": 12345,
"temperature": 1,
"presence_penalty": 0,
"top_p": 1,
"n": 1,
"functions": [
{
"name": "get_current_weather",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "The city and state, e.g. San Francisco"
}
},
"required": [
"city"
]
},
"description": "Get the current weather"
}
]
}Response from OpenAI included a singular function_call
{
"id": "chatcmpl-xxxxxxxxxxxx",
"object": "chat.completion",
"created": 1701336426,
"model": "gpt-3.5-turbo-0613",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": null,
"function_call": {
"name": "get_current_weather",
"arguments": "{\n \"city\": \"San Jose\"\n}"
}
},
"finish_reason": "function_call"
}
],
"usage": {
"prompt_tokens": 71,
"completion_tokens": 17,
"total_tokens": 88
},
"system_fingerprint": null
}The backend program then could call get_current_weather along with the provided arguments to get the proper function result.
The new approach introduces two fields:
tools: An adaptable array listing various tools, currently supporting functions.
tool_choice: Mirroring
function_call, this governs the tool usage by the model.
Example - Morning briefing (weather, news, schedule):
{
"messages":
[
{
"role": "system",
"content": "You are a helpful assistant."
},
{
"role": "user",
"content": "Provide a morning briefing including weather in San Jose, news, and personal schedule."
}
],
"model": "gpt-3.5-turbo-1106",
"temperature": 1,
"top_p": 1,
"n": 1,
"frequency_penalty": 0,
"presence_penalty": 0,
"tool_choice": "auto",
"tools":
[
{
"type": "function",
"function":
{
"name": "get_current_weather",
"description": "Get the current weather in a given city.",
"parameters":
{
"type": "object",
"required":
[
"city"
],
"properties":
{
"city":
{
"type": "string",
"description": "The city e.g. San Francisco"
}
}
}
}
},
{
"type": "function",
"function":
{
"name": "get_top_news",
"description": "Fetch today's top news headlines from a specified news source.",
"parameters":
{
"type": "object",
"required":
[
"source"
],
"properties":
{
"source":
{
"type": "string",
"description": "News source e.g. BBC, CNN"
}
}
}
}
},
{
"type": "function",
"function":
{
"name": "get_calendar_schedule",
"description": "Retrieve today's personal calendar schedule.",
"parameters":
{
"type": "object",
"required":
[
"user_id"
],
"properties":
{
"user_id":
{
"type": "string",
"description": "Unique identifier for the user"
}
}
}
}
}
]
}OpenAI's response now comprises a list of tool_calls.
{
"role": "assistant",
"content": null,
"tool_calls": [
{
"id": "call_UZqrVFqwscEKjn4u1m8Fp1rl",
"type": "function",
"function": {
"name": "get_current_weather",
"arguments": "{\"city\": \"San Jose\"}"
}
},
{
"id": "call_JsyOgiluzCXNRF3ZzK8f1FrU",
"type": "function",
"function": {
"name": "get_top_news",
"arguments": "{\"source\": \"CNN\"}"
}
},
{
"id": "call_MckHUtWfc97t6A9ym0VyBTx9",
"type": "function",
"function": {
"name": "get_calendar_schedule",
"arguments": "{\"user_id\": \"12345\"}"
}
}
]
}Parallel Function Calling in Action
With this response structure, backend programs can simultaneously call multiple functions and relay the results to OpenAI for the final output as below.
This showcases the efficiency of handling multiple tasks in one go. Please note that, only the latest models (GPT-4-1106-Preview and GPT-3.5-Turbo-1106) support parallel function calling. Older models still process a single function call per response.
Wrapping Up
The great news is that Prompter now supports parallel function calling, enhancing user experience with more complex and varied interactions as demonstrated in the screenshots. We invite you to explore and enjoy these new capabilities to their fullest.