Solved: Gemini 1.5 parallel function calls

germanebr · 06-25-2024 09:21 AM

Hi everybody!

I've been reading the documents and samples regarding the new feature of Gemini 1.5 models for parallel function calling and how that helps to get all multiple tool calls with one single API call.

Now that I've tested it a bit with the samples from here, I was able to give a tool to the LLM and extract the function calls:

def extract_function_calls(response:GenerationResponse) -> List[Dict]:
    function_calls = []

    if response.candidates[0].function_calls:
        for function_call in response.candidates[0].function_calls:
            function_call_dict = {function_call.name: {}}
            
            for key,value in function_call.args.items():
                function_call_dict[function_call.name][key] = value

            function_calls.append(function_call_dict)

    return function_calls

llm = GenerativeModel("gemini-1.5-flash-001",
                      generation_config = GenerationConfig(temperature = 0),
                      safety_settings = safety_settings,
                      tools = [add_numbers_tool])
chat = llm.start_chat()
prompt = "Give me the results of adding 3 and 4, 10 and 123, and 54 with 291."
response = chat.send_message(prompt)
function_calls = extract_function_calls(response)

[{'add_numbers': {'y': 4.0, 'x': 3.0}},
 {'add_numbers': {'y': 123.0, 'x': 10.0}},
 {'add_numbers': {'y': 291.0, 'x': 54.0}}]

However, I have a question regarding this feature. Even if I don't manually ask for the parallel function calls (in this case, running my extract_function_calls code), does the Gemini model do that by itself when using the generate_content or the generate_content_async function?

Thanks in advance!

ruthseki

Hi @germanebr,

While Gemini 1.5 Flash is incredibly powerful and capable of integrating with external tools, it doesn't automatically handle parallel function calls. The generate_content and generate_content_async functions, as currently implemented, are primarily for generating text and don't have built-in mechanisms for executing multiple tool calls simultaneously.

Here's what you need to consider:

Parallelism and Tools: Gemini 1.5 Flash's tool integration is designed to enhance its reasoning and response capabilities. While it can invoke multiple tools within a single API request, the execution of these tools is sequential, not truly parallel.

Manual Function Extraction: The code extract_function_calls was necessary to identify and separate function calls from the main prompt before sending them to the model. This is not automatically handled in the current iteration of Gemini.

Future Possibilities: The field of AI and large language models is constantly evolving. While not currently available, the ability to automatically handle parallel tool calls in Gemini could be a potential future development.

In essence, while Gemini 1.5 Flash is remarkably capable, it doesn't inherently provide automatic parallel function call execution. You'd still need to manage the calls and their execution, either through your code or by carefully structuring your prompts to ensure the model understands the desired order of tool interactions.

If you're looking for truly parallel execution, you'll likely need to explore specialized tools and libraries designed for concurrency.

I hope this helps

View solution in original post

ruthseki