Gemini Pro function call tooling errors - "I am sorry, I cannot fulfill this request"

[paulotaylor]

I'm using the gemini-pro model with the function calling functionality using the Vertex AI Java SDK.

I run the same prompt that is supposed to call the function but 1 out of 5 tries Gemini does not call the function but instead it replies: I am sorry, I cannot fulfill this request. The available tools lack the desired functionality.

It's not even an error or anything, it simply replies "I am sorry, I cannot fulfill this request. The available tools lack the desired functionality." which is not a nice output to present to users

Is anyone experiencing  this behavior?

[blicht10069]

I have found that on its own Gemini does not invoke the tools.  Even if you tell it the exact name of the tool to invoke, about 50% of the time it will give this reply:

Notice that its missing the function name!  It also sometimes mangles the function name, leaving off the last few characters of the name.  Other times it passes in empty args.  Of course, sometimes, it does it exactly right.  

This is a very tricky problem to solve with AI.  Part of the issue is that the AI doesn't seem to be fully aware of the function names. If I ask it to list the names of all functions (tools) it knows about, it will tend to try to call a function (but it will pass an empty name and empty args).

I with there was a little more transparency on this.

"candidates": [
{
"content": {
"parts": [
{
"functionCall": {
"args": {
"url": "https://someurl/somepage.html"
}
}
}
],
"role": "model"
},
"finishReason": "STOP",
"index": 0
}
],

[ShaneMiller]

Did anyone make progress on this issue?  I've encountered something similar, except Gemini is trying to call a function named "describe" and is passing a parameter called "extension" with the value of "default_api".  I defined the functions it should be calling and none of them are named "describe".  describe {'extension': 'default_api',

[stephenriley]

I just got a response that listed exactly how it would call my registered functions, but it didn't actually invoke them (by returning function_call objects in the response).  Instead, it just made up return values itself!  Pretty weird.  I even have the temperature set to 0.0.

If you figure out how to consistently get it to behave properly, please let us know!