Gemini API finishReason RECITATION

[MihaminaRKTMB]

Hi all,

I use Gemini through its API on https://generativelanguage.googleapis.com/v1beta/models/gemini-pro:generateContent

I sometimes get 0 candidates, with a finishReason = RECITATION.

What should be the parameters in the POST query if I dont want it to stop for that reason but still answer?

I see that there is a Github Issue about that, here: https://github.com/google/generative-ai-docs/issues/257

Thank you!

[MihaminaRKTMB]

If I prompt Gemini via the API with this, I hit the RECITATION issue:

There is a database that can be reached at the following address:
host: localhost
username: root
password: root
database: test
Write a PHP function that connects to that database and create a table called "users" with the following fields:
id (int, auto increment, primary key)
username (varchar, 255)
password (varchar, 255)
email (varchar, 255)

But if I paste it in the Gemini website, I get my answer : https://gemini.google.com/app/11a4c6b2bdcf6b7f

I think this is unfair...

[aca667]

stream the response if you are not already, it will detect way less recitation than if you get it all in once

[Nayjesy]

Same,
With simple queries like

Count from 1 to 100, print numbers as words

or

List 30 cities

It fails with 

vertexai.generative_models._generative_models.ResponseValidationError: The model response did not completed successfully.
Finish reason: 4.

Gemini Ultra is totally unuseable

[itsaaiml]

I'm also encountering the block reason RECITATION error when using gemini-1.0-pro-001; setting: BOCK_NONE with temperature=0.2 & t-top=0.6. I've narrowed down the issue to a specific day of data. Interestingly, I was able to get the needed response by:

Adjusting the number of JSON objects sent
Modifying multiprocessing settings
Running the code repeatedly for that day
However, this seems like a workaround, and the root cause of the error remains unclear.

[JCarrCES]

Same issue here by simply asking it to explain a simple concept. Do I need to stream responses for simple chat interactions?

[victortong]

Same case here. I asked a question "What is information security?". Then, recitation error.

[Nayjesy]

I've ceased attempts to utilize Google language models through Vertex AI as they proved to be non-functional.

+ No feedback from google

[omarbadran]

Did you manage to get anywhere with this? I'm trying to build RAG on top of this, but it keeps showing this error randomly. It's so frustrating.

[Nayjesy]

Nope, my company just stopped trying to use Vertex AI. It was a terrible experience.

[victortong]

Same case here.

[gaussfer]

Still facing the same issue with gemini-1.5-flash!

[victortong]

Yes, very unstable. I used a lot json object output. response out contains extract "," a lot of time and made a invalid json format. Then, my program crashed. So, annoying. However, it is smarter than GPT-4o mini a little bit. So, I still keep using gemini-1.5 flash. 

[dan7]

Same here. As soon as the gemini-1.5-flash model flipped to production, it all my prompts broke and are returning this Recitation error. Am asking the model to parse and categorize info from the text of a job description, but it fails ~95% of the time because it finds web sources with part or all of the job description in its training data. This is insane behavior. Has anyone from this site actually gotten Google to help with similar issues?

[victortong]

Not only recitation. Exact "," in JSON is another big problem.

[dan7]

For anyone else experiencing this issue, it appears there is an open issue here: https://issuetracker.google.com/issues/331677495. Please +1 it to help drive attention.

Unfortunately, while it is assigned, it's been open 2 months and does not appear to have any updates from google posted to it. So, not sure if we can expect a resolution any time soon. I may just have to go the way of @Nayjesy and abandon gemini

[victortong]

Good suggestion. I just +1.