Hey y'all. Pretty new to Gemini, so pardon ahead of time if I'm just being an idiot. I do that sometimes.
Problem: gemini-1.5-flash response from API calls are consistently much lower quality than response from AI Studio, and this is blocking me from migrating from GPT to Gemini
Summary: I've been absolutely blown away by the quality and consistency of the responses I've gotten from gemini-1.5-flash in AI Studio so far. I'm at the point where I would love to move my project from OAI over to gemini as a result! Unfortunately, when I run the exact same prompt (copy-paste) in my API, the results are pretty terrible. Specifically, the response the API sends me it is not following the directions correctly. it ignores constraints, and returns substantially less comprehensive analysis, that seems to degrade in quality/consistency as the result is processed.
What I've tried so far:
- Verified both modalities are pointing to the same model (gemini-1.5-flash-latest)
- Verified that the temperature is exactly the same as I've set in AI studio (0.25)
- Duplicated other parameters (see example below) from AI-Studio to my API
Example Prompt & Responses (Link)
(Possibly) Relevant Context:
- I only have one Google Cloud account, and one project set up. It is currently set up as Paid
- The request is ~8k tokens, and is sent as stringified JSON (text) that includes basic instructions, source data, and a JSON structure for gemini to use in it's response (~7k tokens).
- The request is self-contained, as It references no objects outside the text in the JSON I'm sending.
- My API call is a simple http POST request, as follows:
{
"contents": {
"role": "user",
"parts": {
"text": <my-prompt>
}
},
"generation_config": {
"temperature": 0.25,
"top_p": 0.95,
"top_k": 64,
"max_output_tokens": 8192,
"response_mime_type": "application/json"
}
}
LinkedIn
Twitter
