VertexAI API input token count issue

[srijan25]

I am trying to run some basic retrieval query on a video on length 38 minutes and 2 seconds. According to the documentation of Gemini-1.5, it should take around 588,000 tokens. However, when I am trying to call the API, it is returning me an error saying that "Unable to submit request because the input token count is 1525684 but model only supports up to 1000000."
My text query is: "List the animals found in the video.”

I do not understand why is it taking so many tokens.

Is there a way to troubleshoot this issue without changing the length of the video?

Thank you for the help in advance. Please let me know if any other information is required.

[gsimons-nzme]

Add some code, but it could be because you are using a batch. I had this with text embedding models and passing it multiple texts at the same time. There is an individual limit but also a limit for all the texts together.

[srijan25]

The request.json:

{
  "contents": {
    "role": "USER",
    "parts": [
      {
        "fileData": {
          "mimeType": "video/mp4",
          "fileUri": "gs://geminivideos/test.mp4"
        }
      },
      {
        "text": "List the animals found in the video."
      }
    ]
  }
}

The curl command to send the request:

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://us-central1-aiplatform.googleapis.com/v1/projects/{PROJECT_ID}/locations/us-central1/publishers/google/models/gemini-1.5-flash-preview-0514:generateContent"

I do not think I am using a batch. If any further information is required please let me know.