how to count total token from gemini multi-turn

[parkjh688]

Hi

I'm using Vertex AI gemini 1.5 pro now.

chat_session = generative_multimodal_model.start_chat(history=[], response_validation=False)
user_queries = [query_1, query_2]

First query input is <image, text> pair and second query input is only text.
So I always get two responses from query_1 and query_2.

I wanna know prompt_token_count and candidates_token_count for whole user_queries.
I wonder if the token results of the last query are accumulated, or if the token results of each response must be added.

[gimaldi · Accepted solution]

They are accumulated.

I did a test with some simple queries and these are the results:

[QUERY_1: 6 tokens]
input tokens: 6
output tokens: 472
total tokens: 479

---

[QUERY_2: 8 tokens]
input tokens: 487 (previous totals, 479, + actual query, 8 = 487)
output tokens: 293
total tokens: 780 (grand total)

But if the purpose is to see the costs, then the reasoning is different. Each single completion session has its own cost. In the example I gave you, assuming that the cost of the input tokens and that of the output tokens is the same, the total cost is given by the sum of the total tokens of QUERY_1 plus the total tokens of QUERY_2.

So 479 + 780 = 1259

Ciao

[parkjh688]

Thanks!