I’m encountering an issue where I receive a limit error when running a query for the first time. For instance, if my query is “<medical conditions> symptoms”, it may trigger the error. However, when I execute the same query a second time, it works without any issues.
Another problem I’m facing is that once this error occurs, I have to restart the kernel. If I don’t, I continue to receive the same error regardless of my input.
In terms of token count, the ‘aspects list’ contains approximately 100 words, which translates to about 250-300 tokens. Assuming the rest of the query comprises another 100 tokens, the maximum input tokens should be around 500, and the output tokens should be about 100. Therefore, I’m unsure as to why I’m seeing a count of over 30,000 tokens.
from vertexai.generative_models import (
GenerativeModel,
HarmCategory,
HarmBlockThreshold,
)
model = GenerativeModel("gemini-1.0-pro")
config = {
"max_output_tokens": 128,
"temperature": 0,
"top_p": 1,
}
safety_settings={
HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_ONLY_HIGH,
HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_ONLY_HIGH,
HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_ONLY_HIGH,
HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_ONLY_HIGH,
}
chat = model.start_chat()
msg=f'''Here are the list of Aspects: {aspects_list}. What aspects should I suggest for the user query from the given aspects list. Find the medical term or disease in query but ignore negations. Give me output like, [{{'Topic':'<medical term except negation>','Aspects':[<2 aspects>]}}]. The input query is {query}'''
chat.send_message(msg, generation_config=config,safety_settings=safety_settings)
LinkedIn
Twitter
