Error when trying to access/print response from vertexai gemini-pro

[chrissie303]

I am encountering a hillarious error, working in google cloud shell, trying to query gemini-pro vertexai (genai currently not accessible for europeans...!).

I am just taking the python code created by the testbed,

and try to run it in a python file in google cloud shell.

When trying to print the response (print(response.text, end="")) I just get an error:

It produces the error: 

"Exception has occurred: TypeError

argument of type 'Part' is not iterable
print(response.text, end="")"

Here is the code:

import vertexai

from vertexai.preview.generative_models import GenerativeModel, Part

def generate():

  model = GenerativeModel("gemini-pro")

  responses = model.generate_content(

    """in which country is new york?""",

    generation_config={

        "max_output_tokens": 2048,

        "temperature": 0.9,

        "top_p": 1

    },

    safety_settings=[],

  stream=True,

  )

  for response in responses:

      print(response.text, end="")

generate()

When I am further analyzing this, I get more info, but still can not access the response text:

response.candidates: candidate.content: 
role: "model" parts {  text: "United States" }
candidate.content.parts:  [text: "United States" ]
candidate.content.parts[0]:
text: "United States"
type(candidate.content.parts[0])
<class 'vertexai.generative_models._generative_models.Part'>

I guess this is a bug that only  occurrs in my "unique" environment in google cloud shell. How can I solve this? 

Of course exactly the same code runs perfectly fine in the colab.

How could I completely reset the google cloud shell? (I guess the problem is a collision with some other package there).

[Mohamedmhh]

i have the same problem , do you find the solution ? 

[chrissie303]

This is my workaround: 

(additionally I try several times, because sometimes no response is delivered, for no obvious reason. Trying a 2nd time soves it normally):

                model = GenerativeModel(MYVERSION)

                chat = model.start_chat(response_validation=False)

                myanswer = chat.send_message(myprompt, generation_config=config, safety_settings={

                    generative_models.HarmCategory.HARM_CATEGORY_HATE_SPEECH: generative_models.HarmBlockThreshold.BLOCK_ONLY_HIGH,

                    generative_models.HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: generative_models.HarmBlockThreshold.BLOCK_ONLY_HIGH,

                    generative_models.HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: generative_models.HarmBlockThreshold.BLOCK_ONLY_HIGH,

                    generative_models.HarmCategory.HARM_CATEGORY_HARASSMENT: generative_models.HarmBlockThreshold.BLOCK_ONLY_HIGH,

                })

                if myanswer.candidates:

                    first_candidate = myanswer.candidates[0]

                    if first_candidate.content.parts:

                        #================================================

                        #workaround for cloudshell error: this works both local and in cloudshell now:

                        myanswer1 = myanswer.candidates[0].content.parts[0]

                        # print (myanswer1)

                        myanswer1string = str(myanswer1)

                        my_answer = myanswer1string.split('"')[1]

                        # print ("my_answer: ")

                        # print (my_answer)

                        #================================================

[hassaanhameed78]

i have the same problem

[hassaanhameed78]

the issue i got it that the server unicorn or the way I run using the FASTAPI 

[xXiXx]

If anyone is facing the same issue, GenerativeModel seems to work differently when you set "stream=True" vs "stream=False". If you use True, the function model.generate_content() returns a 'generator' object which I think generates responses in text once. Once you run through the object, it is spent, and can't be processed further.

If you set to stream = false, you get GenerationResponse object back and then you can parse through that.