I would like to use the streamGenerateContent method to pass a file of any type (image, pdf, etc.) and then have the Gemini model answer a question about the image. The file would be local and not stored on CloudStorage.
Currently, in my Python notebook, I am:
I am then storing this in a JSON dictionary which I am passing to the model via an HTTP put request. This approach works fine. However, if I did want to send the data in base64 (b“<string>”) and essentially skip step 3 above, how can I do this?
Looking at the part of the above documentation which talks about the blob containing the data to be passed to the model, it says: “If possible send as text rather than raw bytes.” Here is a code example to illustrate what I mean:
import base64
import requests
with open(filename, 'rb') as f:
file = base64.b64encode(f.read()).decode('utf-8') # HOW CAN I SKIP THE DECODING STEP?
url = ... # LINK TO GEMINI EXPERIMENTAL MODEL
headers = ... # BEARER TOKEN FOR AUTHORIZATION
data = { ...
"inlineData": {
"mimeType": "image/png", # OR "application/pdf" OR ANY OTHER FILE TYPE
"data": string # HOW CAN I KEEP THIS AS A BASE64 OBJECT INSTEAD OF A STRING?
},
...
}
requests.put(url=url, json=data, headers=headers)
If the data were in bytes, this code would fail because JSON would not be able to serialize it. Alternatively, I tried using the data parameter in requests.put (data=json.dumps(data) instead of json=data), but I think this would also convert everything to a string, which wouldn’t work because I would like my input data to be in bytes format. Another possibility that I’ve seen is to use mimeType=“application/octet-stream”, but that’s not listed as a supported type in the documentation above.
Should I be using something other than JSON to make this request? Is what I’m describing even possible in Python? Any advice on how to make this work would be appreciated.