Thank you for your contribution, I have now spent a few days trying to find out more about the error message. In part, you are also right, the error occurs when one of the chunks has no content. But, as far as I have found out with the tests, it does not have to be at the end of a text, it can also be in the middle of it. I tried it with your approach and wanted to catch the error, I also found the empty chunk, but unfortunately, the error message still comes after that. It occurs at this point in a Node_Module at this place node-modules: "@google-cloud\vertexai\src\functions\post_fetch_processing.ts:133:17 at processTicksAndRejections (node:internal/process/task_queues:95:5. The code in this file looks like this:
/**
* Reads a raw stream from the fetch response and join incomplete
* chunks, returning a new stream that provides a single complete
* GenerateContentResponse in each iteration.
* @ignore
*/
function getResponseStream(
inputStream: ReadableStream<string>
): ReadableStream<unknown> {
const reader = inputStream.getReader();
const stream = new ReadableStream<unknown>({
start(controller) {
let currentText = '';
return pump();
function pump(): Promise<(() => Promise<void>) | undefined> {
return reader.read().then(({value, done}) => {
if (done) {
if (currentText.trim()) {
controller.error(
new GoogleGenerativeAIError(
`Failed to parse final chunk of stream: ${currentText}`
)
);
return;
}
controller.close();
return;
}
currentText += value;
let match = currentText.match(responseLineRE);
let parsedResponse: unknown;
while (match) {
try {
parsedResponse = JSON.parse(match[1]);
} catch (e) {
controller.error(
new GoogleGenerativeAIError(
`Error parsing JSON response from stream chunk: "${match[1]}"`
)
);
return;
}
controller.enqueue(parsedResponse);
currentText = currentText.substring(match[0].length);
match = currentText.match(responseLineRE);
}
return pump();
});
}
},
});
return stream;
}
In the end, I still unfortunately do not know how to avoid the error / why one of the chunks (sometimes it also happens in the middle of the text) is empty. I thank you anyway for your input!
LinkedIn
Twitter
