Hi there folks! Hoping to find an answer to a problem that's frustrating me to no end.
I'm leveraging OpenAPI schema definitions for generated output from gemini flash via API calls and Batch predictions. It's generally working well, however as my schemas become more complex/large I find that I am getting 400 errors with no useful context from them when running generative prompts.
The responses are 400 errors with 'INVALID_ARGUMENT' and errorDetails=undefined (which is unbelievably useless to debug).
Through a ton of anecdotal trial and error I've determined that it appears there might be some kind of unpublished hard limits around the size or depth of the schema you can supply. I've dug into docs and cannot find anything indicating such -- the input token count is still relatively small (I can copy the schema and ask for the exact output with the schema inline in VertexAI freeform and it works perfectly, with ~4000 input tokens). The JSON serialized schema is only about 8k characters long.
A few notes:
- My schemas are using deeply nested structures (~15 levels deep in some case)
- There's a combination of arrays, objects, and lots of string ENUM values in my schema.
- As noted above if I arbitrarily remove some of the schema it will work, but it doesn't really matter which parts I remove explicitly, so I know there's not an error in my schema.
- I've run my schema through multiple validators and checked to verify I'm only using the subset of supported properties.
Thanks in advance anybody who can help!
LinkedIn
Twitter
