This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Accept
Reject
Preferences
Log in to ask a question
Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.
Log in to ask a question

Unable to generate the output up to 8192 tokens in the ouput

Posted on 07-16-2024 10:11 AM
taihuang
Bronze 1
Bronze 1
Reply posted on --/--/---- --:-- AM

I use code-bison-32k and set max_output_tokens = 8192 in my Python code. The prompt asks to geneate the output in a json format. The output is always truncated for larger output. I used a couple free token count web site and they showed the output only has around 3000 tokens. Can code-bison-32k really generate up to 8192 tokens output as advertised?

0 1 578
Topic Labels
0 Likes
1 REPLY 1