Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

Vertex ai takes 19s when executed from nodejs

I'm experimenting with Vertex AI, and while it works perfectly (responding in under 3 seconds) when I run it from Vertex AI Studio, the same code takes 19 seconds or more when executed with Node.js. Here’s the code I’m using—any ideas on what I might be missing?

Vertex AI JavaScript Code

// Initialize Vertex with your Cloud project and location
const vertex_ai = new VertexAI({project: 'test-project', location: 'us-central1'});
const model = 'gemini-1.5-flash-002';

// Instantiate the models
const generativeModel = vertex_ai.preview.getGenerativeModel({
  model: model,
  generationConfig: {
    'maxOutputTokens': 8192,
    'temperature': 1,
    'topP': 0.95,
  },
  safetySettings: [
    {
      'category': 'HARM_CATEGORY_HATE_SPEECH',
      'threshold': 'OFF',
    },
    {
      'category': 'HARM_CATEGORY_DANGEROUS_CONTENT',
      'threshold': 'OFF',
    },
    {
      'category': 'HARM_CATEGORY_SEXUALLY_EXPLICIT',
      'threshold': 'OFF',
    },
    {
      'category': 'HARM_CATEGORY_HARASSMENT',
      'threshold': 'OFF',
    }
  ],
  tools: [
    {
      googleSearchRetrieval: {},
    },
  ],
});

async function generateContent() {
  const req = {
    contents: [
      {role: 'user', parts: [{text: `what\'s a good name for a flower shop that specializes in selling bouquets of dried flowers?\"`}]}
    ],
  };
  console.time("Generate Content Response Time");

  const response = await generativeModel.generateContent(req);
  console.timeEnd("Generate Content Response Time");
  console.log(JSON.stringify(response));

  //process.stdout.write('aggregated response: ' + JSON.stringify(await streamingResp.response));
}

generateContent();
    

 

1 REPLY 1