Vertex ai takes 19s when executed from nodejs

[matrxix007]

I'm experimenting with Vertex AI, and while it works perfectly (responding in under 3 seconds) when I run it from Vertex AI Studio, the same code takes 19 seconds or more when executed with Node.js. Here’s the code I’m using—any ideas on what I might be missing?

Vertex AI JavaScript Code

// Initialize Vertex with your Cloud project and location
const vertex_ai = new VertexAI({project: 'test-project', location: 'us-central1'});
const model = 'gemini-1.5-flash-002';

// Instantiate the models
const generativeModel = vertex_ai.preview.getGenerativeModel({
  model: model,
  generationConfig: {
    'maxOutputTokens': 8192,
    'temperature': 1,
    'topP': 0.95,
  },
  safetySettings: [
    {
      'category': 'HARM_CATEGORY_HATE_SPEECH',
      'threshold': 'OFF',
    },
    {
      'category': 'HARM_CATEGORY_DANGEROUS_CONTENT',
      'threshold': 'OFF',
    },
    {
      'category': 'HARM_CATEGORY_SEXUALLY_EXPLICIT',
      'threshold': 'OFF',
    },
    {
      'category': 'HARM_CATEGORY_HARASSMENT',
      'threshold': 'OFF',
    }
  ],
  tools: [
    {
      googleSearchRetrieval: {},
    },
  ],
});

async function generateContent() {
  const req = {
    contents: [
      {role: 'user', parts: [{text: `what\'s a good name for a flower shop that specializes in selling bouquets of dried flowers?\"`}]}
    ],
  };
  console.time("Generate Content Response Time");

  const response = await generativeModel.generateContent(req);
  console.timeEnd("Generate Content Response Time");
  console.log(JSON.stringify(response));

  //process.stdout.write('aggregated response: ' + JSON.stringify(await streamingResp.response));
}

generateContent();
    

[MarvinLlamas]

Hi @matrxix007,

Welcome to Google Cloud Community!

I understand that you are encountering latency issues when executing your code using Node.js. Possible contributing factors to this problem may include network latency and your local or standard server configuration.

Here are some potential ways to address your issue.

• Local Network Performance:  Make sure that your Node.js environment maintains a robust and stable network connection. 
• Optimize Environment: You may also want to deploy your Node.js application in a cloud VM closer to Vertex AI’s region, as this will help reduce network latency.
• Implement Caching: To avoid repeated API calls, you can use a caching mechanism. It may help speed up the execution of your code.

For further details, I recommend that you contact Google Cloud Support.

I hope the above information is helpful.