Firestore reading is not always successful.

[hpirttin]

I am using cloud functions Gen2 in Cloud Run. The trigger is pub/sub. I am trying to read device information from Firestore. This works about 90% of the time, but fails too often. I get the following error message: Error: 4 DEADLINE_EXCEEDED: Deadline Exceeded. Here is how I try to read data from firestore.

async function readDeviceData(deviceId) {
  try {
    const devRef = firestore.collection(DEVICE_COLLECTION_NAME).doc(deviceId);
    var doc = await devRef.get();
    if (doc.exists) {
      return Promise.resolve(doc.data());
    } else {
      return Promise.resolve();
    }
  } catch (error) {
    console.log("ERROR: READ", error);
    return Promise.resolve();
  }
}

The failed read commands occurred around 10:00 PM (see image -> Request latencies). There are several failed read commands during that time, at other times it works OK. What could be the cause? I use a similar read command on the Gen1 side and have had no problems. Should I move this to Gen1?

  

[Joy_S]

Hi @hpirttin,

Welcome to Google Cloud Community!

The error message DEADLINE_EXCEEDED indicates that your Cloud Function is taking longer than its configured timeout limit to complete the Firestore read operation. Below are the possible causes for the error message and their workarounds:

• Firestore Latency: The increased failure rate around 10:00 PM suggests a potential increase in latency within the Firestore service at that time. This could be due to higher overall traffic on Firestore, or perhaps some background processes running within GCP that might temporarily impact performance. You can check the Google Cloud Service Health Dashboard and the Firebase Status Dashboard to see if there were any reported incidents around that time.
• Function Timeout: Cloud Functions Gen2 for event-driven triggers (like Pub/Sub) have a default timeout of 9 minutes. It's possible that under certain conditions, especially around 10:00 PM, your function's execution, including the Firestore read, is exceeding this limit. You can try increasing the timeout duration for your Cloud Functions Gen2 instance. You can do this through the Google Cloud Console or using the gcloud functions deploy command with the --timeout flag . Experiment with a slightly longer timeout to see if it resolves the issue, keeping in mind the 9-minute limit for event-driven functions. You could also try to  implement a retry mechanism within your Cloud Function code. This would allow the function to automatically retry the Firestore read operation if it fails due to a transient issue like a temporary network hiccup or a brief latency spike .
• Resource Constraints in Gen2: Gen2 Cloud Functions give you more control over resources, but you might still run into limits—especially if your function is getting hit with a bunch of Pub/Sub messages at once. If it's handling several requests at the same time, it could be running low on memory. In fact, if you’re using concurrency, it’s usually a good idea to give it at least 2 GB of memory.

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.