Tensorflow Lite .tflite model - improve latency of Google Cloud bucket retrieval object for Inferenc

[rubenszmm]

I'm following the Image Classification code at MediaPipe.

https://codepen.io/mediapipe-preview/pen/BaVZejK

My model runs successfully, it is smaller than EfficientNet. However the latency of inference I get is 5 seconds.

MediaPipe .tflite object, at:

`https://storage.googleapis.com/mediapipe-models/image_classifier/efficientnet_lite0/float32/1/effici...`

... loads instantly, with an inference time of milliseconds.

I tried with a regional bucket and fine-grained permissions but it didn't solve the problem. I am also using the https://storage.googleapis.com/xxxxxxxx/model.tflite Public URL but it didn't solve the problem. CORS file of the bucket is configured as the following:

[
{
"origin": ["https://your-example-website.appspot.com"],
"method": ["GET"],
"responseHeader": ["Content-Type"],
"maxAgeSeconds": 1
}
]

Do you have any ideas how to improve object retrieval latency to milliseconds?

[jaia]

Hello,

Thank you for contacting Google Cloud Community!

If your code allows, consider pre-loading the model asynchronously during application initialization. This avoids blocking the main thread during image classification, potentially improving perceived performance.

Regards,

Jai Ade

[rubenszmm]

I implemented it

  const initializeApp = async () => {
    await createImageClassifier();
  
    // Add click event listeners for the img elements.
    for (let i = 0; i < imageContainers.length; i++) {
      imageContainers[i].children[0].addEventListener("click", handleClick);
    }
  };
  // Initialize the application
  initializeApp();

However, the TensorFlow Lite XNNPACK delegate for CPU is still loading before inference, what does not happen with he default supported tflite model.

[jaia]

Hi,

I would suggest you to open a support ticket with Google Cloud Support for further assistance.
[1]: https://cloud.google.com/support/docs/manage-cases#creating_cases

Regards,

Jai Ade