Optimizing Cloud Function Usage for Google Prompt Optimizer Custom Metrics

[SayyorYusupov]

**Subject:** Optimizing Cloud Function Usage for Google Prompt Optimizer Custom Metrics

**Body:**

Hello Google Cloud Community,

I am currently working with the Google Prompt Optimizer, which provides support for custom metrics. The service enables users to specify a custom metric name and a corresponding Cloud Function for metric calculation, using the following arguments:

```python
custom_metric_name="custom_engagement_personalization_score" # Metric name as key in dictionary returned from Cloud Function
custom_metric_cloud_function_name="custom_engagement_personalization_metric" # Cloud Function name
```

Current Setup and Limitations

The Cloud Function itself only receives the following inputs from Google Prompt Optimizer:

- `question`: The original query.
- `response`: The output from the LLM.
- `target`: The expected answer.

However, this setup lacks flexibility because it does not provide the `custom_metric_name` as an input to the Cloud Function. This limitation makes it challenging to dynamically determine which specific metric to compute. Without this critical information, I can do one of the following:

1. All Metrics are computed Each Time and the Correct one is taken by GPO:

• Disadvantage: unnecessary computational overhead, computing each of the eval metrics available.

2. Separate Cloud Function for Each Metric:

• I could define individual Cloud Functions for each metric.
• This approach is redundant and inefficient, as the metric scripts are relatively small and do not justify the deployment of separate functions for each.

Possible Solution

Therefore I suggest:

• Adding `custom_metric_name` as Input
• Could Google Prompt Optimizer include `custom_metric_name` as an additional input to the Cloud Function? This would enable the function to compute only the requested metric, reducing computational cost and avoid redundancy.