Conditional parameter spec for hyperparameter tuning for logistic regression

[revanthps]

How to configure conditonalparameterspec for the below code where, when penalty is l2 solver should be 'sag' and when penalty is 'l1', solver should be 'saga'.

from google.cloud.aiplatform import hyperparameter_tuning as hpt

worker_pool_specs = [
        {
            "machine_spec": {
                "machine_type": "n1-standard-4",
                "accelerator_type": "NVIDIA_TESLA_K80",
                "accelerator_count": 1,
            },
            "replica_count": 1,
            "container_spec": {
                "image_uri": container_image_uri,
                "command": [],
                "args": [],
            },
        }
    ]

custom_job = aiplatform.CustomJob(
    display_name='my_job',
    worker_pool_specs=worker_pool_specs,
    labels={'my_key': 'my_value'},
)


hp_job = aiplatform.HyperparameterTuningJob(
    display_name='hp-test',
    custom_job=job,
    metric_spec={
        'loss': 'minimize',
    },
    parameter_spec={
        'C': hpt.DoubleParameterSpec(min=0.001, max=0.1, scale='log'),
        'max_iter': hpt.IntegerParameterSpec(min=4, max=128, scale='linear'),
        'penalty': hpt.CategoricalParameterSpec(values=['l1', 'l2']),
        'solver': hpt.CategoricalParameterSpec(values=['sag', 'saga'])
    },
    max_trial_count=128,
    parallel_trial_count=8,
    labels={'my_key': 'my_value'},
    )

hp_job.run()

print(hp_job.trials)

I have looked into documentations and have tried asking help from GPTs but am not able to figure it out. Could you guys help?

[ibaui]

Hi @revanthps,

Welcome to Google Cloud Community!

To configure ConditionalParameterSpec for hyperparameter tuning in Google Cloud’s Vertex AI, you need to define conditional hyperparameters based on the value of the other hyperparameter. Here’s a step-by-step guide:

• Define the main hyperparameter: These are the primary hyperparameter that will be tuned
• Set up conditional hyperparameter: Use ConditionalParameterSpec to specify the conditions under which certain hyperparameters should be used.

Conditional parameters in hyperparameter tuning allow you to specify that certain hyperparameters should only be considered when specific conditions are met. This is particularly useful when you have hyperparameters that are only relevant under certain circumstances.

In your case, you want the ‘solver’ parameter to depend on the value of the ‘penalty’ parameter. You can consider the following on how the ConditionalParameterSpec works:

• Parent Parameter: This is the parameter that determines the condition. In your example, ‘penalty’ is the parent parameter.
• Conditional Parameter Specs: These are parameters that are conditionally included based on the value of the parent parameter. For each possible value of the parent parameter, you specify a corresponding set of parameters.

For more detailed information, you can refer to the Google Cloud Vertex AI documentation on hyperparameter and the use of hyperparameter tuning.

I hope the above information is helpful.

[revanthps]

Would you be able to provide me with an example code? I am not able to figure out with the documentation.

[cplati]

Hi there, 

I have the same issue as @revanthps , I cannot figure out how to set up the ConditionalParameterSpec using the Python SDK. Would someone be able to provide an example. Thanks in advanced. 

[stuart-king]

I haven't tested this, but based on the source code the setup would be similar to this:

import google.cloud.aiplatform as aip
from google.cloud.aiplatform import hyperparameter_tuning as hpt

aip.init()

hpt_spec = {
    "optimizer": hpt.CategoricalParameterSpec(
        values=["adam", "sgd"], 
        conditional_parameter_spec={
            "momentum": hpt.DoubleParameterSpec(min=0.8, max=0.99, parent_values=["sgd"])
        }
    )
}