bigquery data transfer service for google ads

[Vigneshwaran]

Hi 
I am trying to setup/create a BigQuery data transfer service for google ads.
From the google documentation i know we can setup this in 2 ways.

1. using user account 

2. Service account.

1. using user account  to create a transfer using user account the documentation is really clear it has  step by step instruction, i just followed this link 

https://cloud.google.com/bigquery/docs/google-ads-transfer

in this approach while it is trying to transfer the data, the transfer job is getting failed with below error.

generic::permission_denied: Error while processing subtask: The caller does not have permission Raw error response from Google Ads API: { "error": { "code": 403, "message": "The caller does not have permission", "errors": [ { "message": "The caller does not have permission", "domain": "global", "reason": "forbidden", "debugInfo": "detail: "[ORIGINAL ERROR] generic::permission_denied: User doesn\'t have permission to access customer. Note: If you\'re accessing a client customer, the manager\'s customer id must be set in the \'login-customer-id\' header. See https://developers.google.com/google-

I know the reason for the failure, google trouble shouting  document also explain this.

i posted my question in stackoverflow.

https://stackoverflow.com/questions/77235203/google-ads-bigquery-data-transfer

here the conclusion is simple if i can't authorize my data transfer using different id(which is not used to create a transfer) the i can't go with approach. Because ,

As per my organization setup the id which is access to GCP is don't have access to google ads same way the id which has access to google ads doesn't access to GCP. 

is it not at all possible use different user to authorize google ads data transfer?

2. Service account.  For the data transfer setup using service accounts i can see only this document    https://cloud.google.com/bigquery/docs/use-service-accounts    it is not having step by step procedure as like other one(user account documentation).

which is basically missing the steps of how to ad service account to google ads.

i updated my existing data transfer's transfer owner as my service account, now i am looking for the steps to add my service account to google ads to access google ads accounts.

i can find only below link which explain how to add service account to google ads manager https://support.google.com/admanager/answer/6078734?hl=en   . do i really need to add my service account at ad manage level? is it not possible to add the service account to manager account(MCC) level?

Thanks,

Vicky...

[ms4446]

1. Do I really need to add my service account to Ad Manager level?

No, you do not need to add your service account to the Ad Manager level if you are trying to access Google Ads data. Google Ad Manager and Google Ads are two different platforms. If you're working with Google Ads, you should add the service account directly to your Google Ads account or MCC (Manager Account).

2. Is it not possible to add the service account to the manager account (MCC) level?

Yes, it is possible to add the service account to the MCC (Manager Account) level. Once you add the service account to the MCC, it will have access to all the accounts under that MCC, based on the permissions you assign.

Steps to add your service account to Google Ads (not Ad Manager):

1. Sign in to your Google Ads account.
2. Go to "Tools & Settings" > "Access and security".
3. Click on the "+" button to add a new user.
4. Enter the email address of the service account and assign it the necessary role (e.g., Standard access, Admin access, etc.).
5. Click on "Send Invitation". The service account will be added without needing to accept the invitation.

Steps to update your data transfer to use the service account in BigQuery:

1. Go to the BigQuery Data Transfer Service.
2. Click the Manage transfers tab.
3. Click the name of the data transfer that you want to update.
4. Click the Edit button.
5. Under Transfer owner, select the Service account option.
6. Select the service account that you added to Google Ads in the previous step.
7. Click the Save button.

Once you've updated your data transfer to use the service account, it should be able to access your Google Ads accounts and transfer the data to BigQuery.

[Vigneshwaran]

Thanks for the Response @ms4446 
i added my service account to google ads MCC with read only access and status is awaiting response (since we couldn't verify the invitation).

In the BigQuery data transfer settings "Edit Transfer" Page there is no option to update Transfer Owner 
below is the screen shot of the "Edit Transfer" UI

So i updated the transfer settings with below bq command 

bq update \
--transfer_config \
--update_credentials \
--service_account_name=abcdef-test-sa@abcdef-test.iam.gserviceaccount.com projects/862514376110/locations/us/transferConfigs/5dd12f26-0000-262f-bc38-089e0820fe38 \

now i can see my service account as a user in "Transfer config details" page  but i couldn't see the Transfer owner word any of this transfer config page.

i jus tested my transfer it failing with below error 

Thanks,

Vicky

[ms4446]

The "awaiting response" status you see in Google Ads after adding a service account is expected. This is because service accounts don't manually accept invitations like regular users do. It's not a matter of Google verifying the invitation; it's simply the nature of how service accounts operate within the Google Ads platform.

Regarding the "Edit Transfer" page in the BigQuery Data Transfer Service, it's true that there might not be an option in the UI to update the transfer owner. However, this doesn't necessarily mean that the transfer owner is strictly a read-only field. It's possible that there are other methods, such as API calls, to update this field. For definitive information on this, it would be best to refer to official Google documentation or reach out to Google support.

The error message "Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential." indicates an authentication issue. This could be due to a range of factors, including but not limited to:

• The service account might not have the necessary permissions in Google Ads. Ensure that it has the "Read & analyze" permission level, and depending on the operations, other permissions might be required.
• There could be issues with the service account keys or expired tokens.

Here are a few steps you can consider:

1. Double-check the permissions of the service account in both GCP and Google Ads.
2. Try updating the transfer credentials again using the bq update command.
3. Consider creating a new data transfer using the service account, but ensure it has the appropriate permissions first.
4. If you've tried the above steps and are still facing issues, contacting Google support would be a prudent next step. They can provide more specific guidance based on your setup.

[rishu8]

@Vigneshwaran , were you able to sort this out ? I am facing same issue .

[SersanTech]

@ms4446 I have same issues here. I think Google Ads and BigQuery doesn't have a proper communication system, too complicated setup the authorization. The BigQuery data transfer using Service Account, while Google Ads need Google Workspace.

Doesn't like Google Analytics and Firebase Analytics which they are making it easy for setup the integration with BigQuery.

The references :

- https://cloud.google.com/bigquery/docs/google-ads-transfer

- https://developers.google.com/google-ads/api/docs/oauth/service-accounts

Create an SA with permissions : ✅

• bigquery.transfers.update
• bigquery.datasets.get and 
• bigquery.datasets.update

Add the SA to Google Ads access with Access level "standard" and Awaiting response status. ✅

Impersonate as a User on Google workspace with Domain-wide Delegation and add some scopes ✅

• https://www.googleapis.com/auth/adwords: Grants access to Google Ads data and functionality.
• https://www.googleapis.com/auth/userinfo.email: Allows access to the user's email address.
• https://www.googleapis.com/auth/userinfo.profile: Allows access to the user's basic profile information.

Here is the log says, we got 401.

[wmitchellbeam]

I solved this by adding permission of the user setting up the job to have `roles/iam.serviceAccountUser` role on the Service account being assigned to run the job.

[wmitchellbeam]

Ok I was premature on the above. This fixed some other errors but to get rid of the oAuth errors the only solution was to raise a support case with GCP, they then contacted google ads team who, and I quote

"Kindly note that the Google Ads Engineering Team granted the admin role to the service account mentioned."

It looks like they've manually accepted the invitation somehow in the backend, which we have no access to do as it's a SA, and the docs say an invitation never needs accepting.

[tra-nguyen]

Hi @wmitchellbeam ,

We are facing the same issue right now. Could you please let me know which support email that you reach out to? I have tried google ads support, but they said it's bigquery issue. Asked GCP support and they said it's google ads. Now we are stuck in between the two. 

Thanks in advance!

[wmitchellbeam]

It was GCP support via the support section on the GCP web portal. 

They gave me a recent update (on the 30th of Sept) that said that adding service accounts to google ads should now be self service . But I haven't tested this myself.