Pub/Sub to BigQuery in different project

[xlightecx]

Hello,

I am attempting to transfer data from a Pub/Sub subscription in Project A to BigQuery in Project B. I granted BigQuery Data Editor permissions to the Pub/Sub service account in Project B, but I'm still receiving an error message that says:

...is missing permissions required to write to the BigQuery table: bigquery.tables.get, bigquery.tables.updateData.

What could be the issue? Thank you for any suggestions.

[bihagkashikar]

Hi there, a quick note to clarify this from your question -

"I granted BigQuery Data Editor permissions to the Pub/Sub service account in Project B"

Did you mean the service account is in the target project where bigquery table is?

If yes, then this should not be the case - I say follow these steps:

1. Copy the email addr of  the pub/sub service account of the  in project A.

2. Change project to Project B where your BigQuery Table is. In IAM section, Select Grant Access, Here, paste the email address of service account and grant Data editor role and save it.

Go back to Project A, and publish a sample message. this should work 🙂 I just tested above steps myself and it worked. hope this helps

*Note the data editor is a very generous role" so make sure of all permissions are really needed. 

[xlightecx]

Thank you for letting me know that it's working now.

I have one more question: which role would be more appropriate for this task?

As per the documentation, I should grant the Pub/Sub service account the roles of BigQuery Data Editor (roles/bigquery.dataEditor) and BigQuery Metadata Viewer (roles/bigquery.metadataViewer)

https://cloud.google.com/pubsub/docs/create-subscription#assign_bigquery_service_account

[bihagkashikar]

Hi @xlightecx 

Great to know it worked 🙂

re: which role would be more appropriate. The question can be answered by What operations your solution is going to perform on the BigQuery table. read the best practices here

Hope this helped 🙂 pls mark this as answered

[meerashine1995]

is this the case when we try to access the pub sub message thorough a cloud function or API or is it possible to directly load data from pub sub in project A to BQ in project B

[bihagkashikar]

@meerashine1995  it is the later - "directly load data from pub sub in project A to BQ in project B"