Stripe to BigQuery using GCP Application Integration Services

[blackcoffer]

Hi,
I wish to sync my data from Stripe to Bigquery. I am using GCP Application Integration Services and need some help with setting up the system for the same. I have created the Stripe connector as mentioned in the documents and am succesfully able to see the json output at the ConnectorOutputLoad. Now I wish to sync the output data to a Bigquery table. Let me know if you can help with this
Thanks!

[brijeshpanara]

Hi @blackcoffer, You can refer to Store Shopify Data to BigQuery using Shopify Connector in order to sync your data from stripe to Bigquery, only change would be to use stripe connector output that you already created instead of Shopify connector output. Let us know if you face any difficulty or need additional help. Thanks

[blackcoffer]

Hi @brijeshpanara ,
Thanks for the response. So I have actually gone through the Shopify Bigquery discussion. But the thing is if I'm using lets say a Schedule Trigger and then a Stripe Connector and then the Bigquery Connector, without using any API trigger, how do I setup the for each loop task in between because it asks for a API Trigger ID as an input variable I believe in the subintegration section?

[pramodvallanur]

Hi @blackcoffer 
1. Based on your use case, the main integration will have a scheduled trigger that will run as per your schedule
2. This main integration will have a connector to read data from Stripe and have a for loop to invoke a sub-integration
3. Your sub integration (pls create a new integration) will have an API Trigger / Private Trigger

4. This subintegration will have a data mapping task and the Big Query connector to insert into BQ

5. Please publish the subintegration

6. Come back to the main integration, in the for loop task, now you can set to the API trigger of the subintegration and pass the Stripe data

7. You can publish the main integration now

8. Based on the schedule it will run and be executed.

If you want additional info, dont hesitate to direct message me. 

[blackcoffer]

HI @pramodvallanur , 
Thanks a lot for the above steps . So i did follow your steps, but now am stuck at an error which I think is refering to some problem with my subintegration which I created following your steps. For creating the Stripe and Bigquery connectors, I have selected both (Stripe and Bigquery Connectors) with 'Entities' type config with 'List' operation

[ajayblackcoffer]

Hi ,
Can someone please help me with this?

Thanks

[brijeshpanara]

Based on the error message you shared, It seems like the variable types are mismatching between the variable you are passing to sub-integration from your main integration and variable you have configured in your sub-integration. 

Please refer to https://cloud.google.com/application-integration/docs/invoke-sub-int-for-each-loop for detailed explanation. Please let us know if it still does not help. Thanks.

[blackcoffer]

Hi @brijeshpanara ,
So I have tried the complete process again making sure to keep the variables assigned are of appropriate types and followed through the Shopify to Bigquery forum for this as well, but unfortunately found no success. Let me know how can we resolve this.
Thanks

[fengwan]

I don't think we can support batch upload to Bigquery? @brijeshpanara correct me if I'm wrong. So then you have to create another integration with Api trigger + Bigquery connector.