Guidance on ETL Pipeline from Google Skill Boost to BigQuery

[purna05]

Hi All,

I’m working on a proof of concept (PoC) to extract data from Google Skill Boost and load it into BigQuery. I’d like to know if anyone has successfully implemented a similar ETL pipeline.

Specifically, I’m looking for guidance on:

• The best approach to fetch data from Google Skill Boost (API, export options, etc.).

• Any challenges faced during data extraction and transformation.

• Key requirements to ensure a smooth pipeline.

• Recommendations for tools (e.g., Cloud Functions, Dataflow, Airflow) to orchestrate the ETL process.

If you’ve achieved this or have suggestions, I’d appreciate your insights!

Thanks in advance!