Influence Looker’s Roadmap and join the BI Customer Council (BICC)
Would you like to influence Google Cloud’s BI roadmap more? I’d like to invite you to join the Google Cloud BI...
Data Analytics
Want more from your data? Have questions about products like BigQuery, Dataflow, or Pub/Sub? You're in the right place. Join conversations around data warehouse modernization, streaming analytics, business intelligence, and more.
Focused on Looker? Engage with the Looker Community here.
Focused on Looker? Engage with the Looker Community here.
Forum Posts
Spike in BQ billing
I have sudden spike in streaming insert sku cost increased suddenly.How to find the reason behind the issue an...
Dataform workflow orchestration
I am new to dataform and i have to implement a working Dataform workflow which i have to trigger from Airflow....
BigQuery UI is slow for BigQuery Data Editors, search not working without additional role
When users have the BigQuery Data Editor role in our project, they experience terrible performance when listin...
Dataform workflow orchestration
I have set up workflow to run a specific tag in a branch as per the guide here, but I'm seeing that the workfl...
Dataform<>Airflow Run Using Tags
I am currently managing a dataform project with numerous transformations. These transformations are executed t...
Dataform write different project ID
Hi everyoneI am currently new at working with dataform in bigqueryI would like to know how I can make my sqlx ...
Dataform - additional learning
Dataform just recently went into General Availability. My team is looking to move our transformations away fro...
Connect to AWS RDS (Entrata Postgres SQL) using Composer
Hi Team,I'm trying to perform incremental load of data from AWS RDS (Entrata PostgreSQL) to Cloud SQL PostgreS...
Pivot Macro for Dataform
Hello everyone,I am currently evaluating Dataform and wondering if it is possible to dynamically pivot data us...
Incremental Table w/ Changing columns
Hi there - I use dataform to build an incremental model ("requests_snapshot") that stores snapshots on a model...
Data Profile and Data Quality Scan Results not showing in UI
Hi,I am using Dataplex to run both Data profile and Data Quality scans. I want to be able to publish these res...
Explorer Pane missing in BigQuerry
Can anyone here please help me? I'm taking a Data Analytics Course on Coursera and got to a Reading Assessment...
Hello! Help me please. I have a deck-type view, where primary header - name, summary column - Code.
Hello! Help me please. I have a deck-type view, where primary header - name, summary column - Code. Code is a ...
Dataform Plugin for neovim
Hello guys!For data engineers that like to use neovim, I created this dataform plugin with the following funct...
Dataflow reading multiple text files from cloud storage to bigquery
Hello I currently have a website on compute engine, every time a user enters data into it, it saves that custo...
Unable to connect ADO with google-dataform
I am getting below mentioned error while connecting my ADO repository with google dataform using. Even after p...
Extract BigQuery Table Definition All at Once
Is there a way to get all the Table information in our ORG all at once..These are the table info that we are j...
Deduplication if subscriber application is not idempotent and we can't change it.
If Subscriber application is legacy and not idempotent, Can we handle deduplication (in case of redelivery, ac...
BigQuery returns different results for exactly same query when using LIMIT
On runniing the exactly same query i get different results. Details:Total results are 64155since i have a limi...
BigQuery JavaScript UDFs return different output compared with standard JavaScript.
Output from Bigquery : 3540200Output from standard Javascript environment : 354021 CREATE temp FUNCTION getmes...
How to Update a field in a nested table?
Hi all, I'm currently trying to update a field in an universal analytics sessions table. The table is nested, ...
Insert/Update queries & BigQuery - Scheduled Queries, dbt, or something else?
Apart from usual data transformations, I found myself in need to create data automatically. Mostly it revolves...
Dataform: Unable to run incremental workflow
I have the following SLQX and relative table already created in my dataset:config { type: "incremental", schem...
Data Fusion Driver Issues
Hello, I am new to Data Fusion, and I am having an issue getting a pipeline up and running. I am trying to mov...
Null pointer exception when using macros in source GCS
I am using argument settings to populate as many settings in my pipeline as possible. The pipeline consumes fi...
Read Google Workspace log in Google cloud
Hi all,I need to get Google workspace events in Google Cloud so that with api I can see them for analytics sco...
Big Query Data Type
Hi Team,When I checking big query table schema in schema tab a column data type shows as integer but when I'm ...
Android management api set pub/sub for notification
Hi Team,I've been utilizing the Android Management API for managing notifications in my app, employing the pub...
Ho to reference a variable from js script within .sqlx file?
I am trying to insert a js script inside a .sqlx file and then want to reference a variable from it inside con...
Recommendations for the pull mechanism of Google pub/sub notifications
I am using EMM (Android Management) in my project. For each new enterprise, I am creating a new topic and a su...
Articles from Product Experts and Googlers
Deeper Insights with knowledge engine & dataset-level metadata in BQ
In my previous blog From Data to Metadata: Unlocking Insights with Gen AI in BigQuery, we explored how BigQuery's “Insights” feature can automatically generate metadata for your tables and columns helping to uncover hidden patterns and trends within your data. We are now taking it a step further with metadata and insights generation at the dataset level. This new capability not only describes the dataset but also provides an interactive entity relationship map that visually reveals connections between different entities.
A developer's guide for building with Anthropic’s Claude 4 models on Vertex AI
We recently announced the general availability of two new models from the latest generation of Anthropic’s Claude model family on Vertex AI: Claude Opus 4 and Claude Sonnet 4. In this blog, we’ll guide you through building with the new Claude 4 models on Vertex AI.
Introducing DeepSeek R1 Model-as-a-service on Vertex AI Model Garden
Deploying Deepseek-R1-0528, a 671B parameters model, typically necessitates at least 8x H200 GPUs for a single API request. Such extensive resources are not accessible to everyone.To make Deepseek R1 more accessible to developers, we're excited to announce that DeepSeek R1 is now available as a fully managed API on Vertex AI in Preview.
