Data is available for real-time analysis using GoogleSQL queries immediately after BigQuery successfully acknowledges a tabledata.insertAll request.
Recently streamed rows to an ingestion time partitioned table temporarily have a NULL value for the _PARTITIONTIME pseudo column. For such rows, BigQuery assigns the final non-NULL value of the PARTITIONTIME column in the background, typically within a few minutes. In rare cases, this can take up to 90 minutes.
Some recently streamed rows might not be available for table copy typically for a few minutes. In rare cases, this can take up to 90 minutes. To see whether data is available for table copy, check the tables.get response for a section named streamingBuffer. If the streamingBuffer section is absent, your data is available for copy. You can also use the streamingBuffer.oldestEntryTime field to identify the age of records in the streaming buffer.
What is the workaround for this buffer time issue?
LinkedIn
Twitter
