How to Implement Scheduled Delta Updates from BigQuery to Neo4j?

[Nikita_G]

Hi all,I'm exploring a strategy to optimize Neo4j graph updates by avoiding full reloads and instead focusing on delta migrations.

Goal:

To avoid full reloads and instead perform scheduled delta updates (only modified or new records) from BigQuery to Neo4j.

Proposed Flow or any another which will provide exact result:

BigQuery (delta records based on timestamp)
↓
Scheduled Script (Python / Cloud Function via Cloud Scheduler)
↓
Neo4j (MERGE/UPDATE only delta nodes and relationships)

Specific Questions:

1. Delta detection from BigQuery:

   • Is using a last_updated_timestamp column the best practice?

2. Scheduling & Automation:

   • What’s the most reliable pattern in GCP for automating scheduled delta syncs?
     (e.g., Cloud Scheduler → Pub/Sub → Cloud Function / Cloud Run)

   • Dataflow Flex Template for BigQuery to Neo4j
     https://neo4j.com/docs/dataflow-bigquery/current/
     
     can we update delta records using above template?

3. Recommended Tools / Templates:

   • Are there any open-source tools, libraries, or architectural blueprints for incremental sync from BigQuery to Neo4j?