Announcements
This site is in read only until July 22 as we migrate to a new platform; refer to this community post for more details.
Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

Issue with hyphen - in postgres db name

I am trying to transfer data from on premise Postgres dbs into big query. This works as expected for databases with standard naming conventions. However I have a number of dbs with hyphens in the name app-customer for example. I have tried to escape the db name with "" '' /s but not joy. Is there anything that can be done rather than renaming all my dbs?

0 1 493
1 REPLY 1

For databases with hyphens in the name like app-customer, instead of trying to query them directly, use standard PostgreSQL tools such as the COPY command or pg_dump, which correctly handle quoted identifiers. 

Export to a neutral format (CSV or Avro):

SQL (inside psql): 

-- Export table to stdout
COPY your_schema."your-table" TO STDOUT (FORMAT CSV, HEADER);

-- Or use \copy to write directly to a local file
\copy your_schema."your-table" TO 'local_file.csv' WITH (FORMAT CSV, HEADER);

Bash:

# Export schema or data using pg_dump
pg_dump -h your_host -U your_user -d "app-customer" -t 'your_schema."your-table"' > backup.sql

# Or dump to CSV using COPY/\copy

Then:

  1. Upload the exported files to GCS using gsutil or a client library.

  2. Load the data into BigQuery using the web UI, bq, CLI, or a script.

BigQuery handles ingestion from GCS very well. This avoids the need to rename your databases and keeps your current naming intact, even though it adds a step to the process.