pandas df from Functions to BigQuery

[J03rgPf]

I would like to write a pandas df into Bigquery using load_table_from_dataframe. But it throws me this error:

Got unexpected source_format: 'NEWLINE_DELIMITED_JSON'. Currently, only PARQUET and CSV are supported

this is my code:

from google.cloud import bigquery
import pandas as pd
import requests
import datetime

def hello_pubsub(event, context):
     
     response = requests.get("https://api.openweathermap.org/data/2.5/weather?q=berlin&appid=12345&units=metric&lang=de")
     responseJson = response.json()
     
     # Creates DataFrame
     df = pd.DataFrame({'datetime':pd.to_datetime(format(datetime.datetime.now())),
                   'name':str(responseJson['name']),
                   'temp':float(responseJson['main']['temp']),
                   'windspeed':float(responseJson['wind']['speed']),
                   'winddeg':int(responseJson['wind']['deg'])
                   }, index=[0])  

     project_id = 'myproj'
     client = bigquery.Client(project=project_id)
     dataset_id = 'weather'

     dataset_ref = client.dataset(dataset_id)
     job_config = bigquery.LoadJobConfig()
     job_config.autodetect = True
     job_config.write_disposition = "WRITE_APPEND"
     job_config.source_format = bigquery.SourceFormat.NEWLINE_DELIMITED_JSON
     load_job = client.load_table_from_dataframe(df, dataset_ref.table("weather_de"), job_config=job_config)

[samar]

Hi,

The issue you are reporting seem very similar to this stackoverflow post[1],  Can you try the suggested solution offered by  Ernesto?

Best Regards

/Samar

[1]: https://stackoverflow.com/questions/70172711/pandas-df-from-cloud-functions-to-bigquery-only-parquet...