TypeError with Datastore Key PathElement in Flask App

Hi everyone,

I'm developing a Flask application and encountering a TypeError when trying to save entities to Google Cloud Datastore. I am on a mac, Montery. I have google-cloud-datastore-2.19.0 and protobuf-4.23.2. I am using python-3.9.12. I got this error,

TypeError: Parameter to MergeFrom() must be instance of same class: expected google.datastore.v1.Key.PathElement got PathElement.

I searched StackOverflow and found 1 person who solved it by downgrading protobuf. It did not work for me. I checked my code and I am calling the functions correctly. 

I would appreciate any advice on how to deal with this error.

 

 

save_story_to_datastore(story_content)
def save_story_to_datastore(story_content):
    """
    Saves a new story to Google Cloud Datastore.
    """
    client = get_datastore_client()
    key = client.key('Story')
    entity = datastore.Entity(key=key)
    entity.update({
        'story_text': story_content,
        'created_at': datetime.datetime.utcnow()
    })
    client.put(entity)
def get_datastore_client():
    """
    Creates and returns a Google Cloud Datastore client configured with a service account.
    """
    try:
        # Fetch the service account key JSON from Secret Manager
        sa_key_json = get_secret('content-genie-network-sa-key')
        
        # Load the JSON string into a dictionary
        sa_key = json.loads(sa_key_json)
        
        # Construct the credentials object from the service account key
        credentials = service_account.Credentials.from_service_account_info(sa_key)
        
        # Create a Datastore client using the credentials
        client = datastore.Client(credentials=credentials)
        
        return client
    except json.JSONDecodeError:
        raise ValueError("Failed to decode the JSON key for the service account.")
    except DefaultCredentialsError as e:
        raise RuntimeError(f"Failed to create credentials with the provided key: {e}")
    except Exception as e:
        raise RuntimeError(f"An error occurred while setting up the Datastore client: {e}")

 

 

 

3 3 72
3 REPLIES 3

The TypeError: Parameter to MergeFrom() must be instance of same class: expected google.datastore.v1.Key.PathElement got PathElement indicates a type mismatch. The MergeFrom() method (used internally by the library) expects a google.datastore.v1.Key.PathElement object, but it's receiving a plain PathElement. This often stems from:

  • Version Incompatibility: Mismatched versions of google-cloud-datastore and protobuf libraries.
  • Incorrect Imports/Usage: Directly using PathElement instead of google.datastore.v1.Key.PathElement or manipulating low-level Protobuf objects.

Solutions:

  1. Resolve Version Conflicts:

    • Check Compatibility: Consult the official documentation or release notes for google-cloud-datastore to identify the compatible protobuf version range.

    • Use Virtual Environments: Isolate project dependencies to avoid conflicts:

       
      # Using virtualenv:
      python -m venv env
      source env/bin/activate
      pip install google-cloud-datastore protobuf
      
      # Using conda:
      conda create -n myenv python=3.9  # Replace 3.9 with your desired version
      conda activate myenv
      conda install -c conda-forge google-cloud-datastore protobuf
      
  2. Ensure Correct Imports and Usage:

    • Import PathElement Correctly:

       
      from google.datastore.v1.key_pb2 import PathElement
      
    • Follow Datastore API Best Practices: Stick to the high-level API provided by google-cloud-datastore for database operations. Refer to the official documentation https://cloud.google.com/datastore for recommended usage.

Additional Tips:

  • Logging: Add detailed logging using logging.basicConfig(level=logging.DEBUG) to capture context in case of errors.
  • Separate Testing: Isolate the problematic code using unit tests with mocked Datastore responses.

Hi,

1. You might not have to downgrade your protobuf. Instead, an upgrade might work.

2. I tried to reproduce your issue (on my local machine,  a Mac running Big Sur) but I didn't get the error. Everything worked.

3. My requirements.txt file is

Flask
google-cloud-datastore

4. As you can see, I didn't specify any versions for the listed requirements.

5. I then ran your code and my output was

story_created_in_db.png

6. I checked the installed components and it installed

google-cloud-datastore-2.19.0

protobuf-4.25.3

7. I should mention I have Python 3.9.9 on my machine and I specified Python 3.9 in my app.yaml.

 

   ......NoCommandLine ......
https://nocommandline.com
        Analytics & GUI for 
App Engine & Datastore Emulator

Please forgive me for taking so long to get back to you. Although it took a long time to reply I truly appreciate the responses. I could not get it to work so I switched to Firestore.