PGVector v0.5.0?

[draozoth]

Pgvector v0.5.0 is being released in the next 2 weeks with HNSW index support, which is a huge improvement over previous indexes both in terms of recall and maintenance requirements.  With the rise of AI / embeddings based solutions, is there a plan to get this into Cloud SQL when it is released?  I believe that the extension right now is v0.4.2 when you enable it.  

https://jkatz05.com/post/postgres/pgvector-hnsw-performance/

https://www.pinecone.io/learn/series/faiss/hnsw/

[winpat]

pgvector v0.5.0 has been released in the meantime. We would be very interested in getting support for the new pgvector version in CloudSQL. As the current IVFFlat index does not fulfill our performance requirements, potentially leading us away from CloudSQL for this specific use case.

[megil_paxton]

Would also love to hear details on this!!!

[ms4446 · Accepted solution]

Hi everyone,

Just FYI… 

Pgvector v0.5.0 support in Cloud SQL Postgres was just released today.

Thanks for your patience.

[pkavqnt]

What about for AlloyDB? Is there a place to track what version of pgvector is available for CloudSQL or Alloy?

[ms4446]

Pgvector v0.5.0 support in AlloyDB will be released very soon.

[ms4446]

Just an FYI.. Pgvector v0.5.0 support in AlloyDB was released this week.

https://cloud.google.com/alloydb/docs/release-notes

[chbussler]

@ms4446 thanks for the note on pgvector v0.5.0 availability for AlloyDB. I have AlloyDB Omni installed on my system: could you please provide the command to run to upgrade the extension to v0.5.0? Thank you!

[ms4446]

Hi @chbussler ,

Updating the pgvector extension in AlloyDB Omni involves a few specific steps, similar to installing any postgres extension:

Download pgvector Extension Files

1. Visit the pgvector GitHub releases page (https://github.com/pgvector) and download the latest release.
2. Extract the downloaded archive. Ensure it contains the necessary files for installation, typically a .sql file and possibly a shared library (*.so file), depending on pgvector's distribution format.

Create a Custom Docker Image

1. Start with a base Docker image. Verify if alloydb/omni is available and suitable for customization.
2. Copy the pgvector installation files into the Docker image. For example, place them in /opt/pgvector. Note: The actual installation of the pgvector extension (running SQL commands) should occur within the running database environment, not during the Docker image build process.

Build and Run the Custom Docker Image

1. Use the docker build command to build your custom Docker image, specifying the path to your Dockerfile.
2. Run the image using the docker run command, including necessary environment variables and mounting volumes as needed.

Load the pgvector Extension in AlloyDB Omni

1. Connect to your AlloyDB Omni instance using psql or a similar tool.
2. Execute the SQL command to load the pgvector extension. Ensure the extension name is correct (e.g., pgvector) :

CREATE EXTENSION IF NOT EXISTS pgvector;

Verify the Update

1. Confirm the extension's presence with:

SELECT * FROM pg_extension WHERE extname = 'pgvector';

2. Check the installed version of pgvector:

SELECT extversion FROM pg_extension WHERE extname = 'pgvector';

Additional Notes

1. Replace <version> with the actual version number of pgvector you downloaded.
2. Adjust configurations as per your specific setup and requirements.
3. Refer to the pgvector documentation for detailed usage and feature information.
4. Consult the AlloyDB Omni documentation for specific instructions related to AlloyDB Omni.
5. Conduct thorough testing after installation to confirm functionality.