Difference Google Lens vs Google Cloud Vision API

[bilalhinis]

Dear Google Cloud Support Team,

I am currently using the Google Cloud Vision API—specifically the Web Detection feature—to identify similar images on the web and extract the URLs where those images appear.
My goal is the following:
I upload an image (e.g., of a car, product, etc.).
I want to receive highly similar or identical images.
Most importantly, I need the URLs of the websites where these images appear—ideally from specific domains (such as automotive marketplaces, e-commerce sites, etc.).
However, the results returned by the Vision API are often weak. The images are not visually close enough, and the returned links are either irrelevant or incomplete. When I try the exact same image in Google Lens (manually), I get much more accurate and contextually relevant results, often including the precise websites I’m targeting.
This leads me to a few questions:
Are Google Cloud Vision API and Google Lens using the same backend technology?
If yes, why are the results so different?
If not, is there any way to access similar performance to Google Lens via Google Cloud services?
Is there any recommended way to structure requests to the Vision API to get more precise and targeted results (e.g., better visual similarity, filtered domains, etc.)?

[dawnberdan · Accepted solution]

Hi @bilalhinis,

Welcome to Google Cloud Community!

Google Cloud Vision API and Google Lens are powered by Google's advanced machine learning capabilities for image analysis, they are not identical products using the exact same technology stack or models.

• Cloud Vision API - provides developers with access to pre-trained machine learning models via REST and RPC APIs. It includes features like label detection, OCR, face detection, and web detection. 
• Google Lens - is an app used by regular people and is built into Google products like Google Photos and Google Search. It helps users get quick and useful information about things they see in their photos or through their camera. It also uses extra details like what the user does and special ways of searching that work well for everyday use.

While they share some foundational technologies, Google Lens benefits from deeper integration with Google Search and real-time contextual data, which the Vision API does not expose.

The difference in goals and design explains why the results can vary. Google Lens is made to deliver fast, highly relevant answers for everyday searches, using advanced filtering and ranking methods that aren’t available through the Cloud Vision API’s Web Detection feature.

Accessing Google Lens-Level Performance Through Google Cloud - There is currently no public Google Cloud service that offers the exact same features and performance as Google Lens. Google Lens relies on a complex, proprietary system that goes far beyond what a single API can provide.

However, there are ways to enhance your results with Google Cloud services and post-processing steps to get closer to your desired outcome. Here are some tips to improve precision:

• Use includeGeoResults: true to your request to improve relevance by using location-based data.
• Use high-resolution images with clear, focused subjects.
• Combine features: Use both WEB_DETECTION and LABEL_DETECTION to enrich context.
• Filter results manually: Since the API doesn’t support domain filtering, post-process the returned URLs to match your target domains (e.g., filter for cars.com, autotrader.com, etc.).
• Try Product Search API: If your use case involves retail or e-commerce, Google’s Product Search API (part of Vision AI) will be more suitable.

In addition, Google Cloud Vision API and Google Lens have different goals and work at different levels, you can greatly improve your results by using specific fields in the Web Detection response and adding your own processing in your app. For product-related searches, check out the Vision AI Product Search API, and for more advanced custom search needs, consider using Vertex AI solutions.

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.