Inquiry about Object Detection for Car Damage Analysis

[PratikDubeyIsHe]

Hello,

We are building a web application for car sales and service, and we are looking to integrate a solution that can automatically analyze images of cars to identify and assess damage. Our process involves the following steps:

Upload Image: A picture of a damaged car is uploaded.
Process Image: The image is processed to detect damage.
Mark Scratches, Dents, and Dislocations: The damage areas (scratches, dents, dislocations, etc.) are marked on the image.
Estimate Cost: Based on the detected damage, the system estimates the cost of repairs.
Could you please let us know if your API can support these features? Specifically, we are looking for capabilities related to:

Object detection to identify and mark areas of damage such as scratches, dents, and dislocations.
A way to calculate or estimate the cost of repairs based on the detected damage.
If your solution supports this type of functionality, we would appreciate more details on how we can integrate it into our system.

Thank you in advance for your assistance!

[adrianthompson]

Hi, this is mine and Gemini's take on your solution requirement. Vertex AI can be a valuable tool in building a web application for car sales and service, but it may not be the out of the box solution for all of the features you've outlined but with a little work the specific requirements would be achievable with Vertex AI.

Object Detection:

• Vertex AI offers powerful object detection capabilities through the AutoML Vision Edge and Custom Model Training features. These tools can be used to train models that accurately identify and locate areas of damage on car images, such as scratches, dents, and dislocations.
• You can provide a dataset of annotated car images to train a custom model tailored to your specific needs. AutoML Vision Edge can also be used to create models with smaller size and faster inference times, which is ideal for web applications.

Cost Estimation:

• Vertex AI, on its own, may not be the best solution for directly estimating repair costs. This task often requires domain expertise and knowledge of specific repair procedures and parts pricing.
• However, Vertex AI can provide valuable information to aid in cost estimation. By identifying the severity and location of damage, you can use the model's output to feed into a rule-based system or machine learning model trained on historical repair data to estimate costs.

•  

Additional Considerations:

• Data Quality: A high-quality dataset is crucial for training an accurate object detection model. Ensure that your dataset contains a diverse range of car images with various damage types and severities. In the same way the quality of the images your service users provide is also a major factor in providing an accurate costing.
• It might be worth looking at latest Gemini multi modal models to help with the costing service with some finetuning

hope that helps

[dawnberdan]

Hi @PratikDubeyIsHe,

Welcome to Google Cloud Community!

You're on the right track with your idea for a car sales and service web application! Google Cloud offers a powerful set of AI and machine learning tools that can help you achieve your goals.

Here’s how Google Cloud can assist with your image analysis needs:

1. Object Detection for Damage Identification:

• Vertex AI Vision: This service provides pre-trained models and the ability to train custom models for object detection. You can use it to identify and mark areas of damage like scratches, dents, and dislocations.
• AutoML Vision: This service simplifies the process of training custom models for object detection. You can provide a dataset of images with labeled damage areas, and AutoML Vision will automatically generate a model tailored to your specific needs.

2. Cost Estimation Based on Damage:

• Vertex AI Tabular: This service can be used to build predictive models that estimate repair costs based on the detected damage. You'll need to create a dataset that includes information about the type and severity of damage, along with corresponding repair costs.
• BigQuery ML: This service allows you to build and deploy machine learning models directly within BigQuery, Google's data warehouse. You can use it to analyze your repair cost data and create a model that predicts costs based on damage characteristics.

3. Integration with Your System:

• APIs: Google Cloud provides APIs for both Vertex AI Vision and AutoML Vision, allowing you to easily integrate these services into your web application. You can upload images, receive damage detection results, and use the data to estimate repair costs.
• Cloud Functions: You can use Cloud Functions to trigger image processing and cost estimation tasks when an image is uploaded. This allows you to automate the entire process and provide real-time results to your users.

In addition, I came across an article/blog that covers Detecting Car Damage Using Deep Learning which could be helpful for you. 

I hope the above information is helpful.

[deepakkataria08]

Hi @dawnberdan,

As per my understanding, There are two plateforms available to perform this operation:

1. Vision API: This is used for detecting objects within images. The Vision API offers multiple features, including Label Detection, Text Detection, Face Detection, Object Localization, etc. However, there are limitations, as the Vision API provides general-purpose image analysis rather than specialized damage detection.

2. Vertex AI: This platform is designed to simplify the process of building, deploying, and managing machine learning models.

With this in mind, I have created a project and completed the following steps:

1. Dataset Preparation: Labeled images with bounding boxes.
2. Model Training: Trained the model using 136 images.
3. Deployment: Deployed the model and obtained the endpoint URL.

Am I heading in the right direction? Please advise.

Additionally, I have following questions:

1. What are the costs of using both platforms? I reviewed the Google Pricing Calculator, but I am unsure which solution or product to select. Do we require any additional cloud solutions to implement them?

2. In Vertex AI, are there any pre-built models available for detecting car dents, scratches, and dislocations?

Thank you in advance.