Supercharge AppSheet using Vertex AI & AppsScript - Demo App & Code Samples

[Stefan_QREW]

AppSheet + GenAI 🤖

AppSheet represents a fantastic entry point for Generative AI to start making a practical impact on your business processes.

Google Cloud makes it easy to access a wide variety of ML/AI models through the Vertex AI platform. In this post I am aggregating examples of how to use these models practically in your AppSheet apps. This post will be updated regularly with new additions.

All source code is on this GitHub Repo 💻

AI Scavenger Hunt App

Link to Demo App & AppsScript Files

Uses These AI Endpoints

• Text Prompting with text-bison
• Analyze an Image with Visual Question Asking model

Resume Review - Compare PDF files with Gemini Multi-modal

Link to Demo App & AppsScript Files

Uses These AI Endpoints

• Multi-Modal Prompting with gemini-1.5-flash

Upload an applicant resume PDF, and select which Job opening you want to analyze it against. Gemini's multi modal model will:

1. Extract relevant applicant information fields: Name, Phone, Email, etc.
2. Compare the resume to the related job description and assign a score & summary to it based on relevance for the position.

That data then gets returned to AppSheet for your easy review! Happy highering.

[Saddam]

Dear @Stefan_QREW ,

Thank you so much. It's amazing.

Cloud you please make something related to predictive model. 

For example: I have my database in google drive and cloud sql. So if I ask gemini how many items in my [this database]?  then it will give me number.

[Stefan_QREW]

Great idea! I will see if i can get that in the line-up

[Joseph_Seddik]

Thank you!

[Stefan_QREW]

Just updated top post with an example & code for using Multi-Modal prompting to compare PDF documents 

Resume Review Demo ^^

[Md_Saddam_Hossa]

@Stefan_QREW , Thank you so much once again.

Appreciating your dedication. Could you please create a system where I can ask any questions related to my Cloud SQL Database, and the system will read data from my database to provide answers? It would be greatly appreciated. I need it urgently.

[Md_Saddam_Hossa]

I need it only for text data in Cloud SQL.

[Md_Saddam_Hossa]

@Stefan_QREW 

 It would be greatly appreciated brother

[Md_Saddam_Hossa]

Hi @Stefan_QREW 

I'm bit confused that which code file should I take to Appscripts.

Because I don't see the project name which is connected with app currently.

[Stefan_QREW]

First, create a new AppsScript project.

Add in all the files in the Repo one by one. (the readme is not required, and the appsscript.json should already exist, but needs to be un-hidden in project settings)

Replace all the account variables and Save the file. 

Run the test script for the endpoints you want to run.

Then, in AppSheet, retarget the Task to your new script file and Authorize

[Saddam]

Yes, It's working for me. Thank you so much for you response.

Could you please make something where Ai can read data from cloud SQL database and give answer from there.

[Saddam]

@Stefan_QREW 

Hope this email finds you well.

Could you please create an app script that reads data from an image? I have developed a CRM application where users can upload business cards. The system should then read the information from the business card and enter it into the appropriate fields such as Name, Email, Phone, Company, and Address.

Thank you.

[Stefan_QREW]

Hi @Saddam 

You can achieve this functionality with very small modifications to the Resume Analyzer scripts. 

You would not need to compare 2 documents, so you could remove one of the objects from the content body of the vertex API call. The file you would pass in would be an image, so the mime type would need to be changed to the type of image that you're passing through. 

Then update the prompt to ask those particular questions of the image. 

[Md_Saddam_Hossa]

@Stefan_QREW Thanks for your response.

I have tried it already but it is not working. could you please send me the code again for this?