Feedback Suggestions to Enhance AppSheet AI Task C...

Saddam

Dear AppSheet Team,

We’re excited to explore the newly released AI Task feature in AppSheet. After experimenting with it, we’d like to share the following suggestions and improvements that would significantly enhance its usability—especially for field-heavy industries like oil and gas and construction:

✅ 1. Dynamic Instructions (User-Written Input Support)

In many use cases, we want to allow end-users to write custom instructions. It would be great if we could pass a LongText field as an additional or primary instruction to guide the AI dynamically during execution.

✅ 2. Enum / EnumList Value Selection

When working with Enum or EnumList fields:

The AI should be able to access the list of possible values (from dropdowns) and
Select the most relevant option based on the instruction and the available choices.

This would help the AI provide more accurate outputs aligned with app logic.

✅ 3. Use of Formula in Instructions

There should be support for using expressions or formulas inside the instruction to make them more dynamic and contextual—such as referencing dates, user roles, or other columns for intelligent behavior.

✅ 4. Output Support for HTML / Markdown

It would be incredibly helpful if the AI could return output in HTML or Markdown format, especially for LongText fields.
This would improve readability and allow formatting like:

Bullet points
Headings
Bold/italic highlights
Perfect for descriptions, summaries, or reports.

✅ 5. File Input Support (Video, Excel, CSV)

In our industries, video analysis and structured file input are crucial:

The ability to upload a video (e.g., of a safety incident or process) and have the AI generate a brief, structured description would be a game-changer.
Reading Excel/CSV files would allow extracting tabular data, observations, or logs directly into the app.

✅ 6. Reading Referenced User Tables (Role-Based Assignments)

Often, we need to assign tasks based on user roles. For example:

“Assign this to someone with the ‘Driver’ position.”

If AI could look up user tables, filter based on roles, and assign accordingly, it would allow seamless integration of decision logic into task assignment.

We believe implementing these features will make AI Tasks far more flexible, powerful, and enterprise-ready. Looking forward to seeing these capabilities evolve!

Best regards,
Saddam Hossain
AppSheet Expert
Dubai, UAE
@saddam24k

@Jana_Mandic
@amyplin
@lizlynch
@nico
@ShirleyN
@goannsplain
@zito
@Roderick
@Lauren_vdv
@Arthur_Rallu

*I encourage everyone to upvote this and help bring these improvements to reality!

zito

These are great, thanks for sharing. For #1, it seems like if we did #3, that would accomplish your goal, right? The submitted longtext field could be put as an expression into the instruction box? Or perhaps I'm misunderstanding how you see it working.

#4 is interesting - I know it can be challenging to get the models to reliably return formatted information. Have you tried asking for markdown or html in the instruction box?

In terms of file upload types - out of the box gemini doesn't support excel as a file type, so we'd have to convert it to an intermediary type and then submit it to gemini. Not impossible, but it adds another hurdle. It might be possible to do this yourself by converting uploaded excel files to images and then submitting them to the AI task, but I realize that's more effort on your part.

With regards to video, the issue is that video consumes a LOT of AI inference capacity due to the number of tokens. From a costs and usage perspective, that makes it infeasible for us today. If I remember correctly we did prototype some video support to test this out, it worked, Gemini did a good job at explaining videos, but even short videos consumed a massive number of tokens. However, inference costs continue to drop rapidly and Google TPUs keep getting more powerful, so the economics might work in the future. Alternately, we've discussed allowing video support by having AppSheet admins connect AppSheet to Vertex on Google Cloud, where customers can be billed directly for the inference costs for uploaded videos. Either way, we see the value very clearly, and we've heard this feedback from other folks.

Thanks for sharing the feature requests - appreciate the thought that went into them.

aneeshatgoogle

Thank you for sharing this. This is very valuable feedback for us to consider as we evolve AI Tasks.

I have a few questions and thoughts to get a better understanding of your use case:

> Dynamic Instructions (User-Written Input Support)

Is this for extract or categorize? For categorize you can try something like below:
1. Add the column containing custom instructions as input to the ai task

2. In additional instructions, instruct gemini to categorize based on criteria mentioned in column "column_name" (please make sure the column name is same as what you have in input).

> Enum / EnumList Value Selection

We do currently support Enums in Extract and Categorize AI Task currently. Are you trying to get the enum values from a ref table?

> Output Support for HTML / Markdown

Current Extract AI Task is oriented towards getting structured information from file (image or pdf) i.e. to fill up other fields in the table. For this use case are you trying to generate a summary? It will be great to know what does the output represent.

Saddam

Hi @zito ,

Thanks again for the detailed follow-up. Let me share a few thoughts based on your response:

1. Dynamic Instructions via Formula (LongText + #3)
Yes! You're right—I think if we can use a formula like =[UserInstructionField] inside the AI Task instruction box, that should solve the problem for most use cases.

2. Markdown / HTML Output in LongText Fields
Yes, I’ve tried prompting with:

“Return the output in markdown format” or “Format the result as an HTML list.”

No, It didn't work.

3. Excel/CSV File Handling
Understood about Gemini’s limitations here. However, Gemini does support CSV file. So if you can make it at least CSV supported that would be great.

4. Video Analysis Support via Vertex AI
That idea actually sounds very promising. Connecting AppSheet with Vertex AI (and taking on the cost for advanced inference) would be a fantastic option, especially for industries like ours where short safety videos are routine. Count me in for testing or piloting if this moves forward!

Thanks again for engaging with the feedback—it really means a lot when product teams listen closely to the use cases we face on the ground.

Saddam

Dear @aneeshatgoogle ,
Thanks a lot for your response! Really appreciate the follow-up questions—happy to explain a bit more:

> Dynamic Instructions (User-Written Input Support)
Is this for extract or categorize?
Ans: Primarily we need it for Extract however it will be good for categorize as well.

2. In additional instructions, instruct gemini to categorize based on criteria mentioned in column "column_name" (please make sure the column name is same as what you have in input).
Ans: it will be really good to have instructions field instead of Input. However i will do like that for now.

> Enum / EnumList Value Selection
We do currently support Enums in Extract and Categorize AI Task currently. Are you trying to get the enum values from a ref table?
Ans: Yes! You're Right. In some cases, I would like to instruct to get categories from ref table based on input criteria.

> Output Support for HTML / Markdown
Current Extract AI Task is oriented towards getting structured information from file (image or pdf) i.e. to fill up other fields in the table. For this use case are you trying to generate a summary? It will be great to know what does the output represent.
Ans: Yes! You're absolutely right. In most of the cases, i need to describe the hazards, potential risks and actions. in this cases we need to generate brief description about image or file that user uploaded.

Thanks again for taking the time to explore this! I really appreciate your time and effort on it.

Regards
Saddam

Saddam

Dear Team,

Over the past few days of working with AppSheet AI Tasks, I've seen significant potential and numerous benefits. I would like to kindly request two additional features that could further enhance its capabilities.

Support for JSON Output:
It would be highly beneficial to have the capability to push JSON-formatted data into the application output. For example, if the AI reads an invoice file, it should extract child records and then use JSON to add a list of items or rows into the child table. This would streamline data integration and improve automation processes.
Increased Character Limit for Instruction Field:
The current character limit in the instruction field is restrictive. Increasing it to around 5,000 to 10,000 characters—similar to AppSheet’s long text field—would provide more flexibility and allow for more detailed instructions, enhancing the AI's performance and accuracy.

Thanks

Saddam