I need a solution for linking images

We have thousands of images stored in our server, I can create a VB routine to export certain image folders to an online cloud storage at intervals throughout the day so the numbers would reduce.

I need to think outside the box and find a way for Appsheet to access the folders, has anyone been down this path before and lend some advice or recommendation on how to start, tips - tricks - anything??

Kind Regards

0 66 4,626
66 REPLIES 66

Hi to all,

Clearly an interesting topic over here !!
I would say, as usual when mister @tsuji_koichi is involved.

First, Personal Opinion

On a overall topic, โ€œpublicโ€ endpoint should never be used in a production app. [ google services or any other one ]

Like the uc?export=view&id=xxxxxx but i could name many others.

Those are not reliable, not efficient and are prompt to behaviour change, not to mention some can be deleted. ( google has a long story of switching, turning off those kind of end-point )

Using an API request is the way to do it, at least for production environment.

Return to the main topic,

As @tsuji_koichi mentioned the only way to made a efficient and reliable access to any data is to call an native AppSheet โ€œfunctionโ€ to make the request. ( remember all those request probably goes into some async loop worker )

So setup 2 with file path will always be the best.

some details :

I never know why but Appsheet seems to use two โ€œdifferentโ€ methods to feed image inside app. (maybe it depends of security option or maybe even view type )

with a https://www.appsheet.com/get/?i=
this will return a 600 width image.

[the full resolution image can be open with the fullscreen icon, and it seems no new request is send to access it ]

with the https://www.appsheet.com/template/gettablefileurl?appName=
this will return a 600 width image from the https://www.appsheet.com/fsimage.jpg?filename=

[the full resolution image can be open with the fullscreen icon, but a new requested has to be sended to access it, this will generate a wait time and a lot of 404 error, but i think it really depends how you secure your urlโ€™s app, security options : secure url, secure image etcโ€ฆ]

Both method require a unix relative file path, [many way exist to massively generate those path]

the https://www.appsheet.com/fsimage.jpg?filename= can be directly generated and put into a table column with a size parameter, or setting it inside the app with a virutal column, context view, action, user settings etcโ€ฆ

( this is the fastest way to have the image inside the app, plus you can minimize mobile data usage and app reactiveness )

Any way,
Honeslty, i have an App with more than 200K image, and AppSheet handle it natively like a charm.
( only a classic file path column set as image type, without any โ€œoptimizationโ€ ) [ Iโ€™m still amazed by this ]

End,
I could mention the use of the https://www.appsheet.com/remote/ + Url end-point [view, export, download, http web-app services request ]

( can be useful to if you want to add a document conversion action button but better to use a webhook api for this)

Linked Topic

Itโ€™s funny how time coincidence collide sometimes;

I recently finished the development of an API to give my users access to their personnel directories through a GAS Back-end [a mix of pure drive.app and drive API V2] and a Front-End with JavaScript/JQuery/React/Html.

Basically,
First, i created a JSON Query function to dynamically map and index any requested directorie tree + files.

Then i created a Custom React/JavaS component which take the returned JSON Object to build dynamically the Front-End HTML Code.

This is morish like an AppSheet Deck view type, with some embded icon for go-in-to-folder, delete, rename, view, add or download file or folder.
( the onclick() Html event trigger back the specific GAS Back-end fonction )

So, im more than happy to learn that AppSheet develop a drive access component.
This can maybe solve one of my long time usercase : a fully AppSheet integrated Document Management System

Wait and see

@Scott_Haaland please sign me up for the test.

Iโ€™m just testing it. I never thought the virtual column method works, but it does.
Obviously when Appsheet introduce the Google Drive Table it will be extremely valuable to many users.

Thank you for your assistance.

Yes, once they introduce goole drive file table features, hopefully we say good-bye to script.
Good to hear you seem to solve the problem, and app should now run faster to render image.

Yes there is an improvement, thank you again.

Is this ready to roll out yet?
Regards

Hi @Dave_Willett and others on this thread,

The first version of Drive Data Source will be rolling out soon. Instead of signing up preview users by ID, we are rolling this out as any other feature, so you may see it show up in your account soon.

This is a preview version - it still needs some work.

  1. It is read only for now
  2. For Images, you will still need to add a virtual column with a formula to display the image from drive. Something like this: CONCATENATE(โ€œhttps://drive.google.com/uc?export=view&id=โ€, [_ID])
  • The great news is: The _ID is just a column that will always have the Drive file id value easily accessable to you.
  1. It is single level only - meaning, we donโ€™t support drilling down through subfolders, etcโ€ฆ
  2. You can only Add a Table to your App backed by a Drive Folder, but you canโ€™t โ€œCreate an Appโ€ from a Drive folder only. You will have to start with a Sheet or other data source, and then add the drive folder as a table.
  3. We are only capturing a few basic fields from Drive so far: ID, Path, Name, Created DateTime, Last Modified By. We can add other properties in a subsequent release (as requested above ^^ in this thread).

If you want to collect and maintain other โ€œmetadataโ€ about the file, you should use a sheet or DB and use _ID or Name as a reference to the other table where you can maintain the other details you want.

Please donโ€™t use for production until it is GA. We will continue working on this and hopefully close out these limitations in phases as we go. Once the rollout starts, it could take a week or two (or longer if we have issues) before you see it, so please be patient. Once you have a chance to play around with it, Iโ€™ll be keen to get your feedback.

Thanks,
Scott

This is great news Scott.
How will we know when this is available, will we receive an email?

Kind Regards

Hi @Dave_Willett

I donโ€™t believe you will see any notification. When you see โ€œAutomation - betaโ€ in your left pane, then you will know that your account has been rolled out.

Please refer to this announcement. The Drive data source capability is bundled with the Automation release (What features are in the Preview release?).

R,
Scott

Hi All,

Hopefully you have all seen the new Automation features. I just ran a quick test based on @tsuji_koichiโ€™s suggestions about addressing the image file, and I also saw a very good improvement by just using the file name. Since the Drive data source already knows the path of the folder that it is configured on, I just created a Virtual Column I called ImageLink of type Image, and I tried the following formulas:

Option 1: TEXT([File]) - This will just put the string from the File name into this column. This one rendered the images very fast
Option 2: TEXT([File]) - CONCATENATE(โ€œhttps://drive.google.com/uc?export=view&id=โ€, [_ID]) - This one was quite a bit slower

Just confirming Tsujiโ€™s findingsโ€ฆ You could do some things like check for the file type to see if it has an image extension on the file name, and only populate the ImageLink field if it has an Image, etcโ€ฆ But I put a Document in my folder, and it just shows an icon that indicates no image was found for that file in my gallery view.

Thanks,
Scott

Hi @Scott_Haaland

Thanks for taking your time for testing and sharing your result. Yes, just picking up the file name only to construct the image type field should provide the better solution to render image much faster than using drive.google.com Technical background, i m not pretty much sure, but this is what it is.

This could be backwards operation. โ€ฆ Usually user upload the file from the app then capture the file name on that column. But this time, read as table for google drive folder will generate file name, then we employ the power of VC to convert to image type to show image on the app.

What a nice.

Hi @tsuji_koichi ,

Yes, this is a different than the normal use caseโ€ฆwe are working on adding a couple of features that will bring it back around to the use case you are used to:

  1. Event capability - When a file is added/modified on Drive - start an Automation flow, or set workflow rules on your file table to take actions on add/modified table events - like: Add a row to another table when a row is added to this table, etcโ€ฆ
  2. Ability to add files from the phone camera or desktop upload - By simply adding a row to the File table, it will save the new image as a file on Drive (using a temporary uniqueid()), get the Drive ID and then refresh the ID with the Drive ID for future reference.
  3. Other write operations like update - refresh the image with a new image, or delete a file by removing the row from the table, etcโ€ฆ

Once these Write features are available, you could use this Drive data source for bi-directional use cases: Add files/images, view files/images, take actions based on files/images.

Special thanks to Tsuji for his feedback to me offline after running a battery of cool tests right after he got accessโ€ฆitโ€™s much appreciated and will help to shape this feature to be even better for the whole community!!

Cheers,
Scott

Thank you for this Scott, it seems a little daunting on where to start. Do we have a video instruction to get us on our way?

Kind Regards

*** Ignore me, I found in in creating a new table area ***

Iโ€™m not sure how accurate this is. Created the table from the google drive, added Virtual column with TEXT([File]) - Set to image.
There are images in my Google Drive Folder which are not pulled through to the table.

The folder โ€˜ImagePathโ€™ only contains images and each image has the same naming convention:

Mystery as to why it isnโ€™t collecting ALL the images when I can see them in the driveโ€ฆ

We have a temporary limit (during Preview) of 1000 files in a folder that may be causing your issue? We are evaluating how to set these limits for GA.

How many files do you have in your folder today @Dave_Willett ? This will be a good data point for us.

Thanks,
Scott

Hi @Scott_Haaland there are 394 images all the same naming convention. 6 digits, Hyphen and increment of 3 digits then .jpg

Hi @Dave_Willett

Then you shouldnโ€™t be hitting the 1000 limit. Do you notice any pattern to the files that are showing vs. not showing? I will try to reproduce this today, but if you have any observations that would help out engineering, that would be helpful. Also, can you cut/paste the whole formula for your ImageOrder Virtual Column? I assume this is some kind of sequence for sorting?

@mikeas - FYI - Dave is not seeing all of his images while using the Drive Data Source.

R,
Scott

Sure @Scott_Haaland , I canโ€™t see a pattern really, I tested yesterday and found Images in both the local C:\users\dave.willett\googledrive folder and the online Google drive, they were synced correctly. It seems the Appsheet drive table was incomplete but with no pattern. As mentioned earlier, all images ( and there are only images there) are 6 digits, hyphen, incremented digits and .jpg, example:

***** The table contains 100 rows @ 1 image per row, could it be your test limit is 100 instead of 1000 (thousand) ??? ****

12345-001.jpg
12345-002.jpg
12345-003.jpg
54355-001.jpg
54355-002.jpg

Hi @Scott_Haaland , @mikeas not sure if you guys saw my latest findings? Google table maxed out at 100 rows not 1000.

Regards

Hi @Dave_Willett, I took a look into this and discovered an issue with our handling of pagination from the Drive API that resulted in a maximum of 100 files being returned. Iโ€™m addressing this now, with the fix hopefully being released tomorrow.

Thanks @mikeas , my assumption seems to be correct then. It has set me back a little but hopefully I will be back on track tomorrow.
Kind Regards

Congrats @Dave_Willett on helping us find a bug And thanks @mikeas for squashing it!

R,
Scott

There are also instructions in the Automation Preview Primer doc, which can be found here, in case anyone else wants to learn about how to configure this.

However, it is pretty straightforward once you understand this, as Dave discovered:

  • Add a new Table to your app
  • Choose โ€œDocuments on Googleโ€
  • Choose โ€œCollection of Filesโ€ option on the right hand side
  • Select your Drive Folder that you want to use as your data source โ†’ Table
  • Add your virtual column with the Image link as TEXT([File])
  • Add a View to render your images - the Gallery view works nicely

Hi Scott, I am trying to follow this simple steps but I get the message that there are โ€œNo items in this folderโ€ even though I have .png and .jpeg files saved here. Just starting with Appsheet, so this is frustrating. Also read through tons of conversations on how to use Google Drive images in Appsheet and I canโ€™t even identify an image id inside the generated url from the โ€œget linkโ€ url in google drive. What am I missing?

Thank you or anyone else for helping a newbie.


3X_9_5_9599881ceddacc5aebac0a9322599ae272227380.png

When you add folder as table, you are prompted folder/file picker. Select the folder, and then hit select button at the botton. Dont double click over the folder.
The selected folder is highlighted in blue, then you hit SELECT button at the bottom.
Then move on the rest of process to add folder as table.

Thank you very much. I will try this

Thank you @tsuji_koichi , it worked! Now I can progress.

Top Labels in this Space