Issue using order by with discoveryengine.googleapis.com

[joaocostapt]

Hello. 

My team is currently trying to use the following service:

https://discoveryengine.googleapis.com/v1/projects/{projectid}/locations/global/collections/default_collection/dataStores/{datastoreid}/servingConfigs/default_search:search 

the request body is as follows:

{

    "query":"test",

    "pageSize":3,

    "orderBy": "metatags.my_updated_at desc"  //also tried "my_updated_at desc" and "metatags:my_updated_at desc"

}

the response we are getting is the following:

{

    "results": [

        {

            "id": "0",

            "document": {

                "name": "projects/350627003717/locations/global/collections/default_collection/dataStores/datastoreid/branches/0/documents/0",

                "id": "0",

                "derivedStructData": {

                    "snippets": [

                        {

                            "htmlSnippet": "Jul 25, 2019 <b>...</b> dummy data dummy data",

                            "snippet": "Jul 25, 2019 ... dummy data dummy data ..."

                        }

                    ],

                    "link": "dummydata.com",

                    "htmlFormattedUrl": "dummydata1.com",

                    "formattedUrl": "dummydata1.com",

                    "displayLink": "dummydata1.com",

                    "title": "Dummy title",

                    "htmlTitle": "Dummy title",

                    "pagemap": {

                        "metatags": [

                            {

                                "my_updated_at": "2021-09-14 09:40:43",

                                "published_at": "2019-07-25 13:13:10",

                            }

                        ]

                    }

                }

            }

        },

        {

            "id": "1",

            "document": {

                "name": "projects/350627003717/locations/global/collections/default_collection/dataStores/datastoreid/branches/0/documents/1",

                "id": "1",

                "derivedStructData": {

                    "displayLink": "dummy2.com",

                    "htmlTitle": "Dummy Title2",

                    "title": "Dummy title2",

                    "link": "dummy2.com",

                    "snippets": [

                        {

                            "snippet": "Dec 11, 2020 ... Dummy data",

                            "htmlSnippet": "Dec 11, 2020 <b>...</b> dummy data"

                        }

                    ],

                    "formattedUrl": "dummy2.com",

                    "htmlFormattedUrl": "dummy2.com",

                    "pagemap": {

                        "metatags": [

                            {

                                "my_updated_at": "2020-12-11 15:22:53",

                                "published_at": "2020-12-11 15:03:41"

                            }

                        ],

                        "document": [

                            {

                                "tags": "tagdummy"

                            }

                        ]

                    }

                }

            }

        },

        {

            "id": "2",

            "document": {

                "name": "projects/350627003717/locations/global/collections/default_collection/dataStores/datastoreid/branches/0/documents/2",

                "id": "2",

                "derivedStructData": {

                    "snippets": [

                        {

                            "htmlSnippet": "dummy3",

                            "snippet": "dummy3"

                        }

                    ],

                    "htmlTitle": "dummy data",

                    "htmlFormattedUrl": "dummy3.com",

                    "pagemap": {

                        "document": [

                            {

                                "tags": "otherdummytag"

                            }

                        ],

                        "metatags": [

                            {

                                "my_updated_at": "2024-02-16 14:19:37",

                                "published_at": "2024-02-16 10:09:59"

                            }

                        ]

                    },

                    "formattedUrl": "dummy3.com",

                    "link": "dummy3.com",

                    "displayLink": "dummy3.com",

                    "title": "Dummy data"

                }

            }

        }

    ],

    "totalSize": 17000,

    "attributionToken": "testest",

    "nextPageToken": "testtest",

    "summary": {}

}

The order by in the request seems to not work, despite the my_updated_at existing in the output. Can I get some help with this?

Thanks in advance

[ronnelg]

Hi @joaocostapt, 

Welcome to Google Cloud Community!

I understand that you are having an issue with ordering by date in descending order. You may try to change your 'my_updated_at' field to a predefined, Google-inferred page dates or order by custom structured data attributes.

You can also check this post on how Google picks the right date for your web page.

I hope the above information is helpful.

[GuodongJiang-co]

@joaocostapt how did you get https://discoveryengine.googleapis.com/v1/projects/{projectid}/locations/global/collections/default_... ? I'm working on similar task, but I can only get it work by using cloud shell. Got 503 error when make api call via python or node.js client library. Appreciated if you can share the config you have done. thanks in advance!