Cache Invalidation

[beratsert41]

Hi,

In admin sections, I am reviewing Datagroups to monitor caching resets. However, although there is a cache invalidation in that datagroup, it is not reflecting there. Is there any where else that I can check? (I am 100% sure that the query and the model is the same while I am testing caches.)

Thank you

[Kevin_F_McCarth · Accepted solution]

Yes.  FYI - here is a publicly available page that lists Looker startup options, including a couple settings relating to cache settings. 
https://cloud.google.com/looker/docs/startup-options

Although the page explicitly mentions Customer-Hosted instances, these settings are in play on Looker-hosted instances as well - but for Looker-hosted, only Looker ops team can adjust these settings (if necessary... typically in reaction to specific issues or because of an explicit request from the primary admin contact on the instance).

To be clear: I'm not fully convinced changing these settings will resolve your issue - I think more investigation / reproduction of the scenario needs to be done to be sure.

[Kevin_F_McCarth]

Hello Berat,

Can you please clarify what you are trying to do, or what information you are looking for, or are you saying that cache is not being invalidated as expected when interacting with that admin page?

One thing you can additionally look at is the relevant API calls relating to datagroups.  I recommend installing the API Explorer and then looking at get_datagroups.  There, you will see similar information to what is on the Datagroups Admin page, but I am thinking that format and the ability to use it independent of that particular admin page UI may be useful to you.

[beratsert41]

Hello  Kevin,

Thank you for kind response. Let me clarify:

Problem: The cached results for all dashboards & looks  suddenly disappears. 

Firstly, I am checking the queries. They are identical.

Secondly, I check the model, there is no change on the model.

Thirdly, I am routing to Datagroups  Admin Page to check if there is a cache invalidation for datagroups and there is no cache invalidation close to the incident.

I am wondering what is happening to the cached results. Can you help me on this?

[Kevin_F_McCarth]

OK I now understand that you expect cache to be used, but instead the full queries are running against the database.  And according to the admin page, the relevant datagroups have not been triggered.

It is difficult to troubleshoot without seeing this issue hands-on.  If what you describe is true and cache is not being used when it should (same query, same user, explore's persist_with datagroup not triggered since the relevant results were previously run...) then this might appear to be a bug or other issue with the instance (I understand there is a limit to in-memory-cache-size).  I would recommend you initiate a support ticket from your instance and provide as much detail as possible so that the support team can gather required info to assess.

[beratsert]

Is there a limit on Looker-hosted instances for in-memory-cache-size?

[Kevin_F_McCarth · Accepted solution]

Yes.  FYI - here is a publicly available page that lists Looker startup options, including a couple settings relating to cache settings. 
https://cloud.google.com/looker/docs/startup-options

Although the page explicitly mentions Customer-Hosted instances, these settings are in play on Looker-hosted instances as well - but for Looker-hosted, only Looker ops team can adjust these settings (if necessary... typically in reaction to specific issues or because of an explicit request from the primary admin contact on the instance).

To be clear: I'm not fully convinced changing these settings will resolve your issue - I think more investigation / reproduction of the scenario needs to be done to be sure.

[beratsert41]

One last question: What would be expected after hitting the limit?

[Kevin_F_McCarth · Accepted solution]

Though I don't have direct experience with this scenario, what I'm seeing from prior discussions indicates:
If the max_cache_storage size is reached then the looker invalidates the old cached results in the cache in order to store the new cache results.

[beratsert41]

Thank you!