Application Integration Connectors Carbon Impact

[phertzog]

Hello,

In my organisation, we build a regular report on carbon impact of our projects sorted by services. In this report I was surprised that the carbon cost of the application integration connectors (in our case we have for example 3 FTP connectors) is huge compared to other services like Cloud Run or event Cloud Engine.

How is it that the displayed carbon cost of the connectors is so high?

Philippe

[tyayers]

This is a great question, I will post it in our internal product chat and get back to you...

[greb]

Hi @phertzog,

Welcome to Google Cloud Community!

The high carbon cost of your FTP connectors likely stems from being perpetually active, reserved processor time, and the less sophisticated protocol when compared with the dynamic scaling offered in Cloud Run or Compute Engine.

What is the reason behind this?
1. Connections that are always on use more energy.
2. Even when idle, resources are set aside.
3. FTP is older and less sophisticated than modern protocols.

What can you do:
1. Adjust to enhance efficiency - Utilize event driven or scheduled transfers.
2. Minimize resource allocation - Stop over-provisioning.
3. Switch to SFTP - It is more secure and efficient.
4. Pick less polluted areas - Deploy to Google Cloud regions with a greater proportion of carbon free energy.

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.

[phertzog]

Hello,

We already use low carbon regions (europe-west1) and we have a mix of FTP and FTPS connectors,

Concerning the point of resources running even without usage, it is also the case for compute engines or Cloud SQL. However these components seem to show a lower carbon impact than "simple" connectors. This is what puzzles me.

[shaaland]

@phertzog - one thing you can do is to use a scheduled integration to Suspend your connection when it is not in use, and then another one to Resume the connection prior to it's usage.  The Connector APIs can be called with the REST Task to suspend and resume a connection.  This post has details of how to do this.  A lot of times FTP based interfaces are schedule driven

Sample use case:
ie: check for a new file from my supplier once per day at 8:00 AM
- Schedule a suspend of the connection at 8:30 AM every day (assuming your files are transferred within 30 minutes)
- Schedule a resume of the connection at 7:50 AM (assuming resume will take 2-3 minutes max, you could push it even closer to 8:00 depending on your tolerance for errors). 

[phertzog]

Hello,

Thanks for your feedback. But I think the thread has driffted from the starting question. We already put in place some mitigations to lower the impact of the connectors. But the initial concern was to have an explanation on why the connectors have such a heavy carbon cost compared to other services (VM instances, CLoud SQL ...) we also use. Is the formula wrong or is there a technical reason to explain this difference.

But this information is perhaps not publicly available.

Regards

Philippe