Hi, I am sending ~600 byte payloads via Particle webhooks to post in a Google Sheet. Each payload consists of 1-9 events (each event has a timestamp, temp readings and batt voltage). The webhooks are valid JSON. From particle i can see the webhooks are being sent. I send a payload every 5 seconds. I send a total of 11 payloads every 20 minutes. I am using the following app script to consume each payload and post to a sheet:
function doPost(e) {
try {
// Step 1: Parse the incoming JSON payload
var payload = JSON.parse(e.postData.contents);
// Step 2: Get the current date and time when the batch was received
var receivedTimestamp = new Date();
// Step 3: Extract the device ID (uid) from the payload
var deviceID = payload.uid;
// Step 4: Open the Google Sheet by its ID
var sheet = SpreadsheetApp.getActiveSpreadsheet().getActiveSheet();
// Step 5: Loop through each event in the payload and prepare the data
var data = [];
payload.ev.forEach(function(event) {
var utcTimestamp = new Date(event.ts * 1000); // Convert Unix timestamp to JavaScript Date (UTC)
var pstTimestamp = new Date(utcTimestamp.getTime() + (7 * 60 * 60 * 1000)); // Convert to PST by adding 7 hours
var row = [
receivedTimestamp, // Prepend the received timestamp to each row
pstTimestamp, // Use the converted PST timestamp
deviceID, // Device ID
event.t1, // Temperature 1
event.t2, // Temperature 2
event.b // Battery voltage
];
data.push(row);
});
// Step 6: Append all rows to the sheet
if (data.length > 0) {
sheet.getRange(sheet.getLastRow() + 1, 1, data.length, data[0].length).setValues(data);
}
// Step 7: Return a success response
return ContentService.createTextOutput(JSON.stringify({result: 'success'})).setMimeType(ContentService.MimeType.JSON);
} catch (error) {
return ContentService.createTextOutput(JSON.stringify({result: 'error', message: error.message})).setMimeType(ContentService.MimeType.JSON);
}
}
```
What I am seeing: in executions, I see one execution for each webhook received. the Duration time varies from 0.8 to 22.8 seconds. In real time, many executions are "Running" but when I refresh, they turn to "Complete". In the Sheet, only about 85% of the data is getting posted and about 20% of the data are duplicates. I have confirmed the particle webhooks are only being sent once. Occasionally, on the particle console, I get an error: ESOCKETTIMEDOUT. I believe this could also be caused on Google side.
Update: since writing this message, I have learned that ESOCKETTIMEDOUT error I'm seeing could be causing the webhook to be resent. This could cause duplication in my sheet. I have also learned that executions that take longer than 20 seconds can cause the timed out issue. But, to fix the issue is on the app script side. So, I'm even more convinced that my script is the issue.
Given these payloads are tiny (~600 bytes) and are spaced out by 5 seconds, I can't believe that the Google service can't handle that.
I think the problem is with my app script. But I don't know what to do to fix it. All help is appreciated.