This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Accept
Reject
Preferences
Log in to ask a question
Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.
Log in to ask a question

Strange behaviour of ARIMA model

Posted on 07-22-2022 08:30 AM
bellomimarc
Bronze 2
Bronze 2
Reply posted on --/--/---- --:-- AM

Hi guys, 

I'm working with ARIMA Model and I found a strange behaviour.

I have two dataset called 

  • anomaly_detection_poc.sample10 (the values are these 302.77562,302.0675,300.03862,297.24675,295.94825,293.798,291.0855,29.294441,40.031734,160.875 in loop every minute starting from 2022-07-01)
  • anomaly_detection_poc.sample11 (the values are these 
    302.307,302.77562,302.0675,300.03862,297.24675,295.94825,293.798,291.0855,29.294441,40.031734,160.875 in loop every minute starting from 2022-07-01)

Then I create two model in this way

sample_10_arima

CREATE OR REPLACE MODEL
`anomaly_detection_poc.sample_10_arima`
OPTIONS(MODEL_TYPE = 'ARIMA_PLUS'
    , TIME_SERIES_TIMESTAMP_COL = 'ts'
    , TIME_SERIES_DATA_COL = 'value'
    , HORIZON = 30
    , AUTO_ARIMA = TRUE 
    , AUTO_ARIMA_MAX_ORDER = 5
    , DATA_FREQUENCY = 'PER_MINUTE'
    , CLEAN_SPIKES_AND_DIPS = FALSE
    , MAX_TIME_SERIES_LENGTH = 10080
    )
AS 
SELECT *
FROM `anomaly_detection_poc.sample10`
WHERE ts <= '2022-07-03 02:00:00'
ORDER BY ts DESC
LIMIT 500;

sample_11_arima

CREATE OR REPLACE MODEL
`anomaly_detection_poc.sample_11_arima`
OPTIONS(MODEL_TYPE = 'ARIMA_PLUS'
    , TIME_SERIES_TIMESTAMP_COL = 'ts'
    , TIME_SERIES_DATA_COL = 'value'
    , HORIZON = 30
    , AUTO_ARIMA = TRUE 
    , AUTO_ARIMA_MAX_ORDER = 5
    , DATA_FREQUENCY = 'PER_MINUTE'
    , CLEAN_SPIKES_AND_DIPS = FALSE
    , MAX_TIME_SERIES_LENGTH = 10080
    )
AS 
SELECT *
FROM `anomaly_detection_poc.sample11`
WHERE ts <= '2022-07-03 02:00:00'
ORDER BY ts DESC
LIMIT 500;

Then I call the ML.FORECAST function for both in that way

SELECT
    f.forecast_timestamp,
    f.forecast_value,
    s10.value,
    f.standard_error
FROM
    ML.FORECAST(MODEL `anomaly_detection_poc.sample_10_arima`,
                STRUCT(30 AS horizon, 0.95 AS confidence_level)) f
INNER JOIN `anomaly_detection_poc.sample10` s10 on s10.ts = f.forecast_timestamp;

Result for sample_10:

[{
  "forecast_timestamp": "2022-07-03T02:01:00Z",
  "forecast_value": "306.41173285451686",
  "value": "304.609972052415",
  "standard_error": "2.8199310058400839"
}, {
  "forecast_timestamp": "2022-07-03T02:02:00Z",
  "forecast_value": "305.37788426343457",
  "value": "303.95136089787633",
  "standard_error": "2.8211256733849956"
}, {
  "forecast_timestamp": "2022-07-03T02:03:00Z",
  "forecast_value": "306.739091217752",
  "value": "303.6495107813119",
  "standard_error": "2.8223198352358345"
}, {
  "forecast_timestamp": "2022-07-03T02:04:00Z",
  "forecast_value": "300.3377701753941",
  "value": "298.6072872324707",
  "standard_error": "2.8235134920342255"
}, {
  "forecast_timestamp": "2022-07-03T02:05:00Z",
  "forecast_value": "300.10617792916611",
  "value": "294.65681486796137",
  "standard_error": "2.8247066444204409"
}, {
  "forecast_timestamp": "2022-07-03T02:06:00Z",
  "forecast_value": "298.31753620063341",
  "value": "299.76196578250261",
  "standard_error": "2.8258992930334"
}, {
  "forecast_timestamp": "2022-07-03T02:07:00Z",
  "forecast_value": "30.122356218709314",
  "value": "29.451189674897449",
  "standard_error": "2.8270914385106733"
}, {
  "forecast_timestamp": "2022-07-03T02:08:00Z",
  "forecast_value": "40.60273774219533",
  "value": "41.3004774222539",
  "standard_error": "2.82828308148849"
}, {
  "forecast_timestamp": "2022-07-03T02:09:00Z",
  "forecast_value": "165.86050548201936",
  "value": "162.47927425294355",
  "standard_error": "2.8294742226017382"
}, {
  "forecast_timestamp": "2022-07-03T02:10:00Z",
  "forecast_value": "307.674541548093",
  "value": "314.076719817158",
  "standard_error": "2.8306648624839696"
}, {
  "forecast_timestamp": "2022-07-03T02:11:00Z",
  "forecast_value": "311.2102908255867",
  "value": "305.12615161719276",
  "standard_error": "2.8318550017674053"
}, {
  "forecast_timestamp": "2022-07-03T02:12:00Z",
  "forecast_value": "306.226013674404",
  "value": "313.07125067705323",
  "standard_error": "2.833044641082938"
}, {
  "forecast_timestamp": "2022-07-03T02:13:00Z",
  "forecast_value": "300.75685873687024",
  "value": "304.717664646629",
  "standard_error": "2.8342337810601355"
}, {
  "forecast_timestamp": "2022-07-03T02:14:00Z",
  "forecast_value": "305.61417943694067",
  "value": "310.23180786201976",
  "standard_error": "2.8354224223272473"
}, {
  "forecast_timestamp": "2022-07-03T02:15:00Z",
  "forecast_value": "303.68840718603383",
  "value": "303.64800323499333",
  "standard_error": "2.8366105655112044"
}, {
  "forecast_timestamp": "2022-07-03T02:16:00Z",
  "forecast_value": "301.47419763922971",
  "value": "296.79744824800957",
  "standard_error": "2.8377982112376268"
}, {
  "forecast_timestamp": "2022-07-03T02:17:00Z",
  "forecast_value": "29.893389212037278",
  "value": "29.610424285628419",
  "standard_error": "2.8389853601308261"
}, {
  "forecast_timestamp": "2022-07-03T02:18:00Z",
  "forecast_value": "41.396011596661424",
  "value": "41.5123352698129",
  "standard_error": "2.8401720128138077"
}, {
  "forecast_timestamp": "2022-07-03T02:19:00Z",
  "forecast_value": "163.35720284915556",
  "value": "168.06883872246493",
  "standard_error": "2.8413581699082777"
}, {
  "forecast_timestamp": "2022-07-03T02:20:00Z",
  "forecast_value": "308.69299234826951",
  "value": "302.82608922667845",
  "standard_error": "2.8425438320346439"
}, {
  "forecast_timestamp": "2022-07-03T02:21:00Z",
  "forecast_value": "304.99294433140392",
  "value": "305.39582570271523",
  "standard_error": "2.8437289998120212"
}, {
  "forecast_timestamp": "2022-07-03T02:22:00Z",
  "forecast_value": "310.90293309247426",
  "value": "302.20232853605353",
  "standard_error": "2.8449136738582346"
}, {
  "forecast_timestamp": "2022-07-03T02:23:00Z",
  "forecast_value": "304.67148335427134",
  "value": "302.14484060782462",
  "standard_error": "2.8460978547898241"
}, {
  "forecast_timestamp": "2022-07-03T02:24:00Z",
  "forecast_value": "304.85940187532356",
  "value": "308.93581009174005",
  "standard_error": "2.8472815432220462"
}, {
  "forecast_timestamp": "2022-07-03T02:25:00Z",
  "forecast_value": "301.24253166854521",
  "value": "295.05089549592532",
  "standard_error": "2.8484647397688811"
}, {
  "forecast_timestamp": "2022-07-03T02:26:00Z",
  "forecast_value": "298.28522662702073",
  "value": "293.5837686256578",
  "standard_error": "2.8496474450430318"
}, {
  "forecast_timestamp": "2022-07-03T02:27:00Z",
  "forecast_value": "30.56919200039917",
  "value": "29.483629896147075",
  "standard_error": "2.8508296596559326"
}, {
  "forecast_timestamp": "2022-07-03T02:28:00Z",
  "forecast_value": "40.988491674274059",
  "value": "40.678949191269574",
  "standard_error": "2.8520113842177488"
}, {
  "forecast_timestamp": "2022-07-03T02:29:00Z",
  "forecast_value": "163.75582623522916",
  "value": "163.35355503598811",
  "standard_error": "2.853192619337384"
}, {
  "forecast_timestamp": "2022-07-03T02:30:00Z",
  "forecast_value": "313.92758659728719",
  "value": "305.91143123619014",
  "standard_error": "2.8543733656224797"
}]

Result for sample_11:

[{
  "forecast_timestamp": "2022-07-03T02:01:00Z",
  "forecast_value": "100.53693546817703",
  "value": "41.135001562854711",
  "standard_error": "60.184482472084568"
}, {
  "forecast_timestamp": "2022-07-03T02:02:00Z",
  "forecast_value": "198.78980079180036",
  "value": "161.18096707015462",
  "standard_error": "73.23520166522043"
}, {
  "forecast_timestamp": "2022-07-03T02:03:00Z",
  "forecast_value": "274.06282751495371",
  "value": "309.11862890065476",
  "standard_error": "73.965335710915454"
}, {
  "forecast_timestamp": "2022-07-03T02:04:00Z",
  "forecast_value": "314.73754129211278",
  "value": "317.59056591831012",
  "standard_error": "89.992938647114883"
}, {
  "forecast_timestamp": "2022-07-03T02:05:00Z",
  "forecast_value": "231.57017561331438",
  "value": "302.83582623865914",
  "standard_error": "101.16968833227945"
}, {
  "forecast_timestamp": "2022-07-03T02:06:00Z",
  "forecast_value": "245.46052654438029",
  "value": "309.55330946160922",
  "standard_error": "101.46381053942768"
}, {
  "forecast_timestamp": "2022-07-03T02:07:00Z",
  "forecast_value": "243.14060397122429",
  "value": "310.52346182933485",
  "standard_error": "101.47200274626169"
}, {
  "forecast_timestamp": "2022-07-03T02:08:00Z",
  "forecast_value": "243.52807011748283",
  "value": "308.20566493507033",
  "standard_error": "101.47223125532678"
}, {
  "forecast_timestamp": "2022-07-03T02:09:00Z",
  "forecast_value": "243.46335674461702",
  "value": "306.47721383658927",
  "standard_error": "101.47223762949433"
}, {
  "forecast_timestamp": "2022-07-03T02:10:00Z",
  "forecast_value": "243.47416496791112",
  "value": "304.5239238135868",
  "standard_error": "101.47223780729935"
}, {
  "forecast_timestamp": "2022-07-03T02:11:00Z",
  "forecast_value": "243.47235981256838",
  "value": "29.60521677814036",
  "standard_error": "101.47223781225917"
}, {
  "forecast_timestamp": "2022-07-03T02:12:00Z",
  "forecast_value": "243.47266130391648",
  "value": "41.807929278280589",
  "standard_error": "101.47223781239751"
}, {
  "forecast_timestamp": "2022-07-03T02:13:00Z",
  "forecast_value": "243.47261094978307",
  "value": "166.22813087686413",
  "standard_error": "101.47223781240136"
}, {
  "forecast_timestamp": "2022-07-03T02:14:00Z",
  "forecast_value": "243.4726193597715",
  "value": "311.120674277285",
  "standard_error": "101.47223781240149"
}, {
  "forecast_timestamp": "2022-07-03T02:15:00Z",
  "forecast_value": "243.47261795516178",
  "value": "314.0272359705304",
  "standard_error": "101.47223781240149"
}, {
  "forecast_timestamp": "2022-07-03T02:16:00Z",
  "forecast_value": "243.47261818975525",
  "value": "305.7107591848029",
  "standard_error": "101.47223781240149"
}, {
  "forecast_timestamp": "2022-07-03T02:17:00Z",
  "forecast_value": "243.47261815057419",
  "value": "310.24411130736274",
  "standard_error": "101.47223781240149"
}, {
  "forecast_timestamp": "2022-07-03T02:18:00Z",
  "forecast_value": "243.47261815711809",
  "value": "310.7495483682959",
  "standard_error": "101.47223781240149"
}, {
  "forecast_timestamp": "2022-07-03T02:19:00Z",
  "forecast_value": "243.47261815602516",
  "value": "309.75593252507184",
  "standard_error": "101.47223781240149"
}, {
  "forecast_timestamp": "2022-07-03T02:20:00Z",
  "forecast_value": "243.47261815620769",
  "value": "307.30120520293525",
  "standard_error": "101.47223781240149"
}, {
  "forecast_timestamp": "2022-07-03T02:21:00Z",
  "forecast_value": "243.47261815617719",
  "value": "305.22389101135269",
  "standard_error": "101.47223781240149"
}, {
  "forecast_timestamp": "2022-07-03T02:22:00Z",
  "forecast_value": "243.47261815618231",
  "value": "30.201925796580159",
  "standard_error": "101.47223781240149"
}, {
  "forecast_timestamp": "2022-07-03T02:23:00Z",
  "forecast_value": "243.47261815618145",
  "value": "41.982169328730954",
  "standard_error": "101.47223781240149"
}, {
  "forecast_timestamp": "2022-07-03T02:24:00Z",
  "forecast_value": "243.4726181561816",
  "value": "166.40265635454585",
  "standard_error": "101.47223781240149"
}, {
  "forecast_timestamp": "2022-07-03T02:25:00Z",
  "forecast_value": "243.47261815618157",
  "value": "313.89491002223463",
  "standard_error": "101.47223781240149"
}, {
  "forecast_timestamp": "2022-07-03T02:26:00Z",
  "forecast_value": "243.47261815618157",
  "value": "304.88350360153481",
  "standard_error": "101.47223781240149"
}, {
  "forecast_timestamp": "2022-07-03T02:27:00Z",
  "forecast_value": "243.47261815618157",
  "value": "312.68737586889137",
  "standard_error": "101.47223781240149"
}, {
  "forecast_timestamp": "2022-07-03T02:28:00Z",
  "forecast_value": "243.47261815618157",
  "value": "306.59293265266768",
  "standard_error": "101.47223781240149"
}, {
  "forecast_timestamp": "2022-07-03T02:29:00Z",
  "forecast_value": "243.47261815618157",
  "value": "302.44991069056488",
  "standard_error": "101.47223781240149"
}, {
  "forecast_timestamp": "2022-07-03T02:30:00Z",
  "forecast_value": "243.47261815618157",
  "value": "306.66045835707689",
  "standard_error": "101.47223781240149"
}]

In the first case sample_10_arima the standard_error is low (around 2.8) but in the sample_11_arima the standard_error is high (between 60 and 101). Why this difference occour? The time series are very similar

Thanks, 

Marcello

0 1 225
Topic Labels
0 Likes
1 REPLY 1

Posted on --/--/---- --:-- AM
josegutierrez
Staff
Reply posted on --/--/---- --:-- AM

Can you please file an issue  at issue tracker according to this behavior shown?

Top Solution Authors
View all