Model Card: Carrier Outage Detection

Model Name: Carrier Outage Detection Model (Record Count model; a Delay model existed in the original EU-only version but is no longer built in any region)

Goal: Detect and alert on potential carrier outages

Forecasting Algorithm: ARIMA+

Model Type: Time-series forecasting

Model version: 1.40 (stored as 1.4 in the data)

Developed By: Geotab Embedded Engineering "Data Minions" in collaboration with Data Observability

Primary Intended Uses: The primary intended use of the carrier outage detection model is to proactively identify, classify, and facilitate the resolution of carrier network issues in real time to minimize service disruptions.

Out-of-scope Uses: These include directly resolving carrier outages, root cause analysis, outage prevention, or serving as a customer-facing tool.

Target Users/User Groups: Internal teams within Geotab Embedded Engineering division, such as: Engineering Support, Reliability, Danger Zone, Kinematic, Modem

This section outlines the key aspects of the data used to develop the model.

Description on Training Data

The system relies on hourly GPS data. The dataset contains real-time device connection data, which is used to detect anomalies and identify outages.

The system's machine learning model, the Record Count Model, is built using Google's ARIMA+ and is trained on Hourly-GPS data. This data originates from BigQuery and contains real-time device connection data. The model is trained using a specified number of days to include historical data, which is set to 28 days in every region. This historical data is aggregated by Carrier, Country, State, and rounded time intervals (e.g. rounded to the closest 15 min interval).

The system continuously compares the actual data with the forecasted values. These deviations are detected as anomalies.

Data Pipeline and Preprocessing

The model works by leveraging machine learning models to perform time-series forecasting of expected device connection counts. Deviations from these forecasted "normal" values are detected as anomalies, which are then classified and quantified by severity (e.g., SEVERE, SERIOUS). Upon detection, the system is designed to automatically send notifications to stakeholders and, if thresholds are met, create appropriate Jira tickets and assign them.

The data pipeline and preprocessing steps for the carrier outage detection system involve a sequence of operations to process raw device connection data, train machine learning models, detect anomalies, and classify their severity.

Data Ingestion and Source

• The system's primary input is Hourly-GPS data. This data contains real-time device connection information which is crucial for identifying anomalies and outages.
• An initial data quality assertion is performed. This step verifies that new data is within the expected time range, ensuring the input data is current and complete for processing.

Data Preprocessing for Model Training

• Time Aggregation: Raw GPS data is aggregated into regular time intervals for model training. This involves rounding timestamps down to the nearest "binning", depending on how these bins have been defined (e.g. a bin could be defined as "every 15 minutes".)
• Data Partitioning: The models are trained on historical data which is partitioned by Carrier, Country, State, and the rounded time intervals.
• Historical Data Inclusion: A configurable number of "days to include" of historical data are incorporated for model training (28 days in production).
• Data Availability Checks: Before model creation, procedures perform checks to ensure that:
  • There are enough data points (greater than the pre-defined cutoff value).
  • The time range of the data spans a sufficient duration (for the record count model, more than 4 days of data and more than 96 data points per segment in production). These checks are recorded, but the data-sufficiency check does not currently block model creation (a known gap, tracked internally).
• Data Exclusion (for Record Count Model): When training the Record Count Model, data after 23:15 UTC on the day before the model date (that is, the 45 minutes before midnight UTC and anything later) is explicitly excluded.

Model Training and Storage

• The preprocessed data is used to train an ARIMA+ time-series forecasting model:

1. The Record Count Model
2. No Delay Model is built any more (it existed only in the original EU-only version). A new model is built every day in every region; the configured rebuild interval of two days only decides which model is preferred.

• Forecasting and Anomaly Detection:
  • The trained ARIMA+ models generate forecasts for expected device connections.
  • Anomaly Detection: This procedure aggregates actual device counts by Carrier, Country, State, and time interval.
  • Anomaly Classification: Deviations between actual and forecasted values are detected as anomalies and classified as "LOW", "HIGH", "MISSING", or "NORMAL"
• Severity Classification:
  • The "Severity Classification" procedure refines the anomaly detection process by assigning severity levels ("SEVERE", "SERIOUS", "MODERATE", "MINOR") to the identified anomalies. This process involves calculating the percentage change between actual and forecasted values.
  • It dynamically adjusts predefined thresholds for severity by considering the expected number of devices.
  • Furthermore, it calculates percentiles (20th, 50th, 80th) of the actual values for each hour of the day and day of the week, using this historical data to inform severity assessment.

Post-processing for Forecast Output

To prevent negative forecasted values, which are not logical for device counts, a zero lower-bound constraint can be added directly into the model creation scripts. Alternatively, forecasts can be post-processed by setting any negative values to zero.

Data Privacy

This model, by design, uses aggregated data to determine a carrier outage and create the appropriate notification and/or Jira ticket. The aggregated data does not include any PII. The model itself is trained and run on aggregated counts per carrier, country, state and time interval; it does not use individual device records. Outside the model, HardwareIds (internal Geotab device identifiers, not personally identifiable information on their own) appear in the CarrierInfo and CarrierOutageNotifications tables (up to 3 example devices per failure reason) and in internal-only support tickets used for troubleshooting. They are not included in alert emails. The alert-queue table has an example-devices column that is not populated. These identifiers were reviewed by the Privacy Office, which determined that they are not personally identifiable information on their own and that a Privacy Risk Assessment is not required.

Data retention: a partition expiry of 3 years (1095 days) is set on the forecast and model-metadata tables of every region and on the notification and alert-queue tables, and an expiry of 45 days on the input stream table (GPSDetailsStream). The table that links device identifiers to carrier and location (CarrierInfo) holds a single hourly snapshot and keeps no history. Trained models are kept for 14 days: the pipeline only uses a model built in the last 3 days, and older models cannot be rebuilt because the input stream keeps 45 days. About 3,100 older models were deleted in October 2026 and the remaining ones carry a 14-day expiry, and since 19 October 2026 each newly built model gets a 14-day expiry automatically when it is built (checked on 20 October in all six regions). What each model produced stays in the forecast and model-metadata tables for 3 years. The small table of created support tickets (no device identifiers) is kept for 3 years and will get a scheduled clean-up before any record reaches three years of age (not before May 2028).

A legacy dataset from the original EU-only version of the model was deleted in October 2026.

In this section, we highlight some ethical challenges that were encountered during the model development.

Risks in Training

The ARIMA+ base model performs optimally with one year of data. However, due to the ungovernable volume of data that one year in each state would represent, the time period was reduced to 28 days. Consequently, holidays must be configured manually.

Data Bias Handling

Bias in forecasting can manifest as systematic errors, failure to account for trends or seasonality, or an incorrectly specified model. ARIMA tackles these issues through a combination of differencing, autoregression, and moving averages.

Model Assumptions and Constraints

To enhance model performance, the following constraints were established based on historical data observed by the team:

• The model detects outages at a state/province level.
• The cut-offs on the mean device count are 50 in the US and EU and 30 in CA, BR, AS and AU (the median cut-off is 10 in all regions). The regional configuration also lists higher mean cut-offs for the SEVERE level (100 in the US, 600 in the EU, 60 elsewhere), but the code does not use them.
• For a potential carrier outage, if the device cut-off is met, the scaled percentage change between the actual and the median device count must fall below the severity threshold for the segment. For large US segments the thresholds are -30 (MODERATE), -90 (SERIOUS) and -120 (SEVERE) on a -200 to +200 scale, roughly -26%, -62% and -75% of the median; smaller segments use separate thresholds and holidays use stricter ones.

The model's performance relies heavily on carrier confirmation, which is not consistently provided. Some carriers confirm upon contact, while others are reluctant. This inconsistency prevents the establishment of a robust evaluation pipeline for the model.