Model Card: ADAS

Model name: Advanced Driver Assistance System (ADAS)

Goal: To automatically recognize traffic rule violations, aggressive driving, dangerous situations on the road, near-collisions, and collisions, provide real-time voice alerts to the driver when required, and give an overall evaluation of the driver's driving patterns based on collected data.

Developed by: Geotab Video Team

Primary Intended Uses:

• Automatically recognize traffic rule violations, aggressive driving, dangerous situations on the road, near-collisions, and collisions.
• Evaluate the driver's overall driving patterns based on collected data.
• Provide real-time voice alerts to the driver.

Out-of-scope Uses:

• Replace human driver judgment or control.
• Directly control vehicle functions (e.g., braking, steering).
• Be used as the sole basis for legal or punitive actions without human review.

Target Users/User Groups:

• Fleet managers monitoring driver behavior and ensuring compliance.
• Insurance companies interested in driver risk assessment.
• Researchers studying driving patterns and road safety.

This section outlines the key aspects of the data used to develop the model. First, we describe the data and its features. Then, the pipeline and its specifics are explained.

Description on Data and Features

• Object Detection Data: Bounding box annotations for various road objects (e.g., vehicles, pedestrians, traffic lights, signs) in diverse lighting and weather conditions.
• Lane Detection Data: Pixel-level annotations for lane lines (e.g., solid, dashed, double solid) and road boundaries.
• Lane Classification Data: Categorical labels for different lane types (e.g., straight, turn, merge lanes).
• Sign Classification Data: Categorical labels for various traffic signs (e.g., speed limits, stop signs, warning signs).
• Traffic Light Classification Data: Categorical labels for various traffic lights (e.g., green, yellow, red, off).
• Driving Behavior Data: Accelerometer, gyroscope, and GPS data (including lat/long coordinates) used for analyzing aggressive driving maneuvers (e.g., harsh braking, rapid acceleration, sharp turns) and as direct model inputs for GPS-dependent features: speed estimation, vehicle complete stop estimation, map-based low bridge/clearance detection, and trajectory-based triggers (e.g., rolling stops, turn violations). Lat/long coordinates are also stored as event metadata at the nearest point to each trigger event.
• Event Data: Annotated instances of traffic rule violations, dangerous situations, near-collisions, and collisions.
• Obstruction Classification Data: Labeled images of camera obstructions, miscalibration cases, and insufficient road visibility frames.
• Trajectory Data: Vehicle motion data (speed, heading, GPS trace, turn radius, azimuth) for ego-vehicle trajectory recognition.
• Weather Context Data: Dashcam frames labeled for visibility conditions (fog, rain, snow, clear), road conditions (dry, wet, snowy), and road material (asphalt, potholes, rural).
• Calibration Data: Camera mounting geometry and field-of-view reference data for road camera calibration.
• Clearance Check Data: Map index data and sensor readings for low-bridge and clearance detection.
• License Plate Detection Data: Annotated images of license plates in various lighting, angle, and occlusion conditions.

Data Pipeline and Preprocessing

Data Ingestion and Source

Data is primarily ingested from vehicle dashcams, which capture continuous video footage and associated sensor data (GO device, CAN, GPS, accelerometer, gyroscope). This data is then transferred to a secure cloud storage environment.

Data Preprocessing for Model Training

• Video Frame Extraction: Video streams are broken down into individual frames at a specified rate.
• Image Normalization: Frame pixel values are scaled and normalized to a consistent range.
• Resizing and Cropping: Images are resized to match the input requirements of the respective models. Relevant regions of interest might be cropped to focus on specific features.
• Data Augmentation: Techniques like rotation, flipping, brightness adjustments, and adding noise are applied to increase the diversity of the training data and improve model generalization.
• Annotation Alignment: Video frames are synchronized with corresponding sensor data and event annotations.
• Labeling and Annotation: Manual and semi-automated annotation tools are used to label objects, lanes, signs, and driving events within the video frames.
• Data Splitting: The preprocessed and annotated dataset is split into training, validation, and test sets to ensure robust model evaluation.

Model Training and Storage

• The underlying models (Object Detection, Lane Detection, Lane Classification, Sign Classification, Traffic Light Classification, Temporal Inferencing, Road Camera Calibration, Low Bridge/Clearance Check, License Plate Detection, Road Camera Obstruction Classification, Vehicle Trajectory Recognition, and Weather Context Classifier) are trained independently using their respective preprocessed datasets.
• Training is conducted on high-performance computing infrastructure (e.g., GPUs).
• Trained model weights and configurations are version-controlled and stored in a secure model repository.
• Training Data Residency: Data selected for model training is stored in one of two designated training environments rather than the customer's standard regional storage: training data originating from US customers is stored in the US, and training data from all other regions is stored in the EU.
• No PII in Model Outputs: The output of the training process consists only of optimized model weights, not personal data. Individual information is not embedded in, identifiable within, or recoverable from the resulting models.

Post-processing for Forecast Output

• Fusion of Model Outputs: The outputs from the individual models are combined and integrated to form a comprehensive understanding of the driving scenario.
• Rule-Based Logic: Custom rule-based logic is applied to interpret the combined model outputs and identify specific traffic rule violations, aggressive driving, dangerous situations, near-collisions, and collisions.
• Alert Generation: Based on identified events, appropriate real-time voice alerts are generated.
• Driving Pattern Evaluation: Aggregated data on identified events and driving behaviors is used to generate an overall evaluation of the driver's patterns.
• Data Serialization: Processed outputs, including event logs and driving evaluations, are serialized into a format suitable for storage and further analysis.

Data Privacy

The ADAS system runs on the Geotab Video Platform and inherits that platform's privacy-by-design architecture and Data Controller / Data Processor governance model.

• Data Controller / Data Processor Model: The customer (fleet operator) is the Data Controller — they own the video and sensor data and determine the purpose and means of processing, including which events trigger a cloud upload, how long data is retained, and how any required consent is obtained. Geotab acts as the Data Processor, providing the secure technical infrastructure and processing data solely on the Controller's instructions.
• Controller-Configured Recording & Minimized Transmission: Video and sensor data are captured on the dashcam. Depending on Controller-defined settings, video is either recorded locally with cloud upload occurring on request or triggered by a predefined safety exception ("Continuous Recording"), or automatically uploaded when a specific ADAS event occurs, such as a traffic violation or near-collision ("Event Recording"). Transmission to the cloud is limited to Controller-configured events or requests, not a continuous, unconditional stream.
• AI-Driven Object Blurring: To support Controller privacy obligations, the platform can apply AI-based blurring of personally identifiable visual elements on the road-facing view, configurable in MyGeotab: license plates and pedestrian faces. Where a device also has a cabin-facing view (e.g., paired with a Driver Monitoring System), driver and passenger faces can similarly be blurred.
• Anonymization of Derived Metadata: Behavioral and event metadata generated by the ADAS models (e.g., "harsh braking," "traffic sign violation," object/lane detections) are anonymized and contain no personally identifiable information; they cannot on their own be linked back to an individual.
• Encryption: Video and sensor data are encrypted using AES-256 at rest, both on the camera's local storage and in Geotab's cloud storage, and using TLS 1.3 in transit, with mutual authentication ensuring only verified cameras can communicate with the platform.
• Data Residency: Standard video and event data are stored in data centers corresponding to the customer's region (e.g., US data in the US, EU data in the EU). Data selected for model training follows a separate residency rule: training data originating from US customers is stored in the US, and training data from all other regions is stored in the EU.
• Consent Management: Because the customer is the Data Controller, obtaining any explicit and informed consent required from drivers (e.g., for event-triggered video clip capture) is the customer's responsibility. Geotab provides the technical controls (recording settings, blurring, retention, access management) that support the Controller in meeting this obligation, but Geotab does not itself collect driver consent.
• Secure Storage & Access Control: Raw video is held in access-controlled, audit-logged storage, viewable only by the Controller's authorized MyGeotab users (via role-based security clearances such as View Live Video / View Recorded Video) and a small, restricted set of Geotab personnel. Company Groups further silo access so users only see footage for drivers and vehicles within their assigned groups.
• Retention & Deletion: When a MyGeotab database or driver is removed, an automated process deletes the associated video and derived data in accordance with Geotab's applicable data-retention and deletion policies.
• Purpose Limitation: Data collected for the ADAS system is used strictly for its stated purpose — traffic-rule-violation detection, driving-pattern evaluation, and real-time alerts as configured by the Controller — and not for unrelated activities without the Controller obtaining separate, purpose-specific consent.
• Compliance: The system adheres to relevant data protection regulations and privacy laws (e.g., GDPR, CCPA) through the technical and organizational measures above, supported by regular security and privacy audits.

Risks in Training

The development and training of the ADAS model, especially given its reliance on real-world driving data, present several risks:

• Data Quality and Annotation Errors:
  • Risk: Inaccurate, incomplete, or inconsistent annotations (e.g., mislabeled objects, incorrect lane markings, missing event triggers) in the training dataset can lead to the model learning incorrect patterns, resulting in poor performance, false positives, or missed detections in real-world scenarios.
  • Mitigation: Implement rigorous quality control processes for data annotation, including multiple annotator reviews, consensus mechanisms, and regular audits. Utilize semi-automated tools to aid annotators and reduce human error.
• Data Scarcity for Rare Events:
  • Risk: Dangerous situations, near-collisions, and actual collisions are statistically rare. A lack of sufficient diverse examples of these critical events in the training data can lead to the model being ineffective at recognizing them, potentially failing when it's most crucial.
  • Mitigation: Employ data augmentation techniques specifically tailored for rare events. Explore synthetic data generation for hard-to-capture scenarios. Use near-collision events as proxy labels. Prioritize the collection of data from diverse accident scenarios (where ethically and practically feasible). Leverage transfer learning from larger, more general datasets.
• Overfitting to Training Data:
  • Risk: The model might learn the training data too well, including its noise and specific characteristics, leading to excellent performance on the training set but poor generalization to unseen real-world data, especially in varied driving conditions or environments.
  • Mitigation: Utilize regularization techniques (e.g., L1/L2 regularization, dropout). Implement early stopping during training based on validation set performance. Ensure a diverse and representative validation set.
• Underfitting:
  • Risk: The model might be too simple to capture the complexity of the underlying patterns in the data, leading to consistently poor performance on both training and unseen data.
  • Mitigation: Experiment with more complex model architectures. Increase training time and adjust learning rates. Ensure sufficient and relevant features are provided.
• Bias in Training Data:
  • Risk: If the training data disproportionately represents certain demographics, driving conditions, geographic locations, or vehicle types, the model's performance may be skewed or biased against underrepresented groups or situations. For example, performance might degrade significantly in different lighting, weather, or road infrastructure not adequately represented.
  • Mitigation: Conduct thorough data audits to identify and quantify potential biases. Actively collect diverse data across various conditions, regions, and demographics. Implement fairness metrics during evaluation. Consider data re-sampling or re-weighting techniques to balance skewed distributions.
• Model Brittleness to Adversarial Attacks:
  • Risk: Deep learning models can be susceptible to adversarial attacks, where subtle, imperceptible perturbations to input data (e.g., a sticker on a stop sign) can cause the model to misclassify or fail, potentially leading to dangerous real-world outcomes.
  • Mitigation: Implement adversarial training techniques. Explore robust model architectures. Continuously monitor model performance in deployment for unusual or unexpected behaviors.

Data Bias Handling

To address potential biases in the training data and ensure fair and robust model performance, the following strategies are employed:

• Comprehensive Data Collection Strategy:
  • Diversity in Driving Conditions: Actively collect data from a wide range of driving conditions, including varying weather (sunny, rainy, snowy, foggy), lighting (day, night, dusk, dawn), road types (urban, rural, highway), and traffic densities.
  • Geographic and Demographic Representation: Ensure data is collected from diverse geographic regions and represents a broad spectrum of driver demographics and vehicle types to prevent skewing performance towards specific groups or environments.
  • Inclusion of Edge Cases and Rare Events: Deliberately seek out and include data representing unusual or challenging scenarios that might otherwise be underrepresented, such as unusual road layouts, unique traffic signs, or complex multi-object interactions.
• Proactive Bias Identification and Measurement:
  • Data Audits and Profiling: Regularly audit the training dataset to identify potential biases in feature distributions (e.g., disproportionate representation of certain object types, weather conditions, or vehicle models).
  • Bias Metrics: Employ quantitative metrics to measure and track biases, such as fairness metrics for classification tasks (e.g., equalized odds, demographic parity) across different protected attributes (if applicable and anonymized).
  • Visual Inspection and Exploratory Data Analysis: Conduct thorough visual inspections of annotated data and perform exploratory data analysis to uncover subtle biases that might not be evident through quantitative metrics alone.
• Bias Mitigation Techniques During Data Preprocessing and Training:
  • Resampling and Reweighting: Use techniques like oversampling minority classes or undersampling majority classes to balance the dataset. Alternatively, assign higher weights to underrepresented samples during model training.
  • Data Augmentation for Underrepresented Scenarios: Apply specific data augmentation strategies to artificially increase the representation of underrepresented conditions or objects, creating synthetic variations of existing rare samples.
  • Adversarial Debiasing: Explore advanced techniques where an adversarial network attempts to predict a protected attribute from the model's learned representations, and the main model is trained to minimize this predictability, thus encouraging more fair representations.
  • Regularization: Apply regularization techniques to prevent the model from becoming overly specialized to the dominant patterns in the training data, promoting better generalization across diverse inputs.
• Fairness in Model Evaluation:
  • Segmented Performance Evaluation: Evaluate model performance (accuracy, precision, recall, F1-score) across different subgroups defined by relevant attributes (e.g., performance in different weather conditions, lighting, or for different vehicle types).
  • Error Analysis: Conduct detailed error analysis to understand why the model might be failing in certain scenarios or for particular groups, leading to targeted improvements.
  • Disparity Metrics: Monitor for disparities in false positive and false negative rates across different groups or conditions to ensure that the model does not disproportionately impact certain users or scenarios.
• Human Oversight and Feedback Loops:
  • Continuous Monitoring in Deployment: Implement systems for continuous monitoring of model performance in real-world deployment, specifically looking for signs of bias or performance degradation in specific contexts.
  • User Feedback Integration: Establish mechanisms for collecting user feedback regarding model performance, especially concerning perceived biases or failures in specific situations, and use this feedback to inform future data collection and model retraining.
  • Human-in-the-Loop Review: For critical decisions or edge cases, maintain a human-in-the-loop review process to override or validate model outputs, especially where safety is paramount.

Model Assumptions and Constraints

Assumptions:

• Consistent Dashcam Mounting and Field of View: The models assume a relatively consistent mounting position and orientation of the dashcam, providing a stable field of view of the road ahead. Significant deviations in camera placement or extreme vibrations could impact performance.
• Adequate Lighting and Visibility: While the models are trained on diverse conditions, they assume a minimum level of lighting and visibility. Performance may degrade significantly in extreme low-light conditions (e.g., complete darkness without external light sources), heavy fog, blinding rain, or severe snowstorms where visual information is severely obscured.
• Standard Road Environments: The models primarily assume operation on standard paved roads with discernible lane markings and recognizable traffic signs, as typically found in urban, suburban, and highway environments. Performance might vary on unpaved roads, highly irregular road surfaces, or in areas with non-standard road infrastructure.
• Recognizable Traffic Laws and Signage: The system assumes that traffic laws and road signage largely conform to common international or regional standards represented in the training data. Performance may be affected in regions with highly unique or unfamiliar signage.
• Sufficient Data Quality and Resolution: The input video streams from the dashcam are assumed to have sufficient resolution and frame rate to allow for accurate object detection, lane detection, and sign classification.
• Predictable Vehicle Dynamics (for aggressive driving): The algorithms for aggressive driving detection assume that vehicle movements (acceleration, braking, turns) generally follow physical laws and can be interpreted from sensor data. Anomalous sensor readings or highly erratic vehicle behavior outside of typical driving patterns might be misinterpreted.
• Operational Dashcam and Connected Sensors: The system assumes the dashcam and all integrated sensors (GPS, accelerometer, gyroscope) are fully functional and providing accurate, real-time data.

Constraints:

• Occlusion and View Obstruction: The models' performance is significantly constrained by occlusions (e.g., objects hidden by other vehicles, trees, or blind spots) and obstructions to the dashcam's view (e.g., dirt, glare on the windshield, dashcam malfunction). The system cannot "see" what is hidden.
• Ambiguity in Visual Cues: In certain ambiguous situations (e.g., faded lane markings, damaged signs, complex intersections with multiple conflicting signals), the model's ability to accurately interpret the scene may be constrained.
• Limited Understanding of Human Intent/Context: The ADAS focuses on observable driving patterns and traffic rules. It cannot fully infer human intent, driver fatigue, or other complex cognitive states that contribute to dangerous driving, beyond what's evident in driving behavior and sensor data.
• Not a Replacement for Human Attention: The system is a supplementary tool and cannot fully compensate for inattentive, impaired, or reckless driving. It serves as an alert system, not an autonomous driving system.
• No Direct Vehicle Control: The ADAS is designed solely for monitoring and alerting. It does not have the capability to take direct control of the vehicle (e.g., braking, steering, accelerating), which means the driver is ultimately responsible for vehicle operation.
• Reliance on GPS Accuracy: Several model features rely on GPS lat/long data — including speed estimation, vehicle complete stop estimation, low bridge/clearance detection (map-based index lookup), and trajectory-based trigger logic (rolling stops, turn violations). These features are constrained by the accuracy and availability of GPS signals, particularly in urban canyons or tunnels. Lat/long is also stored as event metadata at the nearest point to each trigger event.

The performance of the ADAS system and its underlying models is evaluated using a combination of standard machine learning metrics and domain-specific KPIs to ensure comprehensive assessment.

For Core Vision Models (Object Detection, Lane Detection, Sign Classification):

• Precision: The proportion of positive identifications that were actually correct.
• Recall (Sensitivity): The proportion of actual positives that were correctly identified.
• F1-Score: The harmonic mean of precision and recall, providing a single metric that balances both.
• Mean Average Precision (mAP): A common metric for object detection, representing the average precision over all classes and different Intersection over Union (IoU) thresholds.
• IoU (Intersection over Union): For detection and segmentation, measures the overlap between the predicted bounding box/segmentation mask and the ground truth.
• Accuracy: For classification tasks, the proportion of correctly classified instances.

For Event Recognition (Traffic Violations, Aggressive Driving, Dangerous Situations, Near-collisions, Collisions):

• Detection Rate/Recall: The percentage of actual events that the system correctly identifies and flags.
• False Positive Rate (FPR): The rate at which the system incorrectly flags a non-event as an event (e.g., false alarm for a collision). This is critical for user acceptance and alert fatigue.
• False Negative Rate (FNR) / Miss Rate: The rate at which the system fails to detect an actual event (e.g., missing a near-collision). This is critical for safety.
• Latency: The time taken from an event occurring to the system generating an alert. Crucial for real-time applications.

For Driver Pattern Evaluation:

• Consistency/Reliability: How consistently the system evaluates similar driving behaviors over time and across different conditions.
• Correlation with Expert Assessments: Comparison of the system's overall driver evaluation with assessments from human driving experts or established safety scores.
• Actionable Insights: Qualitative assessment of whether the generated driving patterns and evaluations provide useful and actionable feedback for improving driver behavior.

Overall System KPIs:

• System Uptime and Stability: Measures the reliability and availability of the deployed system in the vehicle.
• Resource Utilization: Monitoring CPU, memory, and power consumption on the dashcam device to ensure efficient operation.
• User Acceptance and Satisfaction: Often measured through surveys or feedback to understand the perceived usefulness and intrusiveness of the alerts and evaluations.
• Reduced Incident Rate (Long-term): The ultimate measure of success, demonstrating a reduction in actual traffic violations, aggressive driving incidents, and collisions for drivers using the ADAS system over time. (This is a long-term, post-deployment metric).