Deepen AI Introduces Physical AI Safety Assessment Program (PASAP) to Support AV Evaluation with NVIDIA Halos SAFE

BERLIN, DE — October 21, 2026 — Deepen AI, a provider of data infrastructure for Physical AI, today introduced the Physical AI Safety Assessment Program (PASAP), bringing data characterization and safety benchmarks to autonomous vehicle evaluation workflows using NVIDIA Halos SAFE (Safety Assurance Framework for Evaluation).

PASAP helps developers assess the quality and coverage of their driving data, evaluate model behavior across challenging scenarios and trace results to specific safety requirements. Its initial driving benchmarks draw on Safety Pool™ scenarios, the library Deepen AI co-developed with WMG at the University of Warwick, UK.

NVIDIA Halos is a full-stack safety system for building, evaluating and deploying safer autonomous vehicles and robots. Within this system, Halos SAFE supports repeatable AV safety evaluation and assessment of how closely simulation matches real-world testing.

PASAP contributes data characterization, safety benchmarks, Safety Pool™ scenarios, and configurable evaluators to these workflows. Developers can compare model versions under consistent test conditions, identify coverage gaps and evaluate performance against their own safety requirements. The resulting evidence supports safety assessments and helps teams prioritize further testing and model improvements.

Data Characterization

Autonomous driving policy and its associated safety metrics are only as valid, reliable, and accurate as the data used to train, test and measure performance. A calibration error can misplace LiDAR points by about 50 cm at a range of 35 m[3], and label errors in widely used public benchmarks and datasets average at least 3.3%, enough to change which model ranks and drastically skew safety performance[4]. PASAP characterizes four areas, each traceable to the source dataset:

• Calibration Integrity scores multi-sensor extrinsic calibration without ground-truth parameters and flags drift across sessions.
• Label Quality measures precision and recall by object class and distance for human labels and autolabels, reported in ASAM OpenLABEL, the standard Deepen AI helped author.
• ODD Coverage compares collected data against the target environment and operating conditions for the system known as Operational Design Domain (ODD), using the ISO 34503 taxonomy Deepen AI co-authored, and records gaps, provenance and privacy processing.
• Real-world Data Sourcing fills those gaps through Deepen Refinery partner fleets in six geographies, delivering data already characterized.

Safety Benchmarks and Scenarios

PASAP evaluates driving policies, including vision-language-action (VLA) world models and foundation models, in closed-loop simulation scenarios, where the model's own decisions change how each scenario unfolds. NVIDIA Halos SAFE's technical brief calls for closed-loop simulation whenever the system's actions influence surrounding traffic[5]. Following ISO 34502, each scenario is defined once with parameter ranges for speeds, gaps and timing, to generate varied perturbations.

Each closed-loop simulation scenario run is scored by modular evaluators with a standard interface, so developers can plug in their own success criteria on top of the base safety metrics. PASAP safety evaluations include:

• Time-to-collision (TTC) and near misses: an industry-wide standard safety measure, with 1.5 seconds set as a common safe operating threshold[6].
• Collision avoidance: compared with a reference model of an attentive, unimpaired human driver[7].
• Rule and regulatory compliance: for example, UN Regulation 157 lane-keeping requirements.
• Simulation realism: consistency of reported metrics and driving behaviour between simulated scenarios and real-world driving[8, 9].

Flagged events go to expert review. Each run is stored as a verified record with two use cases:

• PASAP Benchmarks apply the same scenarios and evaluators to every model, so scores are directly comparable across models and versions.
• Evaluations with NVIDIA Halos SAFE use the evaluators that match a developer's own safety requirements—for example, no more than one false emergency brake per 10,000 hours—and test the results against those targets.

Failures feed denser scenario sampling and corrective data for post-training.

Figure 1: PASAP runs scenarios through closed-loop simulation built with NVIDIA technology, scores each run with a library of evaluators, and turns the results into benchmark scores, safety case evidence and targeted model improvement.

“A safety case is only as credible as the data and evaluations behind it,” said Mohammad Musa, Founder and CEO of Deepen AI. “PASAP provides developers and regulators with a standardized framework to generate verifiable safety evidence for NVIDIA Halos SAFE, enabling clear, apples-to-apples performance comparisons across autonomous systems.”

“Safety of AI-powered autonomous vehicles should be evidenced and measured objectively. While autonomous vehicles are getting mainstream, a global safety benchmark still evades the industry beyond philosophical aspirations. Through our partnership with Deepen AI and NVIDIA, leveraging the Safety Pool™ platform, we are addressing this gap in our ecosystem.” — Professor Siddartha Khastgir, Head of Safe Autonomy, WMG at the University of Warwick, UK.

“Developers need to understand both how an autonomous vehicle performs and how credible the evidence is,” said Rishi Dhall, vice president of automotive at NVIDIA. “By bringing data characterization, safety benchmarks and Safety Pool™ scenarios to NVIDIA Halos SAFE workflows, Deepen AI helps developers expand evaluation coverage and generate traceable evidence to support their safety cases.”

PASAP will extend the same approach to robotics, assessing how robot policies, VLA models and world foundation models perceive, act and recover from failures. Details are at benchmark.deepen.ai.

About Deepen AI

Founded in 2017 in Sunnyvale, California, Deepen AI provides data infrastructure for Physical AI, from multi-sensor calibration and annotation to data curation and real-world data through Deepen Refinery, to more than 100 automakers, suppliers, autonomous vehicle developers and robotics companies. Deepen is an author of ASAM OpenLABEL, a contributor to ASAM OpenSCENARIO and co-developed Safety Pool™ with WMG at the University of Warwick. Learn more at deepen.ai.

Media Contact: info@deepen.ai

References

1. N. Kalra and S. M. Paddock, Driving to Safety: How Many Miles of Driving Would It Take to Demonstrate Autonomous Vehicle Reliability?, RAND Corporation, 2016. rand.org
2. Wayve, GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Driving, 2025. wayve.ai
3. J. Jiao et al., MLOD: Awareness of Extrinsic Perturbation in Multi-LiDAR 3D Object Detection for Autonomous Driving, IROS 2020. arxiv.org/abs/2010.11702
4. C. G. Northcutt, A. Athalye and J. Mueller, Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks, NeurIPS 2021. arxiv.org/abs/2103.14749
5. NVIDIA, NVIDIA Halos SAFE, technical brief, 2026. NVIDIA technical brief
6. U.S. Federal Highway Administration, Surrogate Safety Assessment Model and Validation: Final Report, FHWA-HRT-08-051, 2008. fhwa.dot.gov
7. K. D. Kusano et al., Collision Avoidance Testing of the Waymo Automated Driving System, Waymo, 2022. arxiv.org/abs/2212.08148
8. N. Montali et al., The Waymo Open Sim Agents Challenge, NeurIPS 2023. NeurIPS proceedings
9. Z. Yang et al., UniSim: A Neural Closed-Loop Sensor Simulator, CVPR 2023. waabi.ai