Autonomous Vehicles and Software Safety Engineering

While the emphasis for autonomous vehicle (AV) design has long been focused on getting the technology to work well in everyday situations, safety remains a significant hurdle for widespread deployment – but achieving safety for life-critical systems requires much more work. While safety encompasses correct operation for the mundane, it also requires special attention to mitigating the risk presented by rare but high-consequence potential-loss events.

This talk will cover some history of AV development and safety at the Carnegie Mellon National Robotics Engineering Center that has led to the development of the ANSI/UL 4600 standard for autonomous vehicle safety. Also covered will be activities specific to safety engineering, why a heavy tail distribution of rare events makes ensuring safety so difficult, why brute force road testing won’t ensure safety, and the emergence of safety assurance cases as the approach of choice for AV safety.

Prof. Philip Koopman is an internationally recognized expert on Autonomous Vehicle (AV) safety whose work in that area spans over 25 years. He is also actively involved with AV policy and standards, as well as more general embedded system design and software quality. Dr. Koopman’s pioneering research work includes software robustness testing and runtime monitoring of autonomous systems to identify how they break and how to fix them. He also has extensive experience in software safety and software quality across numerous transportation, industrial, and defense application domains including conventional automotive software and hardware systems.

Phil was the principal technical contributor to the UL 4600 standard for autonomous system safety issued in 2020. He is a faculty member of the Carnegie Mellon University ECE department where he teaches software skills for mission-critical systems. In 2018, Phil was awarded the prestigious IEEE-SSIT Carl Barus Award for outstanding service in the public interest for his work in promoting automotive computer-based system safety. He is an IEEE Senior Member, ACM Senior Member, and SAE Member.