This disconnect between real-fast CV and real-time safety is already having unfortunate consequences. Several highly publicized recent accidents involving semi-autonomous and fully autonomous cars on open roads have resulted in fatalities. The post-crash analyses of these incidents have revealed a critical trade-off between time and accuracy at the nexus between CV and real time. Today, CV-based features are largely provided in settings where the driver is assumed to be able to retake full control of the vehicle. As such, these features are not yet subject to strict certification requirements. Moving forward, however, we will see greater and greater autonomy in mass-market vehicles, with the line between semi-autonomy and full autonomy being crossed eventually. When this happens, strict certification will be enormously important. Certifying CV applications that are merely "real fast" will be difficult, if not impossible.
Motivated by this disconnect, this project is directed at enabling CV programmers to produce code that is amenable to real-time certification. Specifically, a real-time CV API will be produced by extending OpenVX, which is a recently ratified standard intended for embedded systems. OpenVX utilizes a graph-based formalism for specifying CV workloads that presents several challenges when validating real-time constraints. These challenges will be addressed. A library of new CV algorithms that exploit the features of the API will be created and methods developed to transform existing algorithms. Also, an experimental evaluation of "real-fast" vs. "real-time" CV will be conducted using driving simulators, sub-scale autonomous vehicles, and advanced testing infrastructure at General Motors.
While industry is pushing hard in the area of autonomous driving, autonomous vehicles will never become a common mode of transportation unless methods for certifying real-time safety are produced. This project will focus on a key aspect of certification: validating the real-time correctness of CV applications. The results that are produced will be made available to the world at large through open-source software. This software will include the new OpenVX API to be produced as well as tools for validating the real-time correctness of applications developed using this API.
T. Amert, S. Voronov, and J. Anderson " OpenVX and Real-Time Certification: The Troublesome History", Proceedings of the 40th IEEE Real-Time Systems Symposium, December 2019, to appear. PDF .
M. Yang, S. Wang, J. Bakita, T. Vu, F.D. Smith, J. Anderson, and J.-M. Frahm, " Re-thinking CNN Frameworks for Time-Sensitive Autonomous-Driving Applications: Addressing an Industrial Challenge", Proceedings of the 25th IEEE Real-Time and Embedded Technology and Applications Symposium, pp. 305-317, April 2019. PDF .
Other papers that acknowledge this grant can be found on the PI's Publications Page .
Last modified 25 October 2019