ABOUT    |    MEDIA    |    CONTACT    |    CAREERS
  Stress Testing Autonomous Systems
Back | Next

Testing of RECBot with the ASTAA tool – a mature robotic platform used internally at NREC – found a speed limit violation that had never been discovered in six years of use. The ASTAA tool used RECBot’s external interface to conduct the test and did not have access to its source code or design details.

Stress Tests for Autonomy Architectures (STAA) finds autonomy system safety problems that are unlikely to be discovered by other types of tests.

Our project produces a toolkit called “Automated Stress Testing for Autonomy Architectures” (ASTAA) for testing autonomy software and other software systems with similar interfaces. It feeds potentially abnormal inputs into a system until it exhibits unsafe behavior. These inputs could include out-of-range sensor and vehicle data, inputs that cause divide by zero errors, and other bad data.

The ASTAA tool then generates a report of the problems it’s found. Tracking them down and fixing them improves the robustness and safety of autonomy systems.

The ASTAA tool can be used to test both entire systems and individual components at any point during the development process. It also supports benchmarking and regression testing, which are virtually impossible under traditional testing methods. This gives engineers and programmers a chance to address potential safety problems before overall system integration, field testing, and deployment. The ASTAA tool can uncover safety problems that are unlikely to arise during typical field testing, but may occur when the system is actually put to use.

Distribution Statement A - Approved for public release; distribution is unlimited. NAVAIR Public Affairs Office tracking number 2013-74, NREC internal case number STAA-2012-10-23

National Robotics Engineering Center    SITE MAP    |    ABOUT    |    MEDIA    |    CONTACT    |    CAREERS    © Carnegie Mellon University